Shallow Thoughts : : May

Punctuation Reveals Truth in advertising

This ad appeared in one of the free Santa Fe weeklies. It's got to be one of the funniest mis-uses of quotes I've seen.

Does she not know that putting quotes around something means that you're quoting someone, you're introducing an unfamiliar word or phrase, or you're trying to draw attention to the quoted phrase and cast doubt on it or make fun of it? That third use, by the way, is called scare quotes. Like you'd see in a phrase like this:

One expects lawyers to have a good command of English, and to pay attention to detail, so ... what should we think?

"Injured" isn't an unfamiliar word, so it has to be either the first or third use. And whether she's soliciting clients who only say they're injured, or she's casting doubt on the injury, it's hard not to read this as an offer to help people pretend to be injured to collect a payout.

Which I'm sure happens all the time ... but I don't think I've previously seen an ad implying it so strongly.

Tags: humor, punctuation
[ 13:32 May 30, 2014    More humor | permalink to this entry | ]

Raspberry Pi Motion Camera: Part 2, using gphoto2

I wrote recently about the hardware involved in my Raspberry Pi motion-detecting wildlife camera. Here are some more details.

The motion detection software

I started with the simple and clever motion-detection algorithm posted by "brainflakes" in a Raspberry Pi forum. It reads a camera image into a PIL (Python Imaging Library) Image object, then compares bytes inside that Image's buffer to see how many pixels have changed, and by how much. It allows for monitoring only a test region instead of the whole image, and can even create a debug image showing which pixels have changed. A perfect starting point.

Camera support

As part of the PiDoorbell project, I had already written a camera wrapper that could control either a USB webcam or the pi camera module, if it was installed. Initially that plugged right in.

But I was unhappy with the Pi camera's images -- it can't focus closer than five feet (though a commenter to my previous article pointed out that it's possible to break the seal on the lens and refocus it manually. Without refocusing, the wide-angle lens means that a bird five feet away is pretty small, and even when you get something in focus the images aren't very sharp. And a web search for USB webcams with good optical quality was unhelpful -- the few people who care about webcam image quality seem to care mostly about getting the widest-angle lens possible, the exact opposite of what I wanted for wildlife.

Was there any way I could hook up a real camera, and drive it from the Pi over USB as though it were a webcam? The answer turned out to be gphoto2.

But only a small subset of cameras are controllable over USB with gphoto2. (I think that's because the cameras don't allow control, not because gphoto doesn't support them.) That set didn't include any of the point-and-shoot cameras we had in the house; and while my Rebel DSLR might be USB controllable, I'm not comfortable about leaving it out in the backyard day and night.

With gphoto2's camera compatibility list in one tab and ebay in another, I looked for a camera that was available, cheap (since I didn't know if this was going to work at all), and controllable. I ordered a used Canon A520.

As I waited for it to arrive, I fiddled with my USB-or-pi-camera to make a start at adding gphoto2 support. I ended up refactoring the code quite a bit to make it easy to add new types of cameras besides the three it supports now -- pi, USB webcam, and gphoto2. I called the module pycamera.

Using gphoto2

When the camera arrived, I spent quite a while fiddling with gphoto2 learning how to capture images. That turns out to be a bit tricky -- there's no documentation on the various options, apparently because the options may be different for every camera, so you have to run

$ gphoto2 --set-config capture=1 --list-config

to get a list of options the camera supports, and then, for each of those options, run

$ gphoto2 --get-config name [option]

to see what values that option can take.

Dual-camera option

Once I got everything working, the speed and shutter noise of capturing made me wonder if I should worry about the lifespan of the Canon if I used it to capture snapshots every 15 seconds or so, day and night.

Since I still had the Pi cam hooked up, I fiddled the code so that I could use the pi cam to take the test images used to detect motion, and save the real camera for the high-resolution photos when something actually changes. Saves wear on the more expensive camera, and it's certainly a lot quieter that way.

Uploading

To get the images off the Pi to where other computers can see them, I use sshfs to mount a filesystem from another machine on our local net.

Unfortunately, sshfs on the pi doesn't work quite right. Apparently it uses out-of-date libraries (and gives a warning to that effect). You have to be root to use it at all, unlike newer versions of sshfs, and then, regardless of the permissions of the remote filesystem or where you mount it locally, you can only access the mounted filesystem as root.

Fortunately I normally run the motion detector as root anyway, because the picamera Python module requires it, and I've just gotten in the habit of using it even when I'm not using python-picamera. But if you wanted to run as non-root, you'd probably have to use NFS or some other remote filesystem protocol. Or find a newer version of sshfs.

Testing the gphoto setup

For reference, here's an image using the previous version of the setup, with the Raspberry Pi camera module. Click on the image to see a crop of the full-resolution image in daylight -- basically the best the camera can do. Definitely not what I was hoping for.

So I eagerly set up the tripod and hooked up the setup with the Canon. I had a few glitches in trying to test it. First, no birds; then later I discovered Dave had stolen my extension cord, but I didn't discover that until after the camera's batteries needed recharging.

A new extension cord and an external power supply for the camera, and I was back in business the next day.

And the results were worth it. As you can see here, using a real camera does make a huge difference. I used a zoom setting of 6 (it goes to 12). Again, click on the image to see a crop of the full-resolution photo.

In the end, I probably will order one of the No-IR Raspberry pi cameras, just to have an easy way of seeing what sorts of critters visit us at night. But for daylight shots, an external camera is clearly the way to go.

The scripts

The current version of the script is motion_detect.py and of course it needs my pycamera module. And here's documentation for the motion detection camera.

Tags: crittercam, hardware, raspberry pi, photography, maker
[ 20:09 May 24, 2014    More hardware | permalink to this entry | ]

Comcast contractors showed up! but wouldn't do the actual job

There's a new wrinkle in our ongoing Comcast Odyssey.

I was getting ready to be picked up to carpool to a meeting when the doorbell rang. It was Comcast's contractor!

Dave and I went out and showed them the flags marking the route the cable was supposed to take. They nodded, then asked, in broken English, "Okay to dig under driveway?"

"Whaa-aa?" we said? "The cable goes from there" (indicating the box across the street) "to here" (indicating the line of flags across from the box, same side of the driveway.

They went over and pointed to the box on our side of the street, on the neighbor's property -- the box the Comcast installer had explicitly told us could in no way be used for our cable service. No, we don't know why, we told them, but every Comcast person who's been here has been insistent that we can't use that box, we have to use the one across the street.

We pointed to the painted lines on the street, the ones that have been there for a month or more, the ones that the county people left after inspecting the area and determining it safe to dig. We point out that digging across the street is the reason they had to get a traffic permit. We tell them that the cable under the driveway is why the cable was torn up in the first place, and that we're expecting to have our driveway graded some time soon, so they put a new cable there, it will probably just get torn up again. Not that any of that matters since Comcast says we can't use that box anyway.

They look at us blankly and say "We dig across driveway?"

My ride arrives. I have to leave. Dave tries for another five or ten minutes, but he has to leave too. So he finally gives up, tells them no, don't put the cable across the driveway, go back and confirm with their supervisor about the job they're here to do because that isn't it.

I guess they left. There were no signs of digging when we got back.

Later, I checked the dates. It's been 18 days since they applied for a permit. I'm pretty sure the county told me a permit is only good for 11 days, or was it two weeks? Any, less than 18 days. So they probably didn't have a permit any more to dig across the street anyway ... not that that necessarily has any bearing on whether they'd dig.

Tags: internet, cable, comcast
[ 09:20 May 21, 2014    More misc | permalink to this entry | ]

A Raspberry Pi motion-detecting wildlife camera

I've been working on an automated wildlife camera, to catch birds at the feeder, and the coyotes, deer, rabbits and perhaps roadrunners (we haven't seen one yet, but they ought to be out there) that roam the juniper woodland.

This is a similar project to the PiDoorbell project presented at PyCon, and my much earlier proximity camera project that used an Arduino and a plug computer but for a wildlife camera I didn't want to use a sonar rangefinder. For one thing, it won't work with a bird feeder -- the feeder is always there, so the addition of a bird won't change anything as far as a sonar rangefinder is concerned. For another, the rangefinders aren't very accurate beyond about six feet.

Starting with a Raspberry Pi was fairly obvious. It's low power, cheap, it even has an optional integrated camera module that has reasonable resolution, and I could re-use a lot of the camera code I'd already written for PiDoorbell.

I patched together some software for testing. I'll write in more detail about the software in a separate article, but I started with the simple motion detection code posted by "brainflakes" in the Raspberry Pi forums. It's a slick little piece of code you'll find in various versions all over the net; it uses PIL, the Python Imaging Library, to compare a specified region from successive photos to see how much has changed.

One aside about the brainflakes code: most of the pages you'll find referencing it tell you to install python-imaging-tk. But there's nothing in the code that uses tk, and python-imaging is really all you need to install. I wrote a GUI wrapper for my motion detection code using gtk, so I had no real need to learn the Tk equivalent.

Once I had some software vaguely working, it was time for testing.

The hardware

One big problem I had to solve was the enclosure. I needed something I could put the Pi in that was moderately waterproof -- maybe not enough to handle a raging thunderstorm, but rain or snow can happen here at any time without much warning. I didn't want to have to spend a lot of time building and waterproofing it, because this is just a test run and I might change everything in the final version.

I looked around the house for plastic objects that could be repurposed into a camera enclosure. A cookie container from the local deli looked possible, but I wasn't quite happy with it. I was putting the last of the milk into my morning coffee when I realized I held in my hand a perfect first-draft camera enclosure.

A milk carton must be at least somewhat waterproof, right? Even if it's theoretically made of paper.

I could use the flat bottom as a place to mount the Pi camera with its two tiny screw holes,

and then cut a visor to protect the camera from rain.

It didn't take long to whip it all together: a little work with an X-acto knife, a little duct tape. Then I put the Pi inside it, took it outside and bungeed it to the fence, pointing at the bird feeder.

A few issues I had to resolve:

Raspbian has rather complicated networking. I was using a USB wi-fi dongle, but I had trouble getting the Pi to boot configured properly to talk to our WPA router. In Raspbian networking is configured in about six different places, any one of which might do something like prioritize the not-connected eth0 over the wi-fi dongle, making it impossible to connect anywhere. I ended up uninstalling Network Manager and turning off ifplugd and everything else I could find so it would use my settings in /etc/network/interfaces, and in the end, even though ifconfig says it's still prioritizing eth0 over wlan0, I got it talking to the wi-fi.

I also had to run everything as root. The python-picamera module imports RPi.GPIO and needs access to /dev/mem, and even if you chmod /dev/mem to give yourself adequate permissions, it still won't work except as root. But if I used ssh -X to the Pi and then ran my GUI program with sudo, I couldn't display any windows because the ssh permission is for the "pi" user, not root.

Eventually I gave up on sudo, set a password for root, and used ssh -X root@pi to enable X.

The big issue: camera quality

But the real problem turned out to be camera quality.

The Raspberry Pi camera module has a resolution of 2592 x 1944, or 5 megapixels. That's terrific, far better than any USB webcam. Clearly it should be perfect for this tast.

Update: see below. It's not a good camera, but it turns out I had a lens problem and it's not this bad.

So, the Pi camera module might be okay if all I want is a record of what animals visit the house. This image is good enough, just barely, to tell that we're looking at a house finch (only if we already rule out similar birds like purple finch and Cassin's finch -- the photo could never give us enough information to distinguish among similar birds). But what good is that? I want decent photos that I can put on my web site.

I have a USB camera, but it's only one megapixel and gives lousy images, though at least they're roughly in focus so they're better than the Pi cam.

So now I'm working on a setup where I drive an external camera from the Pi using gphoto2. I have most of the components working, but the code was getting ugly handling three types of cameras instead of just two, so I'm refactoring it. With any luck I'll have something to write about in a week or two.

Meanwhile, the temporary code is in my github rpi directory -- but it will probably move from there soon.

I'm very sad that the Pi camera module turned out to be so bad. I was really looking forward to buying one of the No-IR versions and setting up a night wildlife camera. I've lost enthusiasm for that project after seeing how bad the images were. I may have to investigate how to remove the IR filter from a point-and-shoot camera, after I get the daylight version working.

Update, a few days later: It turns out I had some spooge on the lens. It's not quite as bad as I made it out to be. Here's a sample. It's still not a great camera, and it can't focus anywhere near as close as the 2 feet I've seen claimed -- 5 feet is about the closest mine can focus, which means I can't get very close to the wildlife, which was a lot of the point of building a wildlife camera. I've seen suggestions of putting reading glasses in front of the lens as a cheap macro adaptor.

Instead, I'm going ahead with the gphoto2 option, which is about ready to test -- but the noIR Pi camera module might be marginally acceptable for a night wildlife camera.

Tags: crittercam, hardware, raspberry pi, photography, maker
[ 13:30 May 15, 2014    More hardware | permalink to this entry | ]

Sonograms in Python

I went to a terrific workshop last week on identifying bird songs. We listened to recordings of songs from some of the trickier local species, and discussed the differences and how to remember them. I'm not a serious birder -- I don't do lists or Big Days or anything like that, and I dislike getting up at 6am just because the birds do -- but I do try to identify birds (as well as mammals, reptiles, rocks, geographic features, and pretty much anything else I see while hiking or just sitting in the yard) and I've always had trouble remembering their songs.

One of the tools birders use to study bird songs is the sonogram. It's a plot of frequency (on the vertical axis) and intensity (represented by color, red being louder) versus time. Looking at a sonogram you can identify not just how fast a bird trills and whether it calls in groups of three or five, but whether it's buzzy/rattly (a vertical line, lots of frequencies at once) or a purer whistle, and whether each note is ascending or descending.

The class last week included sonograms for the species we studied. But what about other species? The class didn't cover even all the local species I'd like to be able to recognize. I have several collections of bird calls on CD (which I bought to use in combination with my "tweet" script -- yes, the name messes up google searches, but my tweet predates Twitter -- a tweet Python script and tweet in HTML for Android). It would be great to be able to make sonograms from some of those recordings too.

But a search for Linux sonogram turned up nothing useful. Audacity has a histogram visualization mode with lots of options, but none of them seem to result in a usable sonogram, and most discussions I found on the net agreed that it couldn't do it. There's another sound editor program called snd which can do sonograms, but it's fiddly to use and none of the many color schemes produce a sonogram that I found very readable.

Okay, what about python scripts? Surely that's been done?

I had better luck there. Matplotlib's pylab package has a specgram() call that does more or less what I wanted, and here's an example of how to use pylab.specgram(). (That post also has another example using a library called timeside, but timeside's PyPI package doesn't have any dependency information, and after playing the old RPM-chase game installing another dependency, trying it, then installing the next dependency, I gave up.)

The only problem with pylab.specgram() was that it shows the full range of the sound, both in time and frequency. The recordings I was examining can last a minute or more and go up to 20,000 Hz -- and when pylab tries to fit that all on the screen, you end up with a plot where the details are too small to show you anything useful.

You'd think there would be a way for pylab.specgram() to show only part of the spectrum, but that doesn't seem to be. I finally found a Stack Overflow discussion where "edited" gives an excellent rewritten version of pylab.specgram which allows setting minimum and maximum frequency cutoffs. Worked great!

Then I did some fiddling to allow for analyzing only part of the recording -- Python's wave package has no way to read in just the first six seconds of a .wav file, so I had to read in the whole file, read the data into a numpy array, then take a slice representing the seconds of the recording I actually wanted.

But now I can plot nice sonograms of any bird song I want to see, print them out or stick them on my Android device so I can carry them with me.

Update: Oops! I forgot to include a link to the script. Here it is: Sonograms in Python.

Tags: programming, python, nature, birds, matplotlib
[ 09:17 May 11, 2014    More programming | permalink to this entry | ]

Getting internet at the new house: a Comcast Odyssey

We finally have internet at the house!

Not through any success on the Comcast front, mind you. They're still stalling and dithering. We finally did what we should have done the very first time they failed to install, and ordered DSL.

I've been keeping track of the progress on the Comcast debacle, though, with the intention of blogging it. But as I updated it last week, I realized that it was up to nearly 400 lines already -- at least triple my usual maximum for a blog post -- and though we're over ten weeks in, it's still nowhere near over.

So I've decided to post what I have, as a running web page that I can add to as the odyssey continues, and just post a link to it here. It's a tale of woe, lies, misleading information, crossed wires, chopped-up wires, and about every other problem you could imagine.

If you're thinking of signing up with Comcast but you have any other options, it'll definitely convince you to look elsewhere.

Here's the tale so far: Getting internet at the new house: a Comcast Odyssey.

Now excuse me while I go enjoy my DSL some more.

Tags: internet, cable, comcast
[ 19:13 May 04, 2014    More misc | permalink to this entry | ]