Shallow Thoughts : : Oct

New versions of mapping programs: Pytopo and Ellie

On our recent Mojave trip, as usual I spent some of the evenings reviewing maps and track logs from some of the neat places we explored.

There isn't really any existing open source program for offline mapping, something that works even when you don't have a network. So long ago, I wrote Pytopo, a little program that can take map tiles from a Windows program called Topo! (or tiles you generate yourself somehow) and let you navigate around in that map.

But in the last few years, a wonderful new source of map tiles has become available: OpenStreetMap. On my last desert trip, I whipped up some code to show OSM tiles, but a lot of the code was hacky and empirical because I couldn't find any documentation for details like the tile naming scheme.

Well, that's changed. Upon returning to civilization I discovered there's now a wonderful page explaining the Slippy map tilenames very clearly, with sample code and everything. And that was the missing piece -- from there, all the things I'd been missing in pytopo came together, and now it's a useful self-contained mapping script that can download its own tiles, and cache them so that when you lose net access, your maps don't disappear along with everything else.

Pytopo can show GPS track logs and waypoints, so you can see where you went as well as where you might want to go, and whether that road off to the right actually would have connected with where you thought you were heading.

It's all updated in svn and on the Pytopo page.

Ellie

Most of the pytopo work came after returning from the desert, when I was able to google and find that OSM tile naming page. But while still out there and with no access to the web, I wanted to review the track logs from some of our hikes and see how much climbing we'd done. I have a simple package for plotting elevation from track logs, called Ellie. But when I ran it, I discovered that I'd never gotten around to installing the pylab Python plotting package (say that three times fast!) on this laptop.

No hope of installing the package without a net ... so instead, I tweaked Ellie so that so that without pylab you can still print out statistics like total climb. While I was at it I added total distance, time spent moving and time spent stopped. Not a big deal, but it gave me the numbers I wanted. It's available as ellie 0.3.

Tags: mapping, GIS, programming, python, pytopo, openstreetmap
[ 19:24 Oct 30, 2010    More mapping | permalink to this entry | ]

GIMP tricks for isolating parts of photos

Today's Linux Planet article covers some basic tips for how to cut a foreground object out of a photograph, so you can grab that penguin or flower or motorcycle and paste it somewhere else.

Read it here: GIMP tricks for isolating parts of photos

This is mostly beginner stuff, for people who haven't done this at all. Part II, in two weeks, will cover more advanced techniques.

Tags: writing, gimp
[ 15:37 Oct 28, 2010    More gimp | permalink to this entry | ]

Termite Feast at RSA

At Rancho San Antonio today (Los Altos Hills), high on the High Meadow and PG&E trails, there's an incredible abundance of termite colonies on the trail -- the trail is thick and silvery with them in places.

A few colonies are flying, and around the flying ones there's a great diversity of wildlife partaking in the feast -- in about five minutes I saw wrentits, juncos, chestnut-backed chickadees, Townsend's warblers, woodpeckers (several flew by too fast to identify), spotted towhees, a Bewick's wren that didn't cock its tail like a normal wren, northern flickers ... plus chipmunks.

And the species that normally hide out in thick brush and resist being photographed -- especially the wrentit and the chipmunk -- were so busy gobbling tidbits that they didn't pay much attention to a photographer snapping away.

Quite a show! The lower parts of RSA were fairly nice too -- I got a good look at a red-shouldered hawk that swooped low across the trail, plus lots of quail, rabbits and squirrels. There's a sign just past the farm warning to stay away from "sick bobcats" (the nature of the disease is unspecified) but we didn't see any cats.

Photos: Termite feast at RSA.

Tags: nature, birds, chipmunk, squirrels
[ 21:25 Oct 27, 2010    More nature | permalink to this entry | ]

The non-dairy dairy

On a recent desert trip, we stopped in Jean, NV for some prime rib.

They had creamer containers out on the table in case you should order coffee. It was their own brand of non-dairy creamer. I wonder who really makes it?

Silly question -- the non-dairy creamer comes from a dairy! And includes "sodium caseinate (a milk derivative)".

Leaves me full of curiosity about the cows at this dairy ...

Tags: humor, food
[ 17:39 Oct 23, 2010    More humor | permalink to this entry | ]

Snakes on a Couch! Using Python with CouchDB

Part II of my CouchDB tutorial is out at Linux Planet. In it, I use Python and CouchDB to write a simple application that keeps track of which restaurants you've been to recently, and to suggest new places to eat where you haven't been.

Snakes on a Couch, Part 2: Where do you want to eat?

Tags: writing, python, programming, database, couchdb
[ 21:00 Oct 15, 2010    More writing | permalink to this entry | ]

Creating and mounting a LUKS encrypted disk

I've previously written about how to use 'cryptoloop' encryption on a flash drive or SD card. An encrypted SD card or USB stick is very handy when you have personal files you want to take with you between several different machines.

But modern Gnome systems can't read cryptoloop. Or, rather, they can, but you have to fiddle with them as root -- they won't recognize and mount the filesystem automatically.

It turns out that's because the "new way", instead of cryptoloop, is to use a system called LUKS. But it has a few pitfalls, and there's no documentation about how to use it on a system that doesn't recognize it automatically. So here's some.

Creating a LUKS filesystem

(Updated December 2023)

Palimpsest, described below, no longer seems to exist. But it's just as easy to set up a LUKS filesystem with cryptsetup.

I'm using "SECRET" as the disk label; this is a name you'll use to mount the disk later. Of course, all these commands require root.

cryptsetup -v luksFormat --label SECRET /dev/TARGET_DISK_PARTITION

Read more about luksFormat options in man cryptsetup-luksFormat.

Update: This used to be enough to make the device available in /dev/mapper, but in 2024, you have to open it explicitly: sudo cryptsetup luksOpen /dev/PARTITION SECRET

Now your new encrypted device is available on /dev/mapper/SECRET. Next, create a filesystem on the encrypted device:

mkfs.ext4 /dev/mapper/SECRET

Finally, make sure you can mount it:

mount /dev/mapper/SECRET /mnt

(or wherever you prefer to mount it).

This next part, written in 2010, is now obsolete:

Mounting a LUKS filesystem from Gnome or Another Desktop

In theory, you should be able to plug in the device and after a few seconds you'll be prompted for your password. If it doesn't, which sometimes happens, try again, wait longer this time and cross your fingers. If it still doesn't mount, try the command-line version in the next section. Even if it doesn't work, you might get a useful error message.

Mounting a LUKS filesystem from the commandline

Assuming you used the partition label "SECRET" when you created the LUKS encrypted partition, the physical partition is on /dev/sdb2, and you want to mount it on /media/SECRET (which already exists), these two commands (as root) will mount it:

sudo cryptsetup luksOpen /dev/sdb2 SECRET
  (prompts for sudo password)
  (prompts for LUKS password)
sudo mount /dev/mapper/SECRET /media/SECRET

Easy -- yet a bit frustrating. There seems to be no way to do this purely through /etc/fstab, so you have to remember the cryptsetup command, or write an alias or script to do the two steps for you. And you always have to type your sudo password as well as the password for the filesystem, whereas with cryptoloop you only needed the filesystem's password.

In the end, I'm not convinced LUKS is a win. But since it's so hard to manage cryptoloop filesystems from a Gnome desktop, it's probably worth hassling with LUKS if you need to be able to interoperate with Gnome.

Update: I wrote that yesterday. Today, maybe three weeks after I started using the card on a fairly regular basis to transfer personal files between home and laptop, I had a filesystem failure: I wrote to the card from the desktop, synced, unmounted, put it in the laptop -- and got I/O errors and "You must specify filesystem type" trying to mount it. I was able to fsck, and it apparently restored from an old journal -- including old data.

No loss here, because the card is just a copy of what was on the desktop machine. But the lesson here is: these encrypted cards are great for emergency backups. But you probably don't want to rely on one as your main storage for anything important.

Tags: linux, encryption, security
[ 13:57 Oct 05, 2010    More linux | permalink to this entry | ]