Shallow Thoughts : : mapping
Akkana's Musings on Open Source, Science, and Nature.
Mon, 01 Jun 2009
Someone on the OSM newbies list asked how he could strip
waypoints out of a GPX track file. Seems he has track logs of an
interesting and mostly-unmapped place that he wants to add to
openstreetmap, but there
are some waypoints that shouldn't be included, and he wanted a
good way of separating them out before uploading.
Most of the replies involved "just edit the XML." Sure, GPX files
are pretty simple and readable XML -- but a user shouldn't ever have
to do that! Gpsman and gpsbabel were also mentioned, but they're not
terribly easy to use either.
That reminded me that I had another XML-parsing task I'd been wanting
to write in Python: a way to split track files from my Garmin GPS.
Sometimes, after a day of mapping, I end up with several track
segments in the same track log file. Maybe I mapped several different
trails; maybe I didn't get a chance to upload one day's mapping before
going out the next day. Invariably some of the segments are of zero
length (I don't know why the Garmin does that, but it always does).
Applications like merkaartor don't like this one bit, so I
usually end up editing the XML file and splitting it into
segments by hand. I'm comfortable with XML -- but it's still silly.
I already have some basic XML parsing as part
of PyTopo and Ellie, so I know the parsing very easy to do.
So, spurred on by the posting on OSM-newbies,
I wrote a little GPX parser/splitter called
gpxmgr.
gpxmgr -l file.gpx can show you how many track logs are
in the file; gpxmgr -w file.gpx can write new files for
each non-zero track log. Add -p if you want to be prompted for
each filename (otherwise it'll use the name of the track log,
which might be something like "ACTIVE\ LOG\ #2").
How, you may wonder, does that help the original
poster's need to separate out waypoints from track files?
It doesn't. See, my GPS won't save tracklogs and
waypoints in the same file, even if you want them that way;
you have to use two separate gpsbabel commands to upload a track
file and a waypoint file. So I don't actually know what a
tracklog-plus-waypoint file looks like.
If anyone wants to use gpxmgr to manage waypoints as well as tracks,
send me a sample GPX file that combines them both.
Tags: mapping, programming, python
[
19:43 Jun 01, 2009
More mapping |
permalink to this entry
]
Sun, 04 Jan 2009
I got myself a GPS unit for Christmas.
I've been resisting the GPS siren song for years -- mostly because I
knew it would be a huge time sink involving months of futzing with
drivers and software trying to get it to do something useful.
But my experience at an OpenStreetMap
mapping
party got me fired up about it, and I ordered a Garmin Vista Cx.
Shopping for a handheld GPS is confusing. I was fairly convinced I
wanted a Garmin, just because it's the brand used by most people in
the open source mapping community so I knew they were likely to work.
I wanted one with a barometric altimeter, because I
wanted that data from my hikes and bike rides (and besides,
it's fun to know how much you've climbed on an outing; I used to have
a bike computer with an altimeter and it was a surprisingly good
motivator for working harder and getting in better shape).
But Garmin has a bazillion models and I never found any comparison
page explaining the differences among the various hiking eTrex models.
Eventually I worked it out:
Garmin eTrex models, decoded
- C
- Color display. This generally also implies USB connectivity
instead of serial, just because the color models are newer.
- H
- High precision (a more sensitive satellite receiver).
- x
- Takes micro-SD cards. This may not be important for storing
tracks and waypoints (you can store quite a long track with the
built-in memory) but they mean that you can load extra base maps,
like topographic data or other useful features.
- Vista, Summit
- These models have barometric altimeters and magnetic compasses.
(I never did figure out the difference between a Vista and a Summit,
except that in the color models (C), Vistas take micro-SD cards (x)
while Summits don't, so there's a Summit C and HC while Vistas
come in Cx and HCx. I don't know what the difference is between
a monochrome Summit and Vista.)
- Legend, Venture
- These have no altimeter or compass.
A Venture is a Legend that comes without the bundled
extras like SD card, USB cable and base maps, so it's cheaper.
For me, the price/performance curve pointed to the Vista Cx.
Loading maps
Loading base maps was simplicity itself, and I found lots of howtos
on how to use downloadable maps. Just mount the micro-SD card on any
computer, make a directory called Garmin, and name the file
gmapsupp.img.
I used the CloudMade map
for California, and it worked great.
There are lots of howtos on generating your own maps, too,
and I'm looking forward to making some with topographic data
(which the CloudMade maps don't have). The most promising
howtos I've found so far are the
OSM
Map On Garmin page on the OSM wiki and the much more difficult,
but gorgeous,
Hiking
Biking Mapswiki page.
Uploading tracks and waypoints
But the real goal was to be able to take this toy out on a hike,
then come back and upload the track and waypoint files.
I already knew, from the mapping party, that Garmins have an odd
misfeature: you can connect them in usb-storage mode, where they look
like an external disk and don't need any special software ... but then
you can't upload any waypoints. (In fact, when I tried it with my
Vista Cx I didn't even see the track file.) To upload tracks and
waypoints, you need to use something that speaks Garmin protocol:
namely, the excellent GPSBabel.
So far so good. How do you call GPSbabel?
Luckily for me, just before my GPS arrived,
Iván Sánchez Ortega posted a
useful
little gpsbabel script
to the OSM newbies list and I thought I was all set.
But once I actually had the Vista in hand, complete with track and
waypoints from a walk around the block, it turned out it wasn't quite
that simple -- because Ubuntu didn't create the /dev/ttyUSB0 that
Iván's script used. A web search found tons of people having that
problem on Ubuntu and talking about various workarounds, involving
making sure the garmin_usb driver is blacklisted in
/etc/modprobe.d/blacklist (it was already), adding a
/etc/udev/rules.d/45-garmin.rules file that changes permissions
and ownership of ... um, I guess of the file that isn't being created?
That didn't make much sense. Anyway, none of it helped.
But finally I found the fix: keep the garmin_usb driver blacklisted
use "usb:" as the device to pass to GPSBabel rather than
"/dev/ttyUSB0". So the commands are:
gpsbabel -t -i garmin -f usb: -o gpx -F tracks.gpx
gpsbabel -i garmin -f usb: -o gpx -F waypoints.gpx
Like so many other things, it's easy once you know the secret!
Viewing tracklogs works great in Merkaartor, though I haven't yet
found an app that does anything useful with the elevation data.
I may have to write one.
Update: After I wrote this but before I was able to post it,
a discussion on the OSM Newbies list with someone
who was having similar troubles resulted in this useful wiki page:
Garmin
on GNU/Linux. It may also be worth checking the Discussion
tab on that wiki page for further information.
Second Update:
Also, I did have to create a udev file,
/etc/udev/rules.d/51-garmin.rules, to set the permissions so
that I could access the device without being root. It contains the
line:
ATTRS{idVendor}=="091e", ATTRS{idProduct}=="0003", MODE="0660", GROUP="plugdev"
I did that early on then forgot about that step because I thought that
the /dev/ttyUSB0 vs. usb: issue was the only one causing the problem.
Tags: gps, mapping, linux, ubuntu, udev
[
15:31 Jan 04, 2009
More mapping |
permalink to this entry
]
Sat, 03 Jan 2009
Latest obsession: mapping with
OpenStreetMap.
Last month, OpenStreetMap and its benefactor company
CloudMade
held a "mapping party" in Palo Alto. I love maps and mapping (I wrote
my own little topographic
map viewer when I couldn't find one ready-made) and I've been
wanting to know more about the state of open source mapping.
A mapping party sounded perfect.
The party was a loosely organized affair. We met at a coffeehouse
and discussed basics of mapping and openstreetmap. The hosts tried
to show us newbies how OSM works, but that was complicated by the
coffeehouse's wireless net being down. No big deal -- turns out the
point of a mapping party is to hand out GPSes to anyone who doesn't
already have one and send us out to do some mapping.
I attached myself to a couple of CloudMade folks who had some
experience already and we headed north on a pedestrian path. We spent
a couple of hours walking urban trails and marking waypoints.
Then we all converged on a tea shop (whose wireless worked a little
better than the one at the coffeehouse, but still not very reliably)
for lunch and transfer of track and waypoint files.
This part didn't work all that well. It turned out the units we were
using (Garmin Legend HCx) can transfer files in two modes, USB
mass storage (the easy way, just move files as if from an external
disk) or USB Garmin protocol (the hard way: you have to use software
like gpsbabel, or the Garmin software if you're on Windows).
And in mass storage mode, you get a file but the waypoints aren't there.
The folks running the event all had Macs, and there were several Linux
users there as well, but no Windows laptops. By the time the Macs both
had gpsbabel downloaded over the tea shop's flaky net, it was past
time for me to leave, so I never did get to see our waypoint files.
Still, I could see it was possible (and one of the Linux attendees
assured me that he had no trouble with any of the software; in fact,
he found it easier than what the Mac people at the party were going
through).
But I was still pretty jazzed about how easy OpenStreetMap is
to use. You can contribute to the maps even without a GPS.
Once you've registered on the site, you just click on the Edit tab
on any map, and you see a flash application called "Potlatch" that
lets you mark trails, roads or other features based on satellite
images or the existing map. I was able to change a couple of mismarked
roads near where I live, as well as adding a new trail and correcting
the info on an existing one for one of the nearby parks.
If you prefer (as, I admit, I do) to work offline or don't like flash,
you can use a Java app, JOSM, or a native app, merkaartor. Very cool!
Merkaartor is my favorite so far (because it's faster and works
better in standalone mode) though it's still fairly rough around
the edges. They're all described on the OSM
Map Editing
page.
Of course, all this left me lusting after a GPS. But that's another
story, to be told separately.
Tags: gps, mapping, open source
[
12:00 Jan 03, 2009
More mapping |
permalink to this entry
]