Shallow Thoughts
Akkana's Musings on Open Source, Science, and Nature.
Fri, 12 Oct 2007
On a recent Mojave desert trip, we tried to follow a minor dirt road
that wasn't mapped correctly on any of the maps we had, and eventually
had to retrace our steps. Back at the hotel, I fired up my trusty
PyTopo on the East
Mojave map set and tried to trace the road. But I found that as I
scrolled along the road, things got slower and slower until it
just wasn't usable any more.
PyTopo was taking up all of my poor laptop's memory. Why?
Python is garbage collected -- you're not supposed to have
to manage memory explicitly, like freeing pixbufs.
I poked around in all the sample code and man pages I had available
but couldn't find any pygtk examples that seemed to be doing any
explicit freeing.
When we got back to civilization (read: internet access) I did
some searching and found the key. It's even in the
PyGTK
Image FAQ, and there's also some discussion in a
mailing
list thread from 2003.
Turns out that although Python is supposed to handle its own garbage
collection, the Python interpreter doesn't grok the size of a pixbuf
object; in particular, it doesn't see the image bits as part of the
object's size. So dereferencing lots of pixbuf objects doesn't trigger
any "enough memory has been freed that it's time to run the garbage
collector" actions.
The solution is easy enough: call gc.collect() explicitly
after drawing a map (or any other time a bunch of pixbufs have been
dereferenced).
So there's a new version of PyTopo, 0.6
that should run a lot better on small memory machines, plus
a new collection format (yet another format from
the packaged Topo! map sets) courtesy of Tom Trebisky.
Oh ... in case you're wondering, the ancient USGS maps from
Topo! didn't show the road correctly either.
Tags: programming, python, gtk, pygtk
[
21:21 Oct 12, 2007
More programming |
permalink to this entry
]
Fri, 25 Aug 2006
Belated release announcement: 0.5b2 of my little map viewer
PyTopo
has been working well, so I released 0.5 last week with only a
few minor changes from the beta.
I'm sure I'll immediately find six major bugs -- but hey, that's
what point releases are for. I only did betas this time because
of the changed configuration file format.
I also made a start on a documentation page for the .pytopo file
(though it doesn't really have much that wasn't already written
in comments inside the script).
Tags: programming, python, gtk, pygtk
[
21:10 Aug 25, 2006
More programming |
permalink to this entry
]
Sat, 03 Jun 2006
A few months ago, someone contacted me who was trying to use my
PyTopo map display script for a different set of map data, the
Topo! National Parks series. We exchanged some email about the
format the maps used.
I'd been wanting to make PyTopo more general
anyway, and already had some hacky code in my local version to
let it use a local geologic map that I'd chopped into segments.
So, faced with an Actual User (always a good incentive!), I
took the opportunity to clean up the code, use some of Python's
support for classes, and introduce several classes of map data.
I called it 0.5 beta 1 since it wasn't well tested. But in the last
few days, I had occasion to do some map exploring,
cleaned up a few remaining bugs, and implemented a feature which
I hadn't gotten around to implementing in the new framework
(saving maps to a file).
I think it's ready to use now. I'm going to do some more testing:
after visiting the USGS
Open House today and watching Jim Lienkaemper's narrated
Virtual
Tour of the Hayward Fault,
I'm all fired up about trying again to find more online geologic
map data.
But meanwhile, PyTopo is feature complete and has the known
bugs fixed. The latest version is on
the PyTopo page.
Tags: programming, python, gtk, pygtk
[
17:25 Jun 03, 2006
More programming |
permalink to this entry
]
Tue, 21 Jun 2005
I updated my Debian sid system yesterday, and discovered today that
gnome-volume-control has changed their UI yet again. Now the window
comes up with two tabs,
Playback and
Capture; the
default tab,
Playback, has only one slider in it,
PCM,
and all the important sliders, like
Volume, are under
Capture. (I'm told this is some interaction with how ALSA
sees my sound chip.)
That's just silly. I've never liked the app anyway -- it takes
forever to come up, so I end up missing too much of any clip that
starts out quiet. All I need is a simple, fast window with
a single slider controlling master volume. But nothing like that
seems to exist, except panel applets that are tied to the panels
of particular window managers.
So I wrote one, in PyGTK. vol is
a simple script which shows a slider, and calls aumix
under the hood to get and set the volume. It's horizontal by
default; vol -h gives a vertical slider.
Aside: it's somewhat amazing that Python has no direct way
to read an integer out of a string containing more than just that
integer: for example, to read 70 out of "70,". I had to write a
function to handle that. It's such a terrific no-nonsense
language most of the time, yet so bad at a few things.
(And when I asked about a general solution in the python channel
at [large IRC network], I got a bunch of replies like "use
int(str[0:2])" and "use int(str[0:-1])".
Shock and bafflement ensued when I pointed out that 5, 100, and -27
are all integers too and wouldn't be handled by those approaches.)
Tags: programming, python, gtk, pygtk
[
14:54 Jun 21, 2005
More programming |
permalink to this entry
]
Sat, 09 Apr 2005
A few days ago, I mentioned my woes regarding Python sending spurious
expose events every time the drawing area gains or loses focus.
Since then, I've spoken with several gtk people, and investigated
several workarounds, which I'm writing up here for the benefit of
anyone else trying to solve this problem.
First, "it's a feature". What's happening is that the default focus
in and out handlers for the drawing area (or perhaps its parent class)
assume that any widget which gains keyboard focus needs to redraw
its entire window (presumably because it's locate-highlighting
and therefore changing color everywhere?) to indicate the focus
change. Rather than let the widget decide that on its own, the
focus handler forces the issue via this expose event. This may be a
bad decision, and it doesn't agree with the gtk or pygtk documentation
for what an expose event means, but it's been that way for long enough
that I'm told it's unlikely to be changed now (people may be depending
on the current behavior).
Especially if there are workarounds -- and there are.
I wrote that this happened only in pygtk and not C gtk, but I was
wrong. The spurious expose events are only passed if the CAN_FOCUS
flag is set. My C gtk test snippet did not need CAN_FOCUS,
because the program from which it was taken, pho, already implements
the simplest workaround: put the key-press handler on the window,
rather than the drawing area. Window apparently does not have
the focus/expose misbehavior.
I worry about this approach, though, because if there are any other
UI elements in the window which need to respond to key events, they
will never get the chance. I'd rather keep the events on the drawing
area.
And that becomes possible by overriding the drawing area's default
focus in/out handlers. Simply write a no-op handler which returns
TRUE, and set it as the handler for both focus-in and focus-out. This
is the solution I've taken (and I may change pho to do the same thing,
though it's unlikely ever to be a problem in pho).
In C, there's a third workaround: query the default focus handlers,
and disconnect() them. That is a little more efficient (you
aren't calling your nop routines all the time) but it doesn't seem to
be possible from pygtk: pygtk offers disconnect(), but there's no way to
locate the default handlers in order to disconnect them.
But there's a fourth workaround which might work even in pygtk:
derive a class from drawing area, and set the focus in and out
handlers to null. I haven't actually tried this yet, but it may be
the best approach for an app big enough that it needs its own UI classes.
One other thing: it was suggested that I should try using AccelGroups
for my key bindings, instead of a key-press handler, and then I could
even make the bindings user-configurable. Sounded great!
AccelGroups turn out to be very easy to use, and a nice feature.
But they also turn out to have undocumented limitations on what
can and can't be an accelerator. In particular, the arrow keys can't
be accelerators; which makes AccelGroup accelerators less than
useful for a widget or app that needs to handle user-initiated
scrolling or movement. Too bad!
Tags: programming, python, gtk, pygtk
[
20:52 Apr 09, 2005
More programming |
permalink to this entry
]
Wed, 06 Apr 2005
While on vacation, I couldn't resist tweaking
pytopo
so that I could use it to explore some of the areas we were
visiting.
It seems fairly usable now. You can scroll around, zoom in and out
to change between the two different map series, and get the
coordinates of a particular location by clicking. I celebrated
by making a page for it, with a silly tux-peering-over-map icon.
One annoyance: it repaints every time it gets a focus in or out,
which means, for people like me who use mouse focus, that it
repaints twice for each time the mouse moves over the window.
This isn't visible, but it would drag the CPU down a bit on a
slow machine (which matters since mapping programs are particularly
useful on laptops and handhelds).
It turns out this is a pygtk problem: any pygtk drawing area window
gets spurious Expose events every time the focus changes (whether or
not you've asked to track focus events), and it reports that the
whole window needs to be repainted, and doesn't seem to be
distinguishable in any way from a real Expose event.
The regular gtk libraries (called from C) don't do this, nor
do Xlib C programs; only pygtk.
I filed
bug 172842
on pygtk; perhaps someone will come up with a workaround, though
the couple of pygtk developers I found on #pygtk couldn't think
of one (and said I shouldn't worry about it since most people
don't use pointer focus ... sigh).
Tags: programming, python, gtk, pygtk
[
16:26 Apr 06, 2005
More programming |
permalink to this entry
]
Sun, 27 Mar 2005
I couldn't stop myself -- I wrote up a little topo map viewer in
PyGTK, so I can move around with arrow keys or by clicking near the
edges. It makes it a lot easier to navigate the map directory if
I don't know the exact starting coordinates.
It's called pytopo,
and it's in the same
place as my earlier two topo scripts.
I think CoordsToFilename has some bugs; the data CD also has some
holes, and some directories don't seem to exist in the expected
place. I haven't figured that out yet.
Tags: programming, python, gtk, pygtk
[
17:53 Mar 27, 2005
More programming |
permalink to this entry
]
I've long wished for something like those topographic map packages
I keep seeing in stores. The USGS (US Geological Survey) sells
digitized versions of their maps, but there's a hefty setup fee
for setting up an order, so it's only reasonable when buying large
collections all at once.
There are various Linux mapping applications which do things like
download squillions of small map sections from online mapping sites,
but they're all highly GPS oriented and I haven't had much luck
getting them to work without one. I don't (yet?) have a GPS;
but even if I had one, I usually want to make maps for places I've
been or might go, not for where I am right now. (I don't generally
carry a laptop along on hikes!)
The Topo!
map/software packages sold in camping/hiking stores (sometimes
under the aegis of National Geographic
are very reasonably priced. But of course, the software is
written for Windows (and maybe also Mac), not much help to Linux
users, and the box gives no indication of the format of the data.
Googling is no help; it seems no Linux user has ever
tried buying one of these packages to see what's inside.
The employees at my local outdoor equipment store (Mel Cotton's)
were very nice without knowing the answer, and offered
the sensible suggestion of calling the phone number on the box,
which turns out to be a small local company, "Wildflower Productions",
located in San Francisco.
Calling Wildflower, alas, results in an all too familiar runaround:
a touchtone menu tree where no path results in the possibility of
contact with a human. Sometimes I wonder why companies bother to
list a phone number at all, when they obviously have no intention
of letting anyone call in.
Concluding that the only way to find out was to buy one, I did so.
A worthwhile experiment, as it turned out! The maps inside are
simple GIF files, digitized from the USGS 7.5-minute series and,
wonder of wonders, also from the discontinued but still useful
15-minute series.
Each directory contains GIF files covering the area of one
7.5 minute map, in small .75-minute square pieces,
including pieces of the 15-minute map covering the same area.
A few minutes of hacking with python and
Image Magick
resulted in a script to stitch together all images
in one directory to make one full USGS 7.5 minute map;
after a few hours of hacking, I can stitch
a map of arbitrary size given start and end longitude and latitude.
My initial scripts,
such as they are.
Of course, I don't yet have nicities like a key, or an interactive
scrolling window, or interpretation of the USGS digital elevation
data. I expect I have more work to do. But for now, just
being able to generate and print maps for a specific area is a huge boon,
especially with all the mapping we're doing in Field Geology class.
GIMP's "measure" tool will come in handy for measuring distances
and angles!
Tags: programming, python, gtk, pygtk
[
11:13 Mar 27, 2005
More programming |
permalink to this entry
]