Shallow Thoughts : : Aug

Reading Palm Datebook files on Linux

Over the years, I've kept a few sets of records in the Datebook app on my PalmOS PDA -- health records and such. I've been experimenting with a few python plotting packages (pycha, CairoPlot and a few others) and I wanted to try plotting one of my Datebook databases.

Not so fast. It seems that it's been a year or more since I last crunched any of this data -- and in the time since then, pilot-link has bumped its version numbers and is now shipping libpisock.so.9 instead of .8.

So what? Well, the problem is that Linux hasn't offered any way to read Palm Datebook files for years. The pilot-link package offered on most distros used to include a program called pilot-datebook, but it was deleted from the source several years ago. Apparently it was hard to maintain.

Back when it first disappeared, I built the previous version of the source, stuck the pilot-datebook binary in ~/bin/linux and have been using it ever since. Which worked fine -- until libpisock.so.8 was no longer there. (Linking .9 to .8 didn't work either.) This is all the more ironic because I don't need pilot-datebook to talk to the PDA with libpisock -- all I want to do is parse the format of a file I've already uploaded.

Off to hunt for an old version of the source. I started at pilot-link.org, but gave up after a while -- they don't seem to have source there except for the latest couple of versions, nor do they have any documentation. Ironically, in their FAQ the very first question is "How can I read the databook entries from a Palm backup?" but the FAQ page is broken and the "answer" is actually another unrelated FAQ question.

Anyway, no help there. I tried googling for old tarballs but there doesn't seem to be anything like archive.org for source code. All I found was the original pilot-datebook page, with a tarball that you insert into a copy of pilot-link 0.9.5 then modify the Makefile. Might work but that's really old.

So I fell back on old distributions. I guessed that Ubuntu Dapper was old enough that it might still have pilot-datebook. So I went to the Dapper pilot-link source and downloaded the source tarball (curiously, they don't offer src debs -- you have to download the tarball and patches separately).

Of course, it doesn't build on Ubuntu Jaunty. It had various entertaining errors ranging from wanting a mysterious tcl.m4 file not present in the code ... to not being able to find <iostream.h< because all the C++ stdlib files have recently been renamed to remove the .h ... to a change in the open() system call where I needed to add permissions argument for O_CREAT.

But I did get it working! So now I have a pilot-datebook program that builds and runs on Ubuntu Jaunty, and parses my DatebookDB.pdb file.

Since I bet I'm not the only one in the world who occasionally wants to read a Palm Datebook file, I've put my working version of the source here: pilot-link_0.11.8.jaunty.tar.gz.

After the usual configure and make, if all you want is pilot-datebook, cd src/pilot-datebook then copy both pilot-datebook and the directory .libs to wherever you want to install them.

And yeah, it would be better to write a standalone program that just parsed the format. But it's hard to justify that for what's essentially a dead platform. The real solution is to quit using a Palm for this, import the data into some common format and keep it on my Linux workstation from now on.

Tags: palm, linux, programming, patch
[ 12:39 Aug 31, 2009    More linux | permalink to this entry | ]

Linux Bloat 102

Part 2 of my Linux bloat article looks at information you can get from the kernel via some useful files in /proc, at three scripts that display that info, and also at how to use exmap, an app and kernel module that shows you a lot more about what resources your apps are using. How Do You Really Measure Linux Bloat?

Tags: writing, programming, linux, performance, bloat
[ 20:52 Aug 27, 2009    More writing | permalink to this entry | ]

Fedora 11 not encouraging so far

Noticing that every article I write ends up including a section on "This doesn't work under Ubuntu; here's a link to the year-old bug with a patch and several workarounds," I decided to try Fedora 11 and see if it was any better.

I downloaded and burned the latest netinst CD ISO and booted it on the Atom machine. It greeted me with this prompt:

    1.
    2.
Select CD-ROM boot type: 

I am not getting warm fuzzies about Fedora being the solution to the Linux distro problem.

What do you think? Should I choose 1. or 2. ?

Tags: linux, install, fedora
[ 21:09 Aug 23, 2009    More linux/install | permalink to this entry | ]

Crikey 0.8.3: bug fixes and several new syntaxes

Sometimes I love open source. A user contacted me about my program Crikey!, which lets you generate key events to do things like assign a key that will type in a string you don't want to type in by hand. He had some thorny problems where crikey was failing, and a few requests, like sending Alt and other modifier keys.

We corresponded a bit, figured out exactly how things should work and some test cases, went through a couple iterations of changes where I got lots of detailed and thoughtful feedback and more test cases, and now Crikey can do an assortment of new useful stuff.

New features: crikey now handles number codes like \27, modifier keys like \A for alt, does a better job with symbols like \(Return\), and handles a couple of new special characters like \e for escape. It also works better at sending window manager commands, like "\A\t" to change the active window.

I've added some better documentation on all the syntaxes it understands, both on the web page and in the -h and -l (longhelp) command-line arguments, and made a release: crikey 0.8.3.

Plus: a list of great regression tests that I can use when testing future updates (in the file TESTING in the tarball).

Tags: programming, crikey, X11, linux
[ 17:38 Aug 19, 2009    More programming | permalink to this entry | ]

A Pair of URL-Shortening Bookmarklets

I'm not a big fan of URL-shortening services -- I like to see what page I'm about to load so I know if I want to go there. But with Twitter's 160-character limit, URL shorteners become necessary. It's tiresome to type in bit.ly every time, so I wanted a bookmark to say "give me a shortened version of the current URL".

Surprisingly, I had a hard time finding one. bit.ly itself has one on their front page, but it didn't work for me. Upon examination, it looks like their bookmark wants to read the clipboard, so you'd have to select a URL first before shortening it (though they don't actually tell you that). I don't want that extra step, so I made my own. Actually two of them.

First, a Javascript version that takes the current URL, encodes it and sends it to bit.ly. I gave it the keyword "bitly", so when I'm on a page, I just type "bitly" in the URLbar and it goes to bit.ly and makes the shortened URL.

The only problem with that is that I'd rather have the option of opening it in a new tab, so I can continue to read the original page. Normally I open new tabs by typing in a URL and typing Ctrl-Return (normally it's Alt-Return in Firefox, but it drives me nuts that Firefox uses Ctrl-click for new tab but Alt-Return and I can never keep them straight, and Firefox's normal behavior for Ctrl-Return is brain-dead useless so that's the first thing I fix when I get a Firefox update).

With this bitly bookmarklet, Ctrl-Return and Alt-Return don't work -- because then you lose the original tab's URL, and bitly gives you a shortened URL to nowhere ... "nowhere" being defined, in the bitly universe, as http://about.com (go figure). What to do?

So I made a second bookmarklet using a different technique: instead of using Javascript to get the current page's URL, call the bookmarklet with the URL as argument. I called this one bitly2. So if I'm pointing at http://shallowsky.com/blog/ and I want a shortened version in a new tab, I type:

Ctrl-L to go to the URLbar
Ctrl-A to go to the beginning of the URL
bitly2 and a space (inserted at the beginning before the URL)
so now I'll see bitly2 http://shallowsky.com/blog/
Ctrl-Return (or Alt-Return) to open in a new tab.

I'm not sure which one I'll end up using more, but I'll obviously change the bitly2 name to something better if I end up using it a lot.

If you want to use either of these bookmarklets: right-click on the link and choose Bookmark this link. Then, alas, since Firefox still doesn't let you enter a keyword in its Bookmarks dialog, you have to go to Bookmarks->Organize Bookmarks, find the bookmarklet you just added and click on it, click on More, and finally you can give it a keyword.

There used to be a Firefox extension called Openbook that let you see the Keyword field when you first add a bookmark, but it doesn't work any more in 3.5, alas. There's another extension called "Add Bookmark Here 2" that's supposed to do it, but the file on addons.mozilla.org is apparently corrupted and won't install. I don't understand why the Firefox crew is so obsessed with bookmark tags (for which I've never found any use) but won't let you add something as truly useful as a keyword. (It's bug 242834, marked WONTFIX.)

Of course, after I had my bookmarklets I finally found a page with a decent bit.ly bookmarklet very similar to my first one: A Quick Tutorial on JavaScript Bookmarklets.

Tags: bookmarklets, twitter, javascript
[ 15:34 Aug 18, 2009    More tech/web | permalink to this entry | ]

Linux Bloat 101

Continuing my Linux Planet series on Linux performance monitoring, the latest article looks at bloat and how you can measure it: Finding and Trimming Linux Bloat.

This one just covers the basics. The followup article, in two weeks, will dive into more detail on how to analyze what resources programs are really using.

Tags: writing, programming, linux, performance, bloat
[ 11:27 Aug 13, 2009    More writing | permalink to this entry | ]

Breaking out of the Walgreens Infinite Loop

Ever get caught in the Walgreens Infinite Loop?

You're phoning in a prescription refill, going through the automated prompts, everything's going fine, and you get to the point where it asks you, "If you will be picking up your prescription tomorrow, press 1. If you will be picking up your prescription today, press 2."

And you mistakenly press 2 when you meant to press 1.

Now you're stuck. "Please enter the pickup time in hours and minutes." Except it's already past 11pm, and anything you try gives you "Please allow at least one hour. Please enter the pickup time ..." No option to switch days or go back to an earlier prompt. You can't press 0 for an operator -- they're closed, there's nobody there. But you can't just hang up, either -- what would happen to your order then? What if they marked it against one of your allowed refills and ... gave it to someone else! Oh no!

But I found the solution after some experimentation: pressing 0, when after hours, breaks out of the loop and schedules the refill for 10am the next morning. Sorry about the rush order, folks. Honestly, I would have been fine waiting another day. I just couldn't find any other way to break out of the loop.

Tags: misc
[ 10:36 Aug 11, 2009    More misc | permalink to this entry | ]

On Teaching Programming (GetSET 2009)

I survived another GetSET Javascript-in-a-day workshop for GetSET 2009.

It went okay, but not as well as I'd hoped. This year's class was more distractable than classes of past years -- and, judging by their career goals, less interested in computers or engineering, unfortunate in a program run by the Society of Women Engineers.

In the morning, we had a hard time getting them to focus long enough to learn basics like what a variable was. After a caucus at lunchtime, we decided to skip the next exercise (looping over an array of colors) and spend some time drilling on the basics, and keep at it 'til they got it. It took a while but we eventually got through. We needed more examples in the morning, more interaction, some visceral way of explaining programming basics so they really get it.

They do better working as a group on a concrete problem, like the final whiteboard exercise, "How do we figure out whether the click was on the flower?". That always ends up being a highlight of the class, even though it involves (gasp) doing math. This year was no exception, but it did take a while to get through. Using variables lost them completely ("is the mouse's X coordinate bigger than or less than the flower's X?") but when we used actual numnbers and ran through several examples, things eventually clicked. "The flower starts at (2, 5) and is 200 pixels wide. If the mouse click is at (34, 45), who thinks it's inside the flower? Raise your hands. Who thinks it's not? Now what if I click at (300, 24)?" A couple of them got it right away, but it took a long time to bring the whole class along.

I'm not still sure how to use that method for more basic concepts like "what is a variable?". Perhaps some sort of role-playing? Watching William Phelps guide the girls through planet motions in our astronomy workshop Wednesday, each girl playing the role of a solar system object, inspired me. I'd used role-playing like that with little kids, but William says it works even with adults to get concepts across, and after seeing him with the high schoolers I believe it. But how to adapt that to programming concepts? A recent Slate article on teaching programming had some interesting ideas I want to try.

Printed handouts for GetSET may be a waste of time. Nobody was even bothering to look at them, despite the fact that they had complete instructions for everything we were doing. Do schools not give students printed assignments or homework any more? Last year, they used the printed exercises but not the quick reference guides; this year they wouldn't even read the exercises. On the other hand, it might be worth it for the handful in each class who really love programming. I always hope some of them take the handouts home and try some of the extras on their own.

Finally, the class would be so much easier if we could teach it on a less pointy-clicky OS! Or at least on machines where IE isn't the default browser. The first 3-4 exercises go painfully slowly, guiding a roomful of girls through many GUI navigation steps:

• Open the GetSET folder on your desktop
• Find the file named whatever.html
• Right-click on it, find Open With, and choose Wordpad
• Now find the window where you were just looking at the GetSET files (because everything on Windows tends to open with huge windows, it's now covered by Wordpad)
• Drag whatever.html into Firefox

Then the helpers have to go around the room ensuring that the girls have the correct file loaded in both Wordpad and Firefox. This took way too long with only four people to check the whole class, especially since we had to do it for every exercise. Invariably some girls will doubleclick instead of right-clicking or dragging, and will end up in whatever HTML editor Microsoft is pushing this year, or with an IE window instead of Firefox (and then the Error Console won't be there when she looks for it later).

Suggestions like "Keep that window open, you'll need it throughout the class" or "Try making that window smaller, so you can see both windows at once" don't help. The girls are too used to the standard Windows model of one screen-filling window at a time. Keeping two apps visible at once is too foreign. A few them are good at using the taskbar to switch among apps, but for the rest, loading new files is awkward and error prone.

In postmortems two years ago we talked about having them work on one file throughout the whole workshop. That would solve the problem, but I'm still working on how to do it without a lot of "Now comment out the code you just wrote, so you won't get the prompt every time, then scroll down to the next block of code and uncomment it."

I couldn't help thinking how on Linux, we could just tell them to type leafpad whatever.html; firefox whatever.html and be done. Or even give them an alias that would do it. Hmm ... I wonder if I could make a Windows .bat file that would open the same file in Wordpad and Firefox both? Must try that.

Tags: education, programming
[ 20:12 Aug 07, 2009    More education | permalink to this entry | ]

Twit: Now with pattern searches

During OSCON a couple of weeks ago, I kept wishing I could do Twitter searches for a pattern like #oscon in a cleaner way than keeping a tab open in Firefox where I periodically hit Refresh.

Python-twitter doesn't support searches, alas, though it is part of the Twitter API. There's an experimental branch of python-twitter with searching, but I couldn't get it to work. But it turns out Gwibber is also written in Python, and I was able to lift some JSON code from Gwibber to implement a search. (Gwibber itself, alas, doesn't work for me: it bombs out looking for the Gnome keyring. Too bad, looks like it might be a decent client.)

I hacked up a "search for OSCON" program and used it a little during the week of the conference, then got home and absorbed in catching up and preparing for next week's GetSET summer camp, where I'm running an astronomy workshop and a Javascript workshop for high school girls. That's been keeping me frazzled, but I found a little time last night to clean up the search code and release Twit 0.3 with search and a few other new command-line arguments.

No big deal, but it was nice to take a hacking break from all this workshop coordinating. I'm definitely happier program than I am organizing events, that's for sure.

Tags: twitter, python, programming
[ 18:23 Aug 03, 2009    More programming | permalink to this entry | ]