Shallow Thoughts : : May

JS Jup: now, with variable animation speed

I wrote last week about the sorts of programmer compulsions that lead to silly apps like my animated Javascript Jupiter. I got it working well enough and stopped, knowing there were more features that would be easy to add but trying to ignore them.

My mom, immediately upon seeing it, unerringly zeroed in on the biggest missing feature I'd been trying to ignore. "Can you make it go faster or slower?"

I put it off for a while, but of course I had to do it -- so now there are Faster and Slower buttons. It still goes by hour jumps, so the fastest you can go is an hour per millisecond. Fun to watch. Or you can slow it down to 1 hour per 3600000 milliseconds if you want to see it animate in real time. :-)

Tags: programming, astronomy, javascript
[ 11:42 May 31, 2009    More programming | permalink to this entry | ]

Programming With PyGTK, part 2: pretty screensaver-type graphics

Part 2 of Graphical Python Programming With PyGTK gets into how to do some cool Qix screensaver-style graphics, in: Graphical Python Programming With PyGTK, part 2: Write Your Own Screensaver.

There's also a digg link.

Tags: writing, python, programming
[ 18:09 May 28, 2009    More writing | permalink to this entry | ]

Javascript Jupiter

It's a sickness, I tell you.

It's not like I needed another Jupiter's moons application. I've already written more or less the same app for four platforms.

I don't use the Java web version, Juplet, very much any more, because I often have Java disabled or missing. And I don't use my Zaurus any more so Juplet for Zaurus isn't very relevant. But I can always call up my Xlib or PalmOS Jupiter's moons app if I need to check on those Galilean moons. They work fine. Another version would be really pointless. A waste of time.

So it should have been no big deal when, during the course of explaining to someone the difference between Java and Javascript, it suddenly occurred to me that it would be awfully easy to re-implement that Java Juplet web page using Javascript, HTML and CSS. I mean, a rational person would just say "oh, yeah, I suppose that's true" and go on with life.

But what I'm trying to say is that programming isn't a career path, or a hobby, or a field of academic study. It's a disease. It's a compulsion, where, sometimes, just realizing that something could be done renders you unable to think about anything else until you just ... try ... just a few minutes ... see how well it works ... oh, wow, that really looks a lot better than the Java version, wouldn't it look even nicer if you just added in this one other little tweak ... but wait, now it's so close to working, I bet it wouldn't be all that hard to take the Java class and turn it into ...

... and before you know it, it's tomorrow and you have something that's almost a working app, and it's just really a shame to get that far and not finish it at least to the point where you can share it.

But then, Javascript and web pages are so easy to work on that it really isn't that much extra work to add in some features that the old version didn't have, like an animate button ...

... and your Saturday morning is gone forever, and there's not much you can do about that, but at least you have a nice animated Jupiter's moons (and shadows) page when the sickness passes and you can finally think about other things.

Tags: programming, astronomy, javascript
[ 21:10 May 23, 2009    More programming | permalink to this entry | ]

Graphical Python Programming With PyGTK

This week's Linux Planet article is another one on Python and graphical toolkits, but this time it's a little more advanced: Graphical Python Programming With PyGTK.

This one started out as a fun and whizzy screensaver sort of program that draws lots of pretty colors -- but I couldn't quite fit it all into one article, so that will have to wait for the sequel two weeks from now.

Tags: writing, python, programming
[ 19:53 May 14, 2009    More writing | permalink to this entry | ]

Upgrading without risk

Someone asked on a mailing list whether to upgrade to a new OS release when her current install was working so well. I thought I should write up how I back up my old systems before attempting a risky upgrade or new install.

On my disks, I make several relatively small partitions, maybe 15G or so (pause to laugh about what I would have thought ten or even five years ago if someone told me I'd be referring to 15G as "small"), one small shared /boot partition, a swap partition, and use the rest of the disk for /home or other shared data.

Now you can install a new release, like 9.04, onto a new partition without risking your existing install.

If you prefer upgrading rather than running the installer, you can do that too. I needed a jaunty (9.04) install to test whether a bug was fixed. But my intrepid (8.10) is working fine and I know there are some issues with jaunty, so I didn't want to risk the working install. So from Intrepid, I copied the whole root partition over to one of my spare root partitions, sda5:

mkfs.ext3 /dev/sda5
mkdir /jaunty
mount /dev/sda5 /jaunty
cp -ax / /jaunty

(that last step takes quite a while: you're copying the whole system.)

Now there are a couple of things you have to do to make that /jaunty partition work as a bootable install:

1. /dev on an ubuntu system isn't a real file system, but something magically created by the kernel and udev. But to boot, you need some basic stuff there. When you're up and running, that's stored in /dev/.static, so you can copy it like this: cp -ax /dev/.static/dev/ /jaunty/

2. /etc/fstab on the system you just created points to the wrong root partition, so you have to fix that. As root, edit /etc/fstab in your favorite editor (e.g. sudo vim /etc/fstab or whatever) and find the line for the root filesystem -- the one where the second entry on the line is /. It'll look something like this:

# /dev/sda1
UUID=f7djaac8-fd44-672b-3432-5afd759bc561  /  ext3  relatime,errors=remount-ro  0 1

The easy fix is to change that to point to your new disk partition:

# jaunty is now on /dev/sda5
/dev/sda5  /  ext3  relatime,errors=remount-ro  0 1

If you want to do it the "right", ubuntu-approved way, with UUIDs, you can get the UUID of your disk this way:

ls -l /dev/disk/by-uuid/ | grep sda5

Take the UUID (that's the big long hex number with the dashes) and put it after the UUID= in the original fstab line.

While you're editing /etc/fstab, be sure to look for any lines that might mount /dev/sda5 as something other than root and delete them or comment them out.

The following section describes how to update grub1. Since most distros now use grub2, it is out of date. With any luck, you can run update-grub and it will notice your new partition; but you might want to fiddle with entries in /etc/grub.d to label it more clearly.

Now you should have a partition that you can boot into and upgrade. Now you just need to tell grub about it. As root, edit /boot/grub/menu.lst and find the line that's booting your current kernel. If you haven't changed the file yourself, that's probably right after a line that says:

## ## End Default Options ##

It will look something like this:

title           Ubuntu 8.10, kernel 2.6.27-11-generic
uuid            f7djaac8-fd44-672b-3432-5afd759bc561
kernel          /vmlinuz-2.6.27-11-generic root=UUID=f7djaac8-fd44-672b-3432-5afd759bc561 ro
initrd          /initrd.img-2.6.27-11-generic

Make a copy of this whole stanza, so you have two identical copies, and edit one of them. (If you edit the first of them, the new OS it will be the default when you boot; if you're not that confident, edit the second copy.) Change the two UUIDs to point to your new disk partition (the same UUID you just put into /etc/fstab) and change the Title to say 9.04 or Jaunty or My Copy or whatever you want the title to be (this is the title that shows up in the grub menu when you first boot the machine).

Now you should be able to boot into your new partition. Most things should basically work -- certainly enough to start a do-release-upgrade without risking your original install.

Tags: linux, ubuntu
[ 10:44 May 13, 2009    More linux/install | permalink to this entry | ]

Making desktop recordings with recordmydesktop

Apress asked me to make some short screencast videos illustrating GIMP tips, to help advertise the second edition of Beginning GIMP.

I've never made videos (except for putting a digital camera in video mode) so it's been an interesting learning experience, and I was surprised at how easy it was in the end.

My Apress contact suggested XVidCap and Wink as possible options. XVidCap looked quite interesting but didn't seem to work in its Ubuntu incarnation. Wink worked very nicely and produced flash videos that worked in a browser with no extra fiddling ... but unfortunately when I tried plugging in a microphone, Wink didn't seem able to read it. And it seemed to slow down all my cursor movements, rather than following my actions in real-time.

But while working with those two, I stumbled across recordmydesktop. It records mouse movements in real-time and it handles the microphone too. It has several front ends available (such as gtk-recordmydesktop) but I found the basic command-line version easiest to use.

To make a video in the upper left 1024x768 section of my screen:
recordmydesktop -width 1024 -height 768 -o layermask.ogv

Since I need to make sure all the action happens within that rectangle, I made a special desktop background that has a nice, not too distracting image in just that 1024x768 rectangle. Any other windows I'm using in that desktop (such the terminal window I'm using to control recordmydesktop) stay outside that area.

recordmydesktop starts recording right away. I run through my tutorial steps, narrating as I go, and when I'm done, I move the mouse back to the terminal window where I started the recording and hit Ctrl-C, and recordmydesktop stops recording and encodes the video (which takes a while).

It saves to ogg format, .ogv. Of course, most web surfers can't view that, and youtube doesn't accept it either (at least, ogg isn't on its list of allowed formats) so I needed to translate it into something else. Youtube suggests mpeg4, so that's what I used. Luckily, I already had a mencoder incantation that some helpful person gave me a long time ago:
mencoder movie.ogv -oac pcm -ovc lavc -lavcopts vcodec=mpeg4:vqmin=2:vlelim=-4:vcelim=9:lumi_mask=0.05:dark_mask=0.01:vhq -o movie.mp4

Only one problem: the audio came out very faint and difficult to hear. I'm sure that's a problem with the microphone I'm using, a cheap OEM model that came with some computer or other many years ago -- it's been sitting in a box since I normally have no use for a microphone. But it turns out mencoder can amplify the volume, with -af volume=X where X is decibels. A little experimentation with mplayer -af volume=X on the original ogg suggested a value around 15 or 20, so the final encoding was:
mencoder movie.ogv -af volume=19 -oac pcm -ovc lavc -lavcopts vcodec=mpeg4:vqmin=2:vlelim=-4:vcelim=9:lumi_mask=0.05:dark_mask=0.01:vhq -o movie.mp4

But Apress tells me that their Windows boxen had trouble with the mp4 and they had to run it through something called "Handbrake", so maybe some other format would have worked better. Here are two other mencoder incantations I know (without the sound amplification):
mencoder movie.ogv -oac pcm -ovc lavc -o movie.divx (divx -- Windows sometimes has trouble with this too)
mencoder movie.ogv -oac pcm -ovc lavc -lavcopts vcodec=mpeg1video -o movie.mpeg (mpeg1)

It's not great cinema, but the end result is up on the Apress page for the GIMP book.

Tags: linux, video, screencasts
[ 11:14 May 12, 2009    More linux | permalink to this entry | ]

Pruning those huge Spamassassin files

During a server backup, Dave complained that my .spamassasin directory was taking up 87Mb. I had to agree, that seemed a bit excessive.

The only two large files were auto-whitelist at 42M and bayes_seen at 41M. Apparently these never get pruned by spamassassin.

Unfortunately, these are binary files, so you can't just edit them and remove the early stuff, and spamassassin doesn't seem to have any documentation on how to prune their data files. A thread on the Spamassassin Users list on managing Spamassassin data says it's okay to delete bayes_seen and it will be regenerated.

For pruning auto-whitelist, that same post suggests a program called check-whitelist that is only available in a spamassassin source tarball -- it's not installed as part of distro packages. Run this with --clean. But a search on the spamassassin.com wiki turns up an entry on AutoWhitelist that says you should use tools/sa-awlUtil instead (it doesn't say how to run it or where to get it -- presumably download a source tarball and then RTFSC -- read the source code?)

Really, I'm not sure auto whitelisting is such a good idea anyway, especially auto whitelist entries from several years ago, so I opted for a simpler solution: removing the auto-whitelist file at the same time that I removed bayes_seen. Indeed, both files were immediately generated as new mail came in, but they were now much smaller.

I've run for a few weeks since doing that, and I'm not noticing any difference in either the number of false positives or false negatives. (Both are, unfortuantely, large enough to be noticable, but that was true before the change as well.)

Tags: spam, spamassassin, email
[ 20:38 May 07, 2009    More tech/email | permalink to this entry | ]

Bay Area Wildflowers

Spring is in full swing, and all around the bay area the parks are ablaze with the colors of wildflowers -- blues and lavenders at Alum Rock, blues and yellows at Rancho San Antonio and oranges at Arastradero.

I've been shooting photos of wildflowers for years, always intending to collect them into a web page -- for my own reference (I always forget which wildflowers are which) as much as anyone else's.

At the same time, I've been gradually working on finding better ways of displaying photos on gallery pages. Most of my old pages use tables, which work fine in all browsers but don't scale very well with page size -- 4 images across may look fine in an 800 pixel wide window but look pretty silly at 1600 pixels. After playing with various CSS-based ideas for showing images and captions, I finally found the answer ("display: inline-block" is the key) on this brunildo.org CSS gallery demo. I adapted it for my site and wrote some PHP glue to generate the pages, and here's the result: Bay Area Wildflowers.

Update: Isn't it always the case? Just when you think you're done with something, you find out there's more to do. I wrote the preceding a week ago and then didn't manage to post it before leaving for a desert trip. And the desert was blooming! So here I am, ba-wildflowers site barely made public, newly back from the Mojave with a disk full of desert wildflower photos that aren't from the bay area. Looks like the "Bay Area Wildflowers" site needs to expand to a wider area ...

Tags: nature, wildflowers
[ 22:13 May 01, 2009    More nature | permalink to this entry | ]