Shallow Thoughts : : programming

Fontasia: View and categorize your fonts

We were talking about fonts again on IRC, and how there really isn't any decent font viewer on Linux that lets you group fonts into categories.

Any time you need to choose a font -- perhaps you know you need one that's fixed-width, script, cartoony, western-themed -- you have to go through your entire font list, clicking one by one on hundreds of fonts and saving the relevant ones somehow so you can compare them later. If you have a lot of fonts installed, it can take an hour or more to choose the right font for a project.

There's a program called fontypython that does some font categorization, but it's hard to use: it doesn't operate on your installed fonts, only on fonts you copy into a special directory. I never quite understood that; I want to categorize the fonts I can actually use on my system.

I've been wanting to write a font categorizer for a long time, but I always trip up on finding documentation on getting Python to render fonts. But this time, when I googled, I found jan bodnar's ZetCode Pango tutorial, which gave me all I needed and I was off and running.

Fontasia is initially a font viewer. It shows all your fonts in a list on the left, with a preview on the right. But it also lets you add categories: just type the category name in the box and click Add category and a button for that category will appear, with the current font added to it. A font can be in multiple categories.

Once you've categorized your fonts, a menu at the top of the window lets you show just the fonts in a particular category. So if you're working on a project that needs a Western-style font, show that category and you'll see only relevant fonts.

You can also show only the fonts you've categorized -- that way you can exclude fonts you never use -- I don't speak Tamil or Urdu so I don't really need to see those fonts when I'm choosing a font. Or you can show only the uncategorized fonts: this is useful when you add some new fonts to your system and need to go through them and categorize them.

I'm excited about fontasia. It's only a few days old and already used it several times for real-world font selection problems.

If you want to try it, it's here: Fontasia: View and categorize fonts.

Tags: fonts, programming, python
[ 11:20 Aug 17, 2010    More programming | permalink to this entry ]

Tee in Python

I needed a way to send the output of a Python program to two places simultaneously: print it on-screen, and save it to a file.

Normally I'd use the Linux command tee for that: prog | tee prog.out saves a copy of the output to the file prog.out as well as printing it. That worked fine until I added something that needed to prompt the user for an answer. That doesn't work when you're piping through tee: the output gets buffered and doesn't show up when you need it to, even if you try to flush() it explicitly.

I investigated shell-based solutions: the output I need is on sterr, while Python's raw_input() user prompt uses stdout, so if I could get the shell to send stderr through tee without stdout, that would have worked. My preferred shell, tcsh, can't do this at all, but bash supposedly can. But the best examples I could find on the web, like the arcane prog 2>&1 >&3 3>&- | tee prog.out 3>&- didn't work.

I considered using /dev/tty or opening a pty, but those calls only work on Linux and Unix and the program is otherwise cross-platform.

What I really wanted was a class that acts like a standard Python file object, but when you write to it it writes to two places: the log file and stderr.

I found an example of someone trying to write a Python tee class, but it didn't work: it worked for write() but not for print >>

I am greatly indebted to KirkMcDonald of #python for finding the problem. In the Python source implementing >>, PyFile_WriteObject (line 2447) checks the object's type, and if it's subclassed from the built-in file object, it writes directly to the object's fd instead of calling write().

The solution is to use composition rather than inheritance. Don't make your file-like class inherit from file, but instead include a file object inside it. Like this:

import sys

class tee :
    def __init__(self, _fd1, _fd2) :
        self.fd1 = _fd1
        self.fd2 = _fd2

    def __del__(self) :
        if self.fd1 != sys.stdout and self.fd1 != sys.stderr :
            self.fd1.close()
        if self.fd2 != sys.stdout and self.fd2 != sys.stderr :
            self.fd2.close()

    def write(self, text) :
        self.fd1.write(text)
        self.fd2.write(text)

    def flush(self) :
        self.fd1.flush()
        self.fd2.flush()

stderrsav = sys.stderr
outputlog = open(logfilename, "w")
sys.stderr = tee(stderrsav, outputlog)

And it works! print >>sys.stderr, "Hello, world" now goes to the file as well as stderr, and raw_input still works to prompt the user for input.

In general, I'm told, it's not safe to inherit from Python's built-in objects like file, because they tend to make assumptions instead of making virtual calls to your overloaded methods. What happened here will happen for other objects too. So use composition instead when extending Python's built-in types.

Tags: programming, python
[ 08:48 Apr 16, 2010    More programming | permalink to this entry ]

Configuring git colors

I spent a morning wrestling with git after writing a minor GIMP fix that I wanted to check in. Deceptively simple ideas, like "Check the git log to see the expected format of check-in messages", turned out to be easier said than done.

Part of the problem was git's default colors: colors calculated to be invisible to anyone using a terminal with dark text on a light background. And that sent me down the perilous path of git configuration.

git-config does have a manual page. But it lacks detail: you can't get from there to knowing what to change so that the first line of commits in git log doesn't show up yellow.

But that's okay, thought I: all I need to do is list the default settings, then change anything that's a light color like yellow to a darker color. Easy, right?

Well, no. It turns out there's no way to get the default settings -- because they aren't part of git's config; they're hardwired into the C code.

But you can find most of them with a seach for GIT_COLOR in the source. The most useful lines are these the ones in diff.c, builtin-branch.c and wt-status.c.

gitconfig

The next step is to translate those C lines to git preferences, something you can put in a .gitconfig. Here's a list of all the colors mentioned in the man page, and their default values -- I used "normal" for grep and interactive where I wasn't sure of the defaults.

[color "diff"]
	plain = normal
	meta = bold
	frag = cyan
	old = red
	new = green
	commit = yellow
	whitespace = normal red
[color "branch"]
	current = green
	local = normal
	remote = red
	plain = normal
[color "status"]
	header = normal
	added = red
	updated = green
	changed = red
	untracked = red
	nobranch = red
[color "grep"]
	match = normal
[color "interactive"]
	prompt = normal
	header = normal
	help = normal
	error = normal

The syntax and colors are fairly clearly explained in the manual: allowable colors are normal, black, red, green, yellow, blue, magenta, cyan and white. After the foreground color, you can optionally list a background color. You can also list an attribute, chosen from bold, dim, ul, blink and reverse -- only one at a time, no combining of attributes.

So if you really wanted to, you could say something like

[color "status"]
	header = normal blink
	added = magenta yellow
	updated = green reverse
	changed = red bold
	untracked = blue white
	nobranch = red white bold

Minimal changes for light backgrounds

What's the minimum you need to get everything readable? On the light grey background I use, I needed to change the yellow, cyan and green entries:

[color "diff"]
	frag = cyan
	new = green
	commit = yellow
[color "branch"]
	current = green
[color "status"]
	updated = green

Disclaimer: I haven't tested all these settings -- because I haven't yet figured out where all of them apply. That's another area where the manual is a bit short on detail ...

Tags: git, programming, color
[ 22:26 Feb 02, 2010    More programming | permalink to this entry ]

On Linux Planet: a simple Poker game in Python-Qt

I've written in the past about Python GUI programming using the GTK and Tk toolkits, and several KDE fans felt that I was slighting the much nicer looking Qt.

So my latest article on Linux Planet, Make Pretty GUI Apps Fast with Python-Qt, shows how to develop a little poker game using the python-qt toolkit.

I didn't want to dwell on it in the article (and didn't have space anyway), but pyqt turned out to be a bit of a pain. There's no official documentation -- or at least nothing that's obviously official -- and a lot of the examples on google are out of date because of API changes. None of the tutorial examples explain much, and they never demonstrate the practical features I'd want to do in a real app. It was surprisingly hard to come up with an application idea that worked well, looked good and was still easy to explain.

And don't get me started on this whole "Slots and signals are revolutionarily different even though they look just like the callbacks every other toolkit has used for the last three decades" meme. I'm sure there is a subtle technical difference -- but if there's a difference that matters to the average UI programmer, their documentation sure doesn't make it clear.

All that aside, PyQt (and Qt in general) does produce very pretty apps and is worth trying for that reason.

The suit images in the article are adapted from some suits I found on Wikimedia Commons (the "Naipe" set). I wanted them to look more 3-dimensional, so I applied my blobipy GIMP script as well as scaling and resizing them. I really liked those shiny-looking Tango heart and spade emblems (also on the Wikimedia Commons page) but I couldn't find a diamond or club to match.

The poker program I wrote has menus and a second round of dealing, where you can mark off the cards you want to keep. I couldn't fit all that in a 700-word article, but the complete program is available here: qpoker.py or you can get it in a tarball along with the suit images at qpoker.tar.gz.

Tags: programming, writing, qt, python-qt
[ 09:53 Jan 28, 2010    More programming | permalink to this entry ]

Displaying images from Javascript file inputs

(despite Firefox's attempts to prevent that)

My Linux Planet article last week was on printing pretty calendars. But I hit one bug in Photo Calendar. It had a HTML file chooser for picking an image ... and when I chose an image and clicked Select to use it. it got the pathname wrong every time.

I poked into the code (Photo Calendar's code turned out to be exceptionally clean and well documented) and found that it was expecting to get the pathname from the file input element's value attribute. But input.File.value was just returning the filename, foo.jpg, instead of the full pathname, /home/user/Images/yosemite/foo.jpg. So when the app tried to make it into a file:/// URL, it ended up pointing to the wrong place.

It turned out the cause was a security change in Firefox 3. The issue: it's considered a security hole to expose full pathnames on your computer to Javascript code coming from someone else's server. The Javascript could give bad guys access to information about the directory structures on your disk. That's a perfectly reasonable concern, and it makes sense to consider it as a security hole.

The problem is that this happens even when you're running a local app on your local disk. Programs written in any other language and toolkit -- a Python program using pygtk, say, or a C++ Qt program -- have access to the directories on your disk, but you can't use Javascript inside Firefox to do the same thing. The only ways to make an exception seems to be an elaborate procedure requiring the user to change settings in about:config. Not too helpful.

Perhaps this is even reasonable, given how common cross-site scripting bugs have been in browsers lately -- maybe running a local script really is a security risk if you have other tabs active. But it leaves us with the problem of what to do about apps that need to do things like choose a local image file, then display it.

And it turns out there is: a data URL. Take the entire contents of the file (ouch) and create a URL out of those contents, then set the src attribute of the image to that.

Of course, that makes for a long, horrifying, unreadable URL -- but the user never has to see that part. I suspect it's also horribly memory intensive -- the image has to be loaded into memory anyway, to display it, but is Firefox also translating all of that to a URL-legal syntax? Obviously, any real app using this technique had better keep an eye on memory consumption. But meanwhile, it fixes Photo Calendar's file button.

Here's what the code looks like:

  img = document.getElementById("pic");
  fileinput = document.input.File;
  if (img && fileinput)
    img.src = fileinput.files[0].getAsDataURL();

Here's a working minimal demo of using getAsDataURL() with a file input.

Tags: javascript, web, programming
[ 13:57 Jan 17, 2010    More programming | permalink to this entry ]

Python-GTK regression: How to catch mouse button release

We just had the second earthquake in two days, and I was chatting with someone about past earthquakes and wanted to measure the distance to some local landmarks. So I fired up PyTopo as the easiest way to do that. Click on one point, click on a second point and it prints distance and bearing from the first point to the second.

Except it didn't. In fact, clicks weren't working at all. And although I have hacked a bit on parts of pytopo (the most recent project was trying to get scaling working properly in tiles imported from OpenStreetMap), the click handling isn't something I've touched in quite a while.

It turned out that there's a regression in PyGTK: mouse button release events now need you to set an event mask for button presses as well as button releases. You need both, for some reason. So you now need code that looks like this:

drawing_area.connect("button-release-event", button_event)
drawing_area.set_events(gtk.gdk.EXPOSURE_MASK |
                        # next line wasn't needed before:
                        gtk.gdk.BUTTON_PRESS_MASK |
                        gtk.gdk.BUTTON_RELEASE_MASK )

An easy fix ... once you find it.

I filed bug 606453 to see whether the regression was intentional.

I've checked in the fix to the PyTopo svn repository on Google Code. It's so nice having a public source code repository like that! I'm planning to move Pho to Google Code soon.

Tags: programming, python, pygtk, mapping
[ 13:20 Jan 08, 2010    More programming | permalink to this entry ]

Building a Py-Webkit-GTK presentation tool

I almost always write my presentation slides using HTML. Usually I use Firefox to present them; it's the browser I normally run, so I know it's installd and the slides all work there. But there are several disadvantages to using Firefox:

• In fullscreen mode, it has a small "minimized urlbar" at the top of the screen that I've never figured out to banish -- not only is it visible to users, but it also messes up the geometry of the slides (they have to be 762 pixels high rather than 768);
• It's very heavyweight, bad when using a mini laptop or netbook;
• Any personal browsing preferences, like no-animation, flashblock or noscript, apply to slides too unless explicitly disabled, which I've forgotten to do more than once before a talk.

Last year, when I was researching lightweight browsers, one of the ones that impressed me most was something I didn't expect: the demo app that comes with pywebkitgtk (package python-webkit on Ubuntu). In just a few lines of Python, you can create your own browser with any UI you like, with a fully functional content area. Their current demo even has tabs.

So why not use pywebkitgtk to create a simple fullscreen webkit-based presentation tool?

It was even simpler than I expected. Here's the code:

#!/usr/bin/env python
# python-gtk-webkit presentation program.
# Copyright (C) 2009 by Akkana Peck.
# Share and enjoy under the GPL v2 or later.

import sys
import gobject
import gtk
import webkit

class WebBrowser(gtk.Window):
    def __init__(self, url):
        gtk.Window.__init__(self)
        self.fullscreen()

        self._browser= webkit.WebView()
        self.add(self._browser)
        self.connect('destroy', gtk.main_quit)

        self._browser.open(url)
        self.show_all()

if __name__ == "__main__":
    if len(sys.argv) <= 1 :
        print "Usage:", sys.argv[0], "url"
        sys.exit(0)

    gobject.threads_init()
    webbrowser = WebBrowser(sys.argv[1])
    gtk.main()

That's all! No navigation needed, since the slides include javascript navigation to skip to the next slide, previous, beginning and end. It does need some way to quit (for now I kill it with ctrl-C) but that should be easy to add.

Webkit and image buffering

It works great. The only problem is that webkit's image loading turns out to be fairly poor compared to Firefox's. In a presentation where most slides are full-page images, webkit clears the browser screen to white, then loads the image, creating a noticable flash each time. Having the images in cache, by stepping through the slide show then starting from the beginning again, doesn't help much (these are local images on disk anyway, not loaded from the net). Firefox loads the same images with no flash and no perceptible delay.

I'm not sure if there's a solution. I asked some webkit developers and the only suggestion I got was to rewrite the javascript in the slides to do image preloading. I'd rather not do that -- it would complicate the slide code quite a bit solely for a problem that exists only in one library.

There might be some clever way to hack double-buffering in the app code. Perhaps something like catching the 'load-started' signal, switching to another gtk widget that's a static copy of the current page (if there's a way to do that), then switching back on 'load-finished'.

But that will be a separate article if I figure it out. Ideas welcome!

Tags: programming, hack, python, web, speaking
[ 16:12 Nov 11, 2009    More programming | permalink to this entry ]

Un-unicode: translating web pages to plain ASCII

It's not that I'm a dumb provincial American, really!

I mean, okay, I am a dumb provincial American. But not completely. I know about Unicode, I know what UTF-8 and ISO-8859-1 and -15 are, I even know how to type Spanish characters like ñ and á in email (at least in Ubuntu; I can't seem to make it work in Gentoo).

The real problem is PalmOS -- I've never found any way to create Plucker files for my Palm that display anything beyond the standard ASCII character set. (I'm not clear whether to blame that on Palm or Plucker. Doesn't matter.)

So when I use a program like Sitescooper or my new FeedMe RSS reader to read daily news on my Palm, I'm forever seeing lines like this:

the weather phenomenon known as ÅoEl Ni€oÅq is

It's tiresome to try to read stuff like that.

Strangely, I've found no libraries to do this, in any language. There are lots of ways to translate from one character encoding into another -- but no way to degrade from nonASCII characters to the nearest ASCII equivalent. Googling finds lots of people asking for them -- I'm far from the only one who wants this. There are various partial hacks, but nothing ready-to-go.

Oh, well, welcome to the programming world. Time to roll my own. I started from some nice tricks I picked up in the web discussions I found, and ended up with something reasonably compact. Of course, the table of fallback characters will grow.

But my ace in the hole, this time, is that my little function has a way of logging errors. When it sees a character it doesn't recognize, it can log the character code to a file, making it easy to add a translation for that character. That was always the problem with similar hacks I'd attempted to add to mutt or plucker or sitescooper in the past: figuring out each new character and what its intended meaning was, so I could add it to the translation table.

Here it is: ununicode.

Call it like this:

import ununicode

ununicode.toascii(str, errfilename=os.path.join("/path/to/errfile"))

There's also a minimal test script provided (which will also grow with time as I accumulate good samples).

Tags: unicode, i18n, charsets, ascii, palm
[ 19:48 Oct 21, 2009    More programming | permalink to this entry ]