Shallow Thoughts : : Mar

Emacs: Typing dashes in html mode

What to do with a few extra hours in a boring motel with no net access? How about digging into fixing one of Emacs' more annoying misfeatures?

Whenever I edit an html file using emacs, I find I have to stay away from double dashes -- I can't add a phrase such as this one. If I forget and type a phrase with a double dash, then as soon as I get to the end of that line and emacs decides it's time to wrap to the next line, it "helpfully" treats the double dashes as a comment, and indents the next line to the level where the dashes were, adding another set of dashes. I've googled, I've asked on emacs IRC help channels, but there doesn't seem to be any way out. (I guess no one else ever uses double dashes in html files?)

It's frustrating: I like using double dashes now and then. And aside from the occasional boneheaded misfeature like this one, I like using emacs. But the dash problem been driving me nuts for a long time now. So I finally dug into the code to cure it.

First, the file is sgml-mode.el, so don't bother searching anything with html in the name. On my system it's /usr/share/emacs/21.4/lisp/textmodes/sgml-model.el. Edit that file and search for "--" and the first thing you'll find (well, after the file's preamble comments) is a comment in the definition of "sgml-specials" saying that if you include ?- in the list of specials, it will hork the typing of double dashes, so that's normally left out.

A clue! Perhaps some Debian or Ubuntu site file has changed sgml-specials for me, and all I need to do is change it back! So I typed

M-x describe-variable sgml-specials

to see the current setting.

Um ... it's set to "34". That's not very helpful. I haven't a clue how that translates to the list of characters I see in sgml-mode.el. Forget about that approach for now.

Searching through the file for the string "comment" got me a few more hits, and I tried commenting out various comment handling lines until the evil behavior went away. (I had to remove sgml-mode.elc first, otherwise emacs wouldn't see any changes I made to sgml-mode.el. If you haven't done much elisp hacking, the .el is the lisp source, while the .elc is a byte-compiled version which loads quicker but isn't intended to be edited by humans. For Java programmers, the .elc is sort of like a .class file.)

Commenting out these four lines did the trick:

  (set (make-local-variable 'font-lock-syntactic-keywords)
       '(("\\(<\\)! *--.*-- *\\(>\\)" (1 "!") (2 "!"))))
  ;; This will allow existing comments within declarations to be
  ;; recognized.
  (set (make-local-variable 'comment-start-skip) "\\(?:\\)?")

To regenerate the .elc file so sgml-mode will load faster, I ran emacs as root from the directory sgml-mode.el was in, and typed:

M-x byte-compile-file sgml-mode.el

All better! And now I know where to find documentation for all those useful-looking, but seemingly undocumented, keyboard shortcuts that go along with emacs' html mode. Just search in the file for html-mode-map, and you'll find all sorts of useful stuff.

For instance, that typing Ctrl-C Ctrl-C followed by various letters: u gets you an unordered list, h gets you an href tag, i an image tag, and so on, with the cursor positioned where you want to type next.

It doesn't seem to offer any basic inline formatting (like <i> or <em>), alas; but of course that's easy to add by editing the file (or maybe even in .emacs). To add an <em> tag, add this line to html-mode-map:

    (define-key map "\C-c\C-ce" 'html-em)

then add this function somewhere near where html-headline-1 and friends are defined:

(define-skeleton html-em
  "HTML emphasis tags."
  nil
  "" _ "")

Of course, you can define any set of tags you use often, not just <em>.

HTML mode in emacs should be much more fun and less painful now!

Update: If you don't want to modify the files as root, it also works fine to copy sgml-mode.el to wherever you keep personal elisp files. For instance, put them in a directory called ~/.emacs-lisp then add this to your .emacs:
(setq load-path (cons "~/.emacs-lisp/" load-path))

Tags: emacs, editors
[ 22:48 Mar 29, 2006    More linux/editors | permalink to this entry | ]

Getting rid of all those .serverauth.???? files

Dave has been complaining for a long time about the proliferation of files named .serverauth.???? (where ???? is various four-digit numbers) in his home directory under Ubuntu. I never saw them under Hoary, but now under Breezy and Dapper I'm seeing the same problem.

I spent some time investigating, with the help of some IRC friends. In fact, Carla Schroder, author of O'Reilly's Linux Cookbook, was the one who pinned down the creation of the files to the script /usr/bin/startx.

Here's the deal: if you use gdm, kdm, or xdm, you'll never see this. But for some reason, Ubuntu's startx uses a program called xauth which creates a file containing an "MIT-MAGIC-COOKIE". (Don't ask.) Under most Linux distributions, the magic cookie goes into a file called .Xauthority. The startx script checks an environment variable called XENVIRONMENT for the filename; if it's not set to something else, it defaults to $HOME/.Xenvironment.

Ubuntu's version is a little different. It still has the block of code where it checks XENVIRONMENT and sets it to $HOME/.Xenvironment if it isn't already set. But a few lines later, it proceeds to create the file under another, hardwired, name: you guessed it, $HOME/.serverauth.$$. The XENVIRONMENT variable which was checked and set is never actually used.

Programmers take note! When adding a feature to a script, please take a moment and think about what the script does, and check to see whether it already does something like what you're adding. If so, it's okay -- really -- to remove the old code, rather than leaving redundant and obsolete code blocks in place.

Okay, so why is the current code a problem? Because startx creates the file, calls xinit, then removes the file. In other words, it relies on xinit (and the X server) exiting gracefully. If anything else happens -- for example, if you shut your machine down from within X -- the startx script (which does not catch signals) can die without ever getting to the code that removes the file. So if you habitually shut down your machine from within X, you will have a .serverauth.???? file from each X session you ever shut down that way.

Note that the old setup didn't remove the file either, but at least it was always the same file. So you always have a single .Xauthority file in your home directory whether or not it's currently in use. Not much of a problem.

I wasn't seeing this under Hoary because under Hoary I ran gdm, while with Dapper, gdm would no longer log me in automatically so I had to find another approach to auto-login.

For Ubuntu users who wish to go back to the old one-file XAUTHORITY setup, there's a very simple fix: edit /usr/bin/startx (as root, of course) and change the line:

xserverauthfile=$HOME/.serverauth.$$

to read instead

xserverauthfile=$XAUTHORITY

If you want to track this issue, it's bug bug 35758.

Tags: linux, X11, ubuntu
[ 21:24 Mar 20, 2006    More linux | permalink to this entry | ]

Pandora Now in Script-Fu

Periodically I get requests from people without C compilers (which usually means Windows users) who are interested in using my GIMP plug-in Pandora, which helps with making panoramic images by loading all the images, positioning them approximately, and adding layer masks. I always regret that I don't have a Windows binary to offer people.

I've been thinking for a while that it would be easy to do a Pandora plug-in in script-fu, so that any GIMP user could run it. The only reason Pandora was written in C is that it provides a modified file selection dialog allowing you to choose the files you want in sequence. But it's not like the UI for that dialog is any great shakes (it's always been confusing, even to me, and I wrote it), and of course it uses the old gtk file selector, which has been orphaned for quite some time.

The way to work around this in script-fu is to let the user open all the images as layers in a single image using the normal Open and Open as Layers functions, then transform that image into a panorama by resizing it bigger and moving the layers to the right place. Easy! It's been on my list to do for a long time, but I didn't get motivated to write it until this morning when I wasn't doing anything else and another Windows user showed up on #gimp asking about panoramas.

I should have done it earlier. It only took an hour or two, and works as well as the old C version. It's available on the Pandora page. Feedback or bug reports appreciated!

Tags: gimp
[ 19:02 Mar 19, 2006    More gimp | permalink to this entry | ]

Linux Can't Print Envelopes?

I needed to send a formal letter, so I thought, as a nice touch, I'd print the address on the envelope rather than handwriting it.

I felt sure I remembered glabels, that nice, lightweight, straightforward label printing program, having envelope options. But nope, it doesn't have any paper sizes corresponding to envelopes. (It has a size called A10 but it's 1.0236 x 1.4567 inches, not the 4.13 x 9.50 I'd expect for a number-ten envelope.) Darn, that would have been perfect.

OpenOffice doesn't have anything in its templates list that corresponds to an envelope, nor does it have anything in the Wizards list. But googling showed me -- aha! -- that it's hidden in the Insert menu, Insert->envelope -- so if you want to create a new envelope document, create some other type of document first, go to Insert and bring up the envelope dialog, click New to get the envelope, then close the blank dummy document.

But then it doesn't offer a choice of envelope sizes, and puts the text in odd places so you have to fiddle with the margins to get the recipient and return address in normal places. Then, when you go to Preview or Printer Settings, it has forgotten all about the fact that it's doing an envelope and now tries to print in the middle of a sheet of paper. In theory you could fix this by setting the paper size to the size of the envelope -- except that OOo doesn't actually have any envelope sizes in its paper list.

Okay, let's try abiword instead. Abiword has a nice selection of paper sizes, including some common envelope sizes. Choosing A10 envelope and Landscape mode lets me compose a nice looking envelope. But then when I go to Print Preview, it turns out it wants to print it in portrait mode, with the addresses going across the short dimension of the envelope. The Print dialog offers a Paper tab which includes an Orientation dropdown, but changing that from Landscape to Portrait makes no difference: the preview still shows the addresses disappearing off the short dimension of the envelope.

I suppose I should try kword. But it depends on nine other packages, and I was tired of fighting. I gave up and wrote the address out by hand.

The next day, though, I went back to gLabels, poked around for a bit and found out about "Template Designer" (in the File menu). It's almost there ... it's easy to set up custom sizes, but it prints them in the middle of a US-letter page, rather than lined up against the edge of the printer's feeder. I'm dubious that you could feed real envelopes this way with any reliability. Still, it's a lot closer than the word processing programs could get.

Tags: linux, printing
[ 18:36 Mar 18, 2006    More linux | permalink to this entry | ]

The Notch Gang

Our little squirrel family has grown to four. Notch has returned, after being gone for over a month, and now displays nipples like Nonotchka's. Turns out they were both females!

Notch is still as graceful, strong, and dominant as ever, and hangs around keeping Nonotchka from feeding. But we've found a solution: give Notch a nut in the shell, and she will take it off to bury it, which gives us a little time to sneak some nuts to Nonotchka before Notch flies back like a furry bolt of lightning.

Sometimes the ruse doesn't work. Once Dave went outside and chased Notch across the yard, over the fence and into the cedar while I communed with Nonotchka. Dave though he had her; but Notch vanished into the cedar branches, ran down the trunk and snuck under the gate while Dave was still watching the upper branches. Nonotchka only got a few nuts that time.

But that's not all. We have two other squirrels now, both apparently youngsters (they're scruffy, skinny, slightly smaller than our established squirrels, and markedly less graceful). One has white tufts between his ears, so I'm calling him Tuft; the other doesn't have a name yet and doesn't come by very often. They're both males, and yes, it is possible to tell when they're sitting up, contrary to some web pages I've seen.

Both of the kids are very nervous about us, and won't feed when we're anywhere in sight. But they're not nervous about Notch; the three of them sometimes eat at the same time, sitting on different parts of the fence, something Notch would never allow Nonotchka to do. Dave is convinced that they're Notch's kids from last year, and that he sees a family resemblance. The two kids sometimes quarrel mildly between themselves, and chatter at each other, but only when Notch isn't around; when she is, they're respectful and submissive.

Since the Notch Gang of three all tolerate each other, this makes it difficult to get any food to Nonotchka. She's taken to coming by later in the afternoons; the kids get up early in the morning, and Notch likes coming by around lunchtime.

Dave taped a little wooden shelf at the bottom of the office door where we can put nuts. Notch and Nonotchka learned it pretty quickly: not because they're any good at finding new nut sources (it takes them forever to notice a nut that's in a place where they don't normally find any; sometimes I wonder how the species survives) but because they're both bold enough to come to the door and look in when they're hungry, and eventually they bump their noses into the nuts on the shelf. Tuft is starting to notice the door-nuts, too, and will take one, then run off when he notices he's being watched.

I was able to get some photos of Nonotchka at the door (plus a few new shots of her outside in interesting poses). I tried to photograph Tuft today but he's too nervous.

Tags: nature, squirrels, urban wildlife
[ 19:27 Mar 17, 2006    More nature/squirrels | permalink to this entry | ]

MySQL Losing its Socket!

This morning I was all ready to continue working on an ongoing web project when I discovered that mysql wasn't running.

That's funny, it was running fine yesterday! I tried /etc/init.d/mysql start, but it failed. The only error message was, "Please take a look at the syslog."

So I hied myself over to /var/log, to discover that mysql.log and mysql.err were both there, but empty.

Some poking around /etc/mysql/my.cnf revealed that logging is commented out by default, because: "# Be aware that this log type is a performance killer."

I uncommented logging and tried again, but /var/log/mysql.err remained blank, and all that was in mysql.log was three lines related basically giving its runtime arguments and the name of the socket file.

Back to the drawing board. I was well aware that I had changed the mysql settings yesterday. See, mysqld on Ubuntu likes to create its socket as /var/run/mysqld/mysqld.sock, but other apps, like Ruby, all expect to find it in /tmp/mysql.sock. It's easy enough to change Ruby's expectations. But then I found out that although the cmdline client mysql also expects the socket in /var/run/mysqld, it depends on something called mysqladmin that wants the socket in /tmp. (I may have those two reversed. No matter: the point is that you can't use the client to talk to the database because it and the program it depends on disagree about the name of the socket. This is probably a Dapper bug.)

Okay, so I had to pick one. I decided that /tmp/mysql.sock was easier to remember and more standard with non-Debian setups. I knew where to change it in the server (/etc/mysql/my.cnf is there and well commented) but the mysql client doesn't use that, and it took some googling and help from clueful friends to find out that what it wanted was a new file called /etc/my.cnf (how's that for a nice clear configuration file name?) containing one line:

socket          = /tmp/mysql.sock

That done, mysql started and ran and everything worked. Woo!

Except that it didn't the following morning after a reboot, and didn't give any error messages as to why.

Off I went on a merry chase across init files: /etc/init.d/mysql calls /etc/mysql/debian-start (which made me realize that debian has added yet another config file, debian.cnf, which has yet another copy of the line specifying the socket filename) which calls /usr/share/mysql/debian-start.inc.sh as well as calling various other programs. But those were all red herrings: the trick to debugging the problem was to run mysqld directly (not via /etc/init.d/mysql start: it actually does print error messages, but they were being hidden by using the init.d script.

The real problem turned out to be that I had changed the location of the socket file, but not the pid file, in /etc/mysql/my.cnf, which was also located by default in /var/run/mysqld. Apparently that directory is created dynamically at each boot, and it isn't created unless it's needed for the socket file (whether the pid file needs it doesn't matter). So since I'd moved the socket file to /tmp, /var/run/mysqld wasn't created, mysqld couldn't create its pid file and it bailed. Solution: edit my.cnf to use /tmp for the pid file.

Tags: programming
[ 13:29 Mar 17, 2006    More programming | permalink to this entry | ]

The Amazing Disappearing Nameservers

I updated my Ubuntu "dapper" yesterday. When I booted this morning, I couldn't get to any outside sites: no DNS. A quick look at /etc/resolv.conf revealed that it was empty -- my normal static nameservers were missing -- except for a comment indicating that the file is prone to be overwritten at any moment by a program called resolvconf.

man resolvconf provided no enlightenment. Clearly it's intended to work with packages such as PPP which get dynamic network information, but that offered no clue as to why it should be operating on a system with a static IP address and static nameservers.

The closest Ubuntu bug I found was bug 31057. The Ubuntu developer commenting in the bug asserts that resolvconf is indeed the program at fault. The bug reporter disputes this, saying that resolvconf isn't even installed on the machine. So the cause is still a mystery.

After editing /etc/resolv.conf to restore my nameservers, I uninstalled resolvconf along with some other packages that I clearly don't need on this machine, hoping to prevent the problem from happening again:

aptitude purge resolvconf ppp pppconfig pppoeconf ubuntu-standard wvdial

Meanwhile, I did some reading. It turns out that resolvconf depends on an undocumented bit of information added to the /etc/network/interfaces file: lines like

dns-nameservers 192.168.1.1 192.168.1.2

This is not documented under man interfaces, nor under man resolvconf; it turns out that you're supposed to find out about it from /usr/share/doc/resolvconf/README.gz. But of course, since it isn't a standard part of /etc/network/interfaces, no automatic configuration programs will add the DNS lines there for you. Hope you read the README!

resolvconf isn't inherently a bad idea, actually; it's supposed to replace any old "up" commands in interfaces that copy resolv.conf files into place. Having all the information in the interfaces file would be a better solution, if it were documented and supported.

Meanwhile, be careful about resolvconf, which you may have even if you never intentionally installed it. This thread on a Debian list discusses the problem briefly, and this reply quotes the relevant parts of the resolvconf README (in case you're curious but have already removed resolvconf in a fit of pique).

Tags: linux, ubuntu, networking
[ 15:22 Mar 15, 2006    More linux | permalink to this entry | ]

Quick GIMP Tips: Centering a Layer, or Preventing Centering

People are forever turning up on #gimp to ask (quite reasonably) how to center a layer, since GIMP offers no built-in way to do that.

There are Python and Perl scripts around somewhere (and it's easy to write, a great project if you're thinking about learning how to write GIMP scripts in any language). And the new Align tool can probably center, for those using GIMP 2.3, but the tool is a bit difficult for most people to figure out (fear not, the UI is still being developed).

But for those who want a quick solution:

Center a Layer:

Cut, then paste. The pasted layer comes out centered. (Unfortunately this loses text information, so if it's a text layer this isn't an ideal solution.)

Paste a layer on top of a copy of itself:

Do a Layer to Imagesize before copying. Then copies of the layer will overlap the original.

Tags: gimp
[ 19:51 Mar 14, 2006    More gimp | permalink to this entry | ]

Ruby on Rails on Ubuntu

Back when I laboriously installed Ruby and Rails on Ubuntu "Hoary Hedgehog" (which involved basically ignoring all the Ubuntu packages and building everything, including Ruby itself, from source), I was cheered by the notes in Ubuntu's forums and bugzilla indicating that as of the next release ("Breezy Badger") all the right versions would be there, and all this source compilation would no longer be necessary.

I didn't get around to trying it until today. Breezy and its successor "Dapper Drake" do indeed have a rails package as well as a Ruby package, and I happily installed them. All looked great -- until I actually tried to use them on a real-world application. It turns out that the Ruby and Rails packages don't include gems, Ruby's package manager (similar to the CPAN system familiar to Perl programmers). And gems is required for doing anything useful in Rails.

Drat! After several false starts, I eventually found the instructions on this page. Except that installs way more than seems necessary for what I need to do, and if you copy/paste lines from that page you may end up with a bunch of packages you don't want, like an out of date version of mysql.

So here are simplified instructions for using Ruby on Rails on Ubuntu Breezy or Dapper.

As yourself:

wget http://rubyforge.org/frs/download.php/5207/rubygems-0.8.11.tgz
tar zxvf rubygems-0.8.11.tgz

As root:

cd rubygems-0.8.11
ruby setup.rb
gem install rubygems-update
gem install rails

Say yes to all dependency questions during the gem install of rails. Add your web server and database of choice (you probably already have them installed, anyway) and you should be good to go.

You may note that the page I referenced tells you to install two versions of rails: the Ubuntu package plus the one from gems. At least on Dapper, you don't need both. Installing rails pulls in the packages:
irb irb1.8 libpgsql-ruby1.8 libreadline-ruby1.8 libredcloth-ruby1.8 libruby1.8 rake rdoc rdoc1.8 ruby ruby1.8
I haven't experimented with which of these packages are and are not needed. If you run into problems, some set of packages from this list may solve them.

Update: it seems that none of these are required. Many of them are dummy packages anyway, which contain no files related to the actual package name. For instance, the rake package contains, not rake, but a single bash completion file related to rake. So you should be fine ignoring all of them, installing just Ruby and nothing else.

(I filed bug 34840 requesting that Ubuntu ship a usable version of Rails, since it didn't seem to be filed already.)

Tags: programming
[ 22:56 Mar 13, 2006    More programming | permalink to this entry | ]

Nonotchka has had her litter

Nonotchka has had her litter. Or at least she lost the tummy and regained her old svelte and graceful form as of yesterday afternoon. Of course, we haven't seen any squirrelets; she'll have them stashed away in a nest somewhere safe.

We're slightly worried about her. She came to eat today as usual (ravenously: she ate ten hazelnuts all at once then took several more away to bury), and although she seemed friendly and energetic, she left blood spots on Dave's jeans. I hope this is just some sort of normal postpartum condition and not an injury. She didn't seem to be in pain. (I get this from Dave; I was away when she made her visit. She's definitely spending less time here now that she has a family to take care of.)

So we'll keep an eye on her, make sure she's well fed and hope that she's okay and that in a few months she might start bringing the kids by. Apparently grey squirrels nurse for an amazing three months before they're ready to go out on their own. There are usually four to a litter.

(Update the following day: She seems fine. She's still energetic and hungry, and there's been no more blood.)

Meanwhile, Notch is gone. We haven't seen him at all since getting back from our trip. We're getting occasional visits from a new squirrel: scruffy, young-looking and not terribly well coordinated. Dave thinks the newcomer is a male. He's confused about nuts, or well fed from someone else's yard: he'll sniff at a hazelnut in the shell then leave it where it lies. Perhaps he just doesn't like hazelnuts and is holding out for a walnut.

It seems odd that this scrawny newcomer could have chased the burly, graceful and confident Notch away from his territory. My guess is that Notch decided there was some other yard he liked better, since even before the trip we'd been seeing him only infrequently.

Tags: nature, squirrels, urban wildlife
[ 23:30 Mar 07, 2006    More nature/squirrels | permalink to this entry | ]

Nonotchka is going to be a mom!

We went away for a week, to visit family for my grandmother's 100^th birthday (yay, Grandma!) Of course, before we left we made sure our squirrels had lots of nuts buried, so they weren't dependent on the shelled nuts we've been feeding them.

When we got back, Nonotchka wasted little time in visiting us, and she's just as friendly as ever (to someone with a walnut in hand). But there are some other changes. At first, we weren't sure if she seemed fatter; but eventually we saw her from angles that left no doubt. And her belly fur has changed; instead of the brownish grey, now it's white like Notch's, except for six dark spots arranged in pairs down her abdomen.

Looks like we guessed right about Nonotchka's gender (well, we had a 50% chance) and she's going to be a mom!

I hope we get to see the baby squirrels when they're old enough to leave the nest. Maybe she'll even bring them by when they're old enough to be weaned.

We haven't seen Notch at all since we got back. I hope he's all right. He'd been spending a lot of time across the street anyway, so perhaps he's found a territory he likes better than our yard.

Tags: nature, squirrels, urban wildlife
[ 12:29 Mar 02, 2006    More nature/squirrels | permalink to this entry | ]