Shallow Thoughts

Using (or not) an Apple Cinema Display on a non-Apple

Dave and I recently acquired a lovely trinket from a Mac-using friend: an old 20-inch Apple Cinema Display.

I know what you're thinking (if you're not a Mac user): surely Akkana's not lustful of Apple's vastly overpriced monitors when brand-new monitors that size are selling for under $200!

Indeed, I thought that until fairly recently. But there actually is a reason the Apple Cinema displays cost so much more than seemingly equivalent monitors -- and it's not the color and shape of the bezel.

The difference is that Apple cinema displays are a technology called S-IPS, while normal consumer LCD monitors -- those ones you see at Fry's going for around $200 for a 22-inch 1680x1050 -- are a technology called TN. (There's a third technology in between the two called S-PVA, but it's rare.)

The main differences are color range and viewing angle. The TN monitors can't display full color: they're only 6 bits per channel. They simulate colors outside that range by cycling very rapidly between two similar colors (this is called "dithering" but it's not the usual use of the term). Modern TN monitors are astoundingly fast, so they can do this dithering faster than the eye can follow, but many people say they can still see the color difference. S-IPS monitors show a true 8 bits per color channel.

The viewing angle difference is much easier to see. The published numbers are similar, something like 160 degrees for TN monitors versus 180 degrees for S-IPS, but that doesn't begin to tell the story. Align yourself in front of a TN monitor, so the colors look right. Now stand up, if you're sitting down, or squat down if you're standing. See how the image suddenly goes all inverse-video, like a photographic negative only worse? Try that with an S-IPS monitor, and no matter where you stand, all that happens is that the image gets a little less bright.

(For those wanting more background, read TN Film, MVA, PVA and IPS – Which one's for you?, the articles on TFT Central, and the wikipedia article on LCD technology.)

Now, the comparison isn't entirely one-sided. TN monitors have their advantages too. They're outrageously inexpensive. They're blindingly fast -- gamers like them because they don't leave "ghosts" behind fast-moving images. And they're very power efficient (S-IPS monitors, are only a little better than a CRT). But clearly, if you spend a lot of time editing photos and an S-IPS monitor falls into your possession, it's worth at least trying out.

But how? The old Apple Cinema display has a nonstandard connector, called ADC, which provides video, power and USB1 all at once. It turns out the only adaptor from a PC video card with DVI output (forget about using an older card that supports only VGA) to an ADC monitor is the $99 adaptor from the Apple store. It comes with a power brick and USB plug.

Okay, that's a lot for an adaptor, but it's the only game in town, so off I went to the Apple store, and a very short time later I had the monitor plugged in to my machine and showing an image. (On Ubuntu Hardy, simply removing xorg.conf was all I needed, and X automatically detected the correct resolution. But eventually I put back one section from my old xorg.conf, the keyboard section that specifies "XkbOptions" to be "ctrl:nocaps".)

And oh, the image was beautiful. So sharp, clear, bright and colorful. And I got it working so easily!

Of course, things weren't as good as they seemed (they never are, with computers, are they?) Over the next few days I collected a list of things that weren't working quite right:

• The Apple display had no brightness/contrast controls; I got a pretty bad headache the first day sitting in front of that full-brightness screen.
• Suspend didn't work. And here when I'd made so much progress getting suspend to work on my desktop machine!
• While X worked great, the text console didn't.

The brightness problem was the easiest. A little web searching led me to acdcontrol, a commandline program to control brightness on Apple monitors. It turns out that it works via the USB plug of the ADC connector, which I initially hadn't connected (having not much use for another USB 1.1 hub). Naturally, Ubuntu's udev/hal setup created the device in a nonstandard place and with permissions that only worked for root, so I had to figure out that I needed to edit /etc/udev/rules.d/20-names.rules and change the hiddev line to read:

KERNEL=="hiddev[0-9]*", NAME="usb/%k", GROUP="video", MODE="0660"

That did the trick, and after that acdcontrol worked beautifully.

On the second problem, I never did figure out why suspending with the Apple monitor always locked up the machine, either during suspend or resume. I guess I could live without suspend on a desktop, though I sure like having it.

The third problem was the killer. Big deal, who needs text consoles, right? Well, I use them for debugging, but what was more important, also broken were the grub screen (I could no longer choose kernels or boot options) and the BIOS screen (not something I need very often, but when you need it you really need it).

In fact, the text console itself wasn't a problem. It turns out the problem is that the Apple display won't take a 640x480 signal. I tried building a kernel with framebuffer enabled, and indeed, that gave me back my boot messages and text consoles (at 1280x1024), but still no grub or BIOS screens. It might be possible to hack a grub that could display at 1280x1024. But never being able to change BIOS parameters would be a drag.

The problems were mounting up. Some had solutions; some required further hacking; some didn't have solutions at all. Was this monitor worth the hassle? But the display was so beautiful ...

That was when Dave discovered TFT Central's search page -- and we learned that the Dell 2005FPW uses the exact same Philips tube as the Apple, and there are lots of them for sale used,. That sealed it -- Dave took the Apple monitor (he has a Mac, though he'll need a solution for his Linux box too) and I bought a Dell. Its image is just as beautiful as the Apple (and the bezel is nicer) and it works with DVI or VGA, works at resolutions down to 640x480 and even has a powered speaker bar attached.

Maybe it's possible to make an old Apple Cinema display work on a Mac. But it's way too much work. On a PC, the Dell is a much better bet.

Tags: linux, tech, photo, graphics, monitor, S-IPS, TN, ADC, DVI
[ 20:57 Nov 15, 2008    More tech | permalink to this entry ]

Spamassassin false positives: obsolete rules on Etch

I checked my Spam Assassin "probably" folder for the first time in too long, and discovered that I was getting tons of false positives, perfectly legitimate messages that were being filed as spam.

A little analysis of the X-Spam-Status: headers showed that all of the misfiled messages (and lots of messages that didn't quite make it over the threshold) were hitting a rule called DNS_FROM_SECURITYSAGE.

It turned out that this rule is obsolete and has been removed from Spam Assassin, but it hasn't yet been removed from Debian, at least not from Etch.

So I filed a Debian bug. Or at least I think I did -- I got an email acknowledgement from submit@bugs.debian.org but it didn't include a bug number and Debian's HyperEstraier based search engine linked off the bug page doesn't find it (I used reportbug).

Anyway, if you're getting lots of SECURITYSAGE false hits, edit /usr/share/spamassassin/20_dnsbl_tests.cf and comment out the lines for DNS_FROM_SECURITYSAGE and, while you're at it, the lines for RCVD_IN_DSBL, which is also obsolete. Just to be safe, you might also want to add
score DNS_FROM_SECURITYSAGE 0
in your .spamassassin/user_prefs (or equivalent systemwide file) as well.

Now if only I could figure out why it was setting FORGED_RCVD_HELO and UNPARSEABLE_RELAY on messages from what seems to be perfectly legitimate senders ...

Tags: linux, spam, bugs
[ 21:54 Nov 12, 2008    More linux | permalink to this entry ]

Linux Planet: Why Firefox Rocks on Linux

My latest Linux Planet article, Why Firefox Rocks on Linux, discusses Linux-specific Firefox shortcuts involving the middle mouse button, the URLbar and the scrollbar. It's getting good Diggs, too, and comments from people who found the tips helpful, which is great. A lot of people don't know about some of these great Linux time-savers, but these are the sort of things that make me love Linux and stick with it even when it gets frustrating. I hate to think of people missing out just because there's no obvious way to discover some of the shortcuts!

Tags: writing, mozilla, firefox, linux
[ 20:44 Nov 06, 2008    More writing | permalink to this entry ]

More fun with regexps: Adding "[no output]" in shell logs

Someone on LinuxChix' techtalk list asked whether she could get tcsh to print "[no output]" after any command that doesn't produce output, so that when she makes logs to help her co-workers, they will seem clearer.

I don't know of a way to do that in any shell (the shell would have to capture the output of every command; emacs' shell-mode does that but I don't think any real shells do) but it seemed like it ought to be straightforward enough to do as a regular expression substitute in vi. You're looking for lines where a line beginning with a prompt is followed immediately by another line beginning with a prompt; the goal is to insert a new line consisting only of "[no output]" between the two lines.

It turned out to be pretty easy in vim. Here it is:

:%s/\(^% .*$\n\)\(% \)/\1[no results]\r\2/

Explanation:

:

starts a command

%

do the following command on every line of this file

s/

start a global substitute command

\(

start a "capture group" -- you'll see what it does soon

^

match only patterns starting at the beginning of a line

%

look for a % followed by a space (your prompt)

.*

after the prompt, match any other characters until...

$

the end of the line, after which...

\n

there should be a newline character

\)

end the capture group after the newline character

\(

start a second capture group

%

look for another prompt. In other words, this whole

expression will only match when a line starting with a prompt

is followed immediately by another line starting with a prompt.

\)

end the second capture group

/

We're finally done with the mattern to match!

Now we'll start the replacement pattern.

\1

Insert the full content of the first capture group

(this is also called a "backreference" if you want

to google for a more detailed explanation).

So insert the whole first command up to the newline

after it.

[no results]

After the newline, insert your desired string.

\r

insert a carriage return here (I thought this should be

\n for a newline, but that made vim insert a null instead)

\2

insert the second capture group (that's just the second prompt)

/

end of the substitute pattern

Of course, if you have a different prompt, substitute it for "% ". If you have a complicated prompt that includes time of day or something, you'll have to use a slightly more complicated match pattern to match it.

Tags: regexp, shells, linux, editors
[ 13:34 Oct 12, 2008    More linux/editors | permalink to this entry ]

Getting rid of .sudo_as_admin_successful

Ever been annoyed by the file in your home directory, .sudo_as_admin_successful? You know, the one file with the name so long that it alone is responsible for making ls print out your home directory in two columns rather than three or four? And if you remove it, it comes right back after the next time you run sudo?

Here's what's creating it (credit goes to Dave North for figuring out most of this).

It's there because you're in the group admin, and it's there to turn off a silly bash warning. It's specific to Ubuntu (at least, Fedora doesn't do it). Whenever you log in under bash, if bash sees that you're in the admin group in /etc/groups, it prints this warning:

To run a command as administrator (user "root"), use "sudo ".
See "man sudo_root" for details.

Once you sudo to root, if you're in the admin group, sudo creates an empty file named .sudo_as_admin_successful in your home directory. That tells bash, the next time you log in, not to print the stupid warning any more. Sudo creates the file even if your login shell isn't bash and so you would never have seen the stupid warning. Hey, you might some day go back to bash, right?

If you want to reclaim your ls columns and get rid of the file forever, it's easy: just edit /etc/group and remove yourself from the admin group. If you were doing anything that required being in the admin group, substitute another group with a different name.

Tags: linux, bash, sudo, annoyances, ubuntu
[ 17:33 Oct 09, 2008    More linux | permalink to this entry ]

Console Setup in Ubuntu

Dave and I were testing some ways of speeding up the booting process, which is how he came to be looking at my Vaio's console with no X running. "What's wrong with that font?" he asked.

I explained how Ubuntu always starts the boot process with a perfectly fine font, then about 80% of the way through boot it deliberately changes it to a garbled, difficult to read that was clearly not designed for 1024x761. Been meaning for ages to figure out how to fix it, never spent the time ... Okay, it said "Setting up console font and keymap" just before it changes the font. That message should be easy to find. Maybe I should take a few minutes now and look into it.

The message comes from /etc/init.d/console-setup, which runs a program called setupcons, which has a man page. setupcons uses /etc/default/console-setup which includes the following section:

# Valid font faces are: VGA (sizes 8, 14 and 16), Terminus (sizes
# 12x6, 14, 16, 20x10, 24x12, 28x14 and 32x16), TerminusBold (sizes
# 14, 16, 20x10, 24x12, 28x14 and 32x16), TerminusBoldVGA (sizes 14
# and 16), Fixed (sizes 13, 14, 15, 16 and 18), Goha (sizes 12, 14 and
# 16), GohaClassic (sizes 12, 14 and 16).
FONTFACE="Fixed"
FONTSIZE="16"

The hard part of changing the console font in the past has always been finding out what console fonts are available. So having a list right there in the comment is a big help. Okay, let's try changing it to Terminus and running setupcons again. Nope, error message. How about VGA? Success, looks fine. That was easy!

But while I was in that file, what about the keymap? That's another thing I've been meaning to fix for ages ... under Debian, Redhat and earlier Ubuntu versions I had a .kmap.gz console map that turned my capslock key into a Control key (the way God intended). But Ubuntu changed things all around so the old fix didn't work any more.

I found a thread from December from someone who wanted to make the exact same change, for the same reason, but the only real advice in the thread involved an elaborate ritual involving defining keymaps for X and Gnome then applying them to the console. Surely there was a better way.

It seemed pretty clear that /etc/console-setup/boottime.kmap.gz was the keymap it was using. I tried substituting my old keymap, but since I'd written it to inherit from other keymaps that no longer existed, loadkeys can't use it. Eventually I just gunzipped boottime.kmap.gz, found the Caps Lock key (keycode 29), replaced all the Caps_Locks with Controls and gzipped it back up again. And it worked!

Gary Vollink has a more detailed description, and the process hasn't changed much since his page on Getting "Control" on the "Caps Lock".

Another gem linked to from the Ubuntu thread was this excellent article on keyboard layouts under X by Daniel Paul O'Donnell. It's not relevant to the problem of setting the console keymap, but it looks like a very useful reference on how various international character input methods work under X.

Tags: linux, ubuntu, fonts
[ 21:33 Oct 04, 2008    More linux | permalink to this entry ]

Linux Planet: Linux Astronomy part III: Stellarium and Celestia

Part III in the Linux Astronomy series on Linux Planet covers two 3-D apps, Stellarium and Celestia.

Writing this one was somewhat tricky because the current Ubuntu, "Hardy", has a bug in its Radeon handling and both these apps lock my machine up pretty quickly, so I went through a lot of reboot cycles getting the screenshots. (I found lots of bug reports and comments on the web, so I know it's not just me.) Fortunately I was able to test both apps and grab a few screenshots on Fedora 8 and Ubuntu "Feisty" without encountering crashes. (Ubuntu sure has been having a lot of trouble with their X support lately! I'm going to start keeping current Fedora and Suse installs around for times like this.)

Tags: writing, astronomy, linux, ubuntu, bugs
[ 21:10 Sep 22, 2008    More writing | permalink to this entry ]

Linux Planet: Linux Astronomy part II: XEphem

I have a new article on XEphem on Linux Planet, following up to the KStars article two weeks ago: Viewing the Night Sky with Linux, Part II: Visit the Planets With XEphem.

Tags: writing, astronomy, linux
[ 10:50 Sep 12, 2008    More writing | permalink to this entry ]

Useful shell pipeline: Who checks in how much?

I wanted to get a list of who'd been contributing the most in a particular open source project. Most projects of any size have a ChangeLog file, in which check-ins have entries like this:

2008-08-26  Jane Hacker  <hacker@domain.org>

        * src/app/print.c: make sure the Portrait and Landscape
        * buttons update according to the current setting.

I wanted to take each entry, save the name of the developer checking in, then eventually count the number of times each name occurs (the number of times that developer checked in) and print them in order from most check-ins to least.

Getting the names is easy: for check-ins in the last 9 years, I just want the lines that start with "200". (Of course, if I wanted earlier check-ins I could make the match more general.)

grep "^200" ChangeLog

But now I want to trim the line so it includes only the contributor's name. A bit of sed geekery can do that: the date is a fixed format (four characters, a dash, two, dash, two, then two spaces, so "^....-..-.. " matches that pattern.

But I want to remove the email address part too (sometimes people use different email addresses when they check in). So I want a sed pattern that will match something at the front (to discard), something in the middle (keep that part) and something at the end (discard).

Here's how to do that in sed:

grep "^200" ChangeLog | sed 's/^....-..-..  \(.*\)<.*$/\1/'

In English, that says: "For each line in the ChangeLog that starts with 200, find a pattern at the beginning consisting of any four characters, a dash, two characters, dash, two characters, dash, and two spaces; then immediately after that, save all characters up to a < symbol; then throw away the < and any characters that follow until the end of the line."

That works pretty well! But it's not quite right: it includes the two spaces after the name as part of the name. In sed, \s matches any space character (like space or tab). So you'd think this should work:

grep "^200" ChangeLog | sed 's/^....-..-..  \(.*\)\s+<.*$/\1/'

\s+ means it will require that at least one and maybe more space characters immediately before the < are also discarded. But it doesn't work. It turns out the reason is that the \(.*\) expression is "greedier" than the \s+: so the saved name expression grabs the first space, leaving only the second to the \s+.

The way around that is to make the name expression specify that it can't end with a space. \S is the term for "anything that's not a space character"; so the expression becomes

grep "^200" ChangeLog | sed 's/^....-..-..  \(.*\S\)\s\+<.*$/\1/'

(the + turned out to need a backslash before it).

We have the list of names! Add a | sort on the end to sort them alphabetically -- that will make sure you get all the "Jane Hacker" lines listed together. But how to count them? The Unix program most frequently invoked after sort is uniq, which gets rid of all the repeated lines. On a hunch, I checked out the man page, man uniq, and found the -c option: "prefix lines by the number of occurrences". Perfect! Then just sort them by the number, from largest to smallest:

grep "^200" ChangeLog | sed 's/^....-..-..  \(.*\S\)\s+<.*$/\1/' | sort | uniq -c | sort -rn

And we're done!

Now, this isn't perfect since it doesn't catch "Checking in patch contributed by susan@otherhost.com" attributions -- but those aren't in a standard format in most projects, so they have to be handled by hand.

Disclaimer: Of course, number of check-ins is not a good measure of how important or productive someone is. You can check in a lot of one-line fixes, or you can write an important new module and submit it for someone else to merge in. The point here wasn't to rank developers, but just to get an idea who was checking into the tree and how often.

Well, that ... and an excuse to play with nifty Linux shell pipelines.

Tags: shells, linux, pipelines, regexp
[ 11:12 Aug 31, 2008    More linux | permalink to this entry ]

Writing for Linux Planet: Stargazing with KStars

I have an article on Linux Planet! The first of many, I hope. At least the first of a short series on Linux astronomy programs, starting with the one that's easiest to use: KStars. It's oriented toward binocular observing, with suggestions for good targets for beginners.

Viewing the Night Sky with Linux, Part I: KStars

Tags: writing, astronomy, linux
[ 21:46 Aug 28, 2008    More writing | permalink to this entry ]

Back from OSCON

No postings for a while -- I was too tied up with getting ready for OSCON, and now that it's over, too tied up with catching up with stuff that gotten behind.

A few notes about OSCON:

It was a good conference -- lots of good speakers, interesting topics and interesting people. Best talks: anything by Paul Fenwick, anything by Damian Conway.

The Arduino tutorial was fun too. It's a little embedded processor with a breadboard and sockets to control arbitrary electronic devices, all programmed over a USB plug using a Java app. I'm not a hardware person at all (what do those resistor color codes mean again?) but even I, even after coming in late, managed to catch up and build the basic circuits they demonstrated, including programming them with my laptop. Very cool! I'm looking forward to playing more with the Arduino when I get a spare few moments.

The conference's wi-fi network was slow and sometimes flaky (what else is new?) but they had a nice touch I haven't seen at any other conference: Wired connections, lots of them, on tables and sofas scattered around the lounge area (and more in rooms like the speakers' lounge). The wired net was very fast and very reliable. I'm always surprised I don't see more wired connections at hotels and conferences, and it sure came in handy at OSCON.

The AV staff was great, very professional and helpful. I was speaking first thing Monday morning (ulp!) so I wanted to check the room Sunday night and make sure my laptop could talk to the projector and so forth. Everything worked fine.

Portland is a nice place to hold a convention -- the light rail is great, the convention center is very accessible, and street parking isn't bad either if you have a car there.

Dave went with me, so it made more sense for us to drive. The drive was interesting because the central valley was so thick with smoke from all the fires (including the terrible Paradise fire that burned for so long, plus a new one that had just started up near Yosemite) that we couldn't see Mt Shasta when driving right by it. It didn't get any better until just outside of Sacramento. It must have been tough for Sacramento valley residents, living in that for weeks! I hope they've gotten cleared out now.

I finally saw that Redding Sundial bridge I've been hearing so much about. We got there just before sunset, so we didn't get to check the sundial, but we did get an impressive deep red smoky sun vanishing into the gloom. Photos here.

End of my little blog-break, and time to get back to scrambling to get caught up on writing and prep for the GetSET Javascript class for high school girls. Every year we try to make it more relevant and less boring, with more thinking and playing and less rote typing. I think we're making progress, but we'll see how it goes next week.

Tags: oscon08, conferences, linux, travel, portland, hardware
[ 22:00 Aug 04, 2008    More conferences | permalink to this entry ]

Custom ringtones on a Motorola phone, from Linux

I decided to stick a tentative toe into the current millennium and get myself a cellphone.

I sense your shock and amazement -- from people who know me, that I would do such a thing, and from everybody else at the concept that there's anybody in 2008 who didn't already have one.

I really don't think cellphones are evil, honest! (Except in the hands of someone driving a car -- wouldja please just put the phone down and pay attention to the friggin' road?) The truth is that I just don't much like talking on the phone, and generally manage fine with email. The land-line phone works fine for the scant time I spend on the phone, and I have to have the land line anyway (as part of the DSL package) so why pay another monthly bill for a second phone?

Prepaid plans looked like just the ticket, and that's what I got. With a cute little Motorola V195s. New toy! Rock! It can take custom MP3 ringtones and Java games ... but of course I don't want theirs, I want to make my own. So I wanted to talk to the phone from Linux.

The charger plug was a familiar shape -- looked a lot like a standard mini USB connector. Could the hardware be that easy? Sure enough, it's a standard mini USB. Kudos to Motorola for making that so easy! Now what about software?

My initial web searches led me down a false trail paved with programs like wammu and gnokii. I learned that I needed to enable ACM in my kernel (that's the modem protocol most cellphones use over USB), so as long as I was building a new kernel anyway, I grabbed the latest tarball from kernel.org (2.6.25.7). With that done, I was able to talk to the phone with gnokii, but the heavily Nokia-oriented program didn't show me much that looked useful.

Moto4lin is the answer

I set the project aside for a while. But half a week later while looking for something else, I stumbled across moto4lin, which turned out to be exactly what I needed. I had to run as root, or else when I try to connect, it prints on stderr:

sendControl Error:[error sending control message: Operation not permitted]

) but I'm sure that can be solved somehow.

So run as root, click Connect, click File Manager if you're not already in that mode, then click Update List and it reads the files. Once they're there, you can click around in the folder list on the left looking for the audio files (on my phone, they're in a directory called audio somewhere under C, not A). Excellent!

Creating a ringtone leads to a kernel debugging digression

Okay, now I needed a ringtone. I wanted to use a bit of birdsong, so I loaded one of the tracks I use for tweet into Audacity and fiddled semi-randomly until I figured out how to cut and save a short clip. It would only save as WAV, but lame clip.wav clip.mp3 solved that just fine.

(Update: the easiest way is to select the clip you want, then do File->Export Selection...)

Except ... somewhere along the way, the clips stopped playing. I couldn't even play the original ogg track from tweet. It *looked* like it was playing ... it found the track, printed information about it, showed a running time-counter for the appropriate amount of time ... but made no sound.

It eventually turned out that the problem was that shiny new 2.6.25.7 kernel I'd downloaded. A bug introduced in 2.6.24 to the ymfpci sound card driver makes Yamaha sound cards unable to play anything with a bitrate of 44100 (which happens to be the typical CD bitrate). After a lot of debugging I eventually filed bug 10963 with a patch that reverts the old, working code from 2.6.23.17.

Ringtone success

Okay, a typical open source digression. But while I was still trying to track down the kernel bug, I meanwhile found this Razr page that tipped me off that I might need a different bitrate for ringtones anyway. So I converted it with:

lame -b 40 mock.wav mock.mp3

(which also made it playable on the new kernel.) I also found some useful information in the lengthy Ubuntu forums discussion of moto4lin.

In the end, I was able to transfer the file easily to the motorola phone, and to use it as my nifty new ringtone. Success! Too bad nobody ever calls me and this phone is mostly for outgoing calls ...

Now to look for some fun Java apps.

Tags: cellphone, usb, linux
[ 19:27 Jun 22, 2008    More linux | permalink to this entry ]

Fixing scanner permissions on Hardy

Dave needed something scanned. Oh, good! The first use of a scanner under a new distro is always an interesting test. Though the last few Ubuntu releases have been so good about making scanners "just work" that I was beginning to take scanners for granted. "Sure, no problem," I told Dave, taking the sketch he gave me.

Ha! Famous last words. For Hardy, I guess the Ubuntu folks decided that users had had it too easy for a while and it was time to throw us a challenge. Under Hardy, scanning works fine as root, but normal users can't access the scanner. sane-find-scanner sees the scanner, but xsane and the xsane-gimp plug-in can't talk to it (except as root).

It turns out the code for noticing you plugged in a scanner and setting appropriate permissions (like making it group "scanner") has been removed from udev, the obvious place for it ... and moved into hal. Except, you guessed it, whatever hal is supposed to be doing isn't working, so the device's group is never set to "scanner" to make it accessible to non-root users. Lots of people are hitting this and filing bugs (search for scanner permissions), in particular bug 121082 and bug 217571.

Fortunately, the fix is quite easy if you have a copy of your old gutsy install: just copy /etc/udev/rules.d/45-libsane.rules from gutsy to the same place on hardy. (If you don't have your gutsy udev rules handy, I attached the file to the latter of the two bugs I linked above.)

Then udev will handle your scanner just like it used to, and you don't have to wait for the hal developers to figure out what's wrong with the new hal rules.

Tags: linux, ubuntu, scanner, udev, hal
[ 15:56 May 22, 2008    More linux | permalink to this entry ]

How to set your time zone

My laptop's clock has been drifting. I suspect the clock battery is low (not surprising on a 7-year-old machine). But after an hour of poking and prodding, I've been unable to find a way to expose the circuit board under the keyboard, either from the top (keyboard) side -- though I know how to remove individual keycaps, thanks to a reader who sent me detailed instructions a while back (thanks, Miles!) -- or the bottom. Any expert on Vaio SR laptops know how this works?

Anyway, that means I have to check and reset the time periodically. So this morning I did a time check and found it many hours off. No, wait -- actually it was pretty close; it only looked like it was way off because the system had suddenly decided it was in UTC, not PDT. But how could I change that back?

I checked /etc/timezone -- sure enough, it was set to UTC. So I changed that, copying one from a debian machine -- "US/Pacific", but that didn't do it, even after a reboot.

I spent some time reading man hwclock -- there's a lot of good reading in that manual page, about the relation between the system (kernel) clock and the hardware clock. Did you know that you're not supposed to use the date command to set the system time while the system is running? Me neither -- I do that all the time. Hmm. Anyway, interesting reading, but nothing useful about the system time zone.

It has an extensive SEE ALSO list at the end, so I explored some of those documents. /usr/share/doc/util-linux/README.Debian.hwclock is full of lots of interesting information, well worth reading, but it didn't have the answer. man tzset sounded promising, but there was no such man page (or program) on my system. Just for the heckofit, I tried typing tz[tab] to see if I had any other timezone-related programs installed ... and found tzselect. And there was the answer, added almost as an afterthought at the end of the manual page:

Note that tzselect will not actually change the timezone for you. Use 'dpkg-reconfigure tzdata' to achieve this.

Sure enough, dpkg-reconfigure tzdata let me set the time zone. And it even seems to be remembered through a reboot.

Tags: linux, debian, ubuntu, vaio
[ 10:04 May 16, 2008    More linux | permalink to this entry ]

Two font mysteries solved

This has been a good week for fonts: two longstanding mysteries solved.

The first concerns the bitstream vera sans mono I've been using as a terminal font in apps like rxvt and xterm. I'd been specifying it in ~/.Xdefaults like this:
XTerm*font: -bitstream-bitstream vera sans mono-bold-r-normal-*-12-*-*-*-*-*-iso10646-1

The mystery is that I'd noticed that in xterm, the font looked slightly different -- slightly uglier -- than in rxvt (both apps use the same X class name of XTerm). It was hard to put my finger on what was different -- the shape of all the letters looked the same, but it just seemed a little more ragged, and a little less compact, in xterm. I figured it was just a minor difference in their drawing code, or something.

Well, I was fiddling with fonts (trying to get the new-to-me "Inconsolata" font working) and I noticed that iso10646 bit. I didn't know what 10646 was, but shouldn't it be 8859-1 or 8859-15, the codes for the Latin-1 alphabet? After finishing up my Inconsolata experiments, when I set the font back to Vera I changed the line to XTerm*font: -bitstream-bitstream vera sans mono-bold-r-normal-*-12-*-*-*-*-*-iso8859-15 and moved on to other things.

Until the next morning, when I booted up to a surprise: my main terminal window no longer fit on the screen. It seems it had reverted to the other (uglier) version of Vera Sans Mono, which is also very slightly taller, so instead of being a couple of lines shorter than the screen height, it was a couple of lines too tall to fit.

I checked .Xdefaults -- yes, it was still Vera. What was going on? I finally remembered the one thing I had changed: the language setting on the font, from 10646-1 to 8858-15. I changed it back: sure enough, now the font was pretty again and the terminal was short enough to fit.

I fired up xfontsel and did some experimenting. It turned out the difference between the two almost-identical Vera sans mono bold roman fonts is a field xfontsel calls "spc". It can be either 'c' or 'm'. The 'c' version is the pretty, compact font; the 'm' is the uglier, taller one. For some reason, specifying 10646-1 makes "spc" default to 'c', while 8859-15 makes it default to 'm'. But specifying 'c' in the font specifier gets the good version regardless of which language is specified.

So this would work: XTerm*font: -bitstream-bitstream vera sans mono-bold-r-normal-*-12-*-*-*-c-*-*-*

But then I read up on 10646-1 and it turns out to mean "the whole unicode character set". That sounds like a good idea, so I kept it in my font specifier after all: XTerm*font: -bitstream-bitstream vera sans mono-bold-r-normal-*-12-*-*-*-c-*-iso10646-1

(For the moment I still didn't know what spc, c or n meant; read on if you're curious.)

The second insight concerned a longstanding mystery of Dave's. He has been complaining for quite a while about the way Ubuntu's modern pango-based apps all refuse to see bitmapped fonts. (It bothered me too, but less so, because the terminal and editor apps I use can see X fonts.)

Dave has an Ubuntu install on one machine that he's been upgrading release after release, which does see his bitmapped fonts. But any fresh Ubuntu installation fails to see the fonts. What was the difference?

We knew about the trick of going into /etc/fonts/conf.d, removing the symbolic link 70-yes-bitmaps.conf and replacing it with a link to /etc/fonts/conf.avail/70-yes-bitmaps.conf ... But doing that doesn't actually change anything, and bitmap fonts still don't show up.

The secret turned out to be that you need to run fc-cache -fv after changing the font/conf.d links. This apparently never happens on its own -- not on a reboot, not on installing or uninstalling font packages. Somehow it had happened once on Dave's good install, and that's why it worked there but nowhere else.

I'm not sure how anyone is supposed to find out about fc-cache -- there's no man fontconfig, and the /etc/fonts/conf.avail/README offers no clue, just misleadingly says "Fontconfig scans this directory". man fc-cache mentions /usr/share/doc/fontconfig/fontconfig-user.html, which doesn't exist; it turns out on Ubuntu it's actually /usr/share/doc/fontconfig-config/fontconfig-user.html. But wait, that's just an html-ized manual page for fonts-conf, so actually you could just run man fonts-conf ... your guess is as good as mine why the fc-cache man page sends you on a hunt for html files instead.

man fonts-conf is good reading -- it even solves the mystery of that spc parameter. It stands for spacing and can be proportional, dual-width, monospace or charcell. Aha! And there's lots more useful-looking information in that manual page as well.

Tags: linux, fonts, i18n
[ 14:58 May 02, 2008    More linux | permalink to this entry ]

How often does updatedb really need to run?

Since updating to Hardy, I've been getting mail from Anacron:

/etc/cron.weekly/slocate:
slocate: fatal error: load_file: Could not open file: /etc/updatedb.conf: No such file or directory

That's the script that updates the database for locate, Linux's fast find system. I figured I must have screwed something up when I moved that slocate cron script from cron.daily to cron.weekly (because I hate having my machine slow to a crawl as soon as I boot it in the morning, and it doesn't bother me if the database doesn't necessarily have files added in the last day or two).

But after talking to some other folks and googling for Ubuntu bugs, I discovered I wasn't the only one getting that mail, and there was already a bug covering it. Comparing my setup with another Hardy user's, I found that the file slocate was failing to find, /etc/updatedb.conf, belongs to a different package, mlocate. If mlocate is installed, then slocate's cron script works; otherwise, it doesn't. Sounds like slocate should have a dependency that pulls in mlocate, no?

But wait, what do these two packages do? Let's try a little aptitude search locate:

p   dlocate                         - fast alternative to dpkg -L and dpkg -S   
p   kio-locate                      - kio-slave for the locate command          
i   locate                          - maintain and query an index of a directory
p   mlocate                         - quickly find files on the filesystem based
i   slocate                         - Secure replacement of findutil's locate   

Okay, forget the first two, but we have locate, mlocate, and slocate. How do they relate?

Worse, if I install mlocate (so slocate will work) and then look in my cron directories, it turns out I now have, count 'em, five different cron scripts that run updatedb. They are:

In cron.daily:

locate: 72 lines! but a lot of that is comments and pruning, and a lot of fiddling to figure out what version of the kernel is running to see whether it can pass any advanced flags when it tries to renice the process. In the end it calls updatedb.findutils (note no full path, though it uses a full path when it checks for it earlier in the script).

slocate: A much simpler but unfortunately buggy 20 lines. It checks for /etc/updatedb.conf, runs it if it exists, fiddles with ionice, checks again for /etc/updatedb.conf, and based on whether it finds it, runs either /usr/bin/slocate -u or /usr/bin/slocate -u -f proc. The latter path is what was failing and sending root mail every time the script was run.

mlocate: an even slimmer 12 line script, which checks for /usr/bin/updatedb.mlocate and, if it exists, fiddles ionice then runs /usr/bin/updatedb.mlocate.

In cron.weekly:

Two virtually identical scripts called find.notslocate and find.notslocate.dpkg-new, which differ only in dpkg-new having more elaborate ionice options. They both run updatedb. And which updatedb would that be? Probably /usr/bin/updatedb, which links to /etc/alternatives/updatedb, which probably links to either updatedb.mlocate or updatedb.slocate, whichever you've installed most recently. But in either case, it's hard to see why you'd need this script running weekly if you're already running both flavors of updatedb from other scripts cron.daily. And having two copies of the script is just plain wrong (and there was already a bug filed on it). (As long as you're poking around in cron.daily and cron.weekly, check and see if you have any more of these extra dpkg-new or dpkg-old scripts -- they might be slowing down your machine for no reason.)

Further research reveals that mlocate is a new(ish) package intended to replace slocate. (There was a long discussion of that on ubuntu-devel, leading to the replacement of slocate with mlocate very late in the Hardy development cycle. There was also lots of discussion of "tracker", apparently a GUI fast find tool that can only search in the user's home directory.)

What is this mlocate? The m stands for "merge": the advantage of mlocate is that it can merge new results into its existing database instead of replacing the whole thing every time. Sounds good, right? However, the down side is that mlocate apparently can't to purge its database of old files that no longer exist, and these files will clutter up your locate results. Running locate -e will keep them from being printed -- but there seems to be no way to set this permanently, via an environment variable or .locaterc file, nor to tell updatedb.mlocate to clean up its database. So you'll need to alias locate to locate -e if you want sensible behavior. Or go back to slocate. Sigh.

Cleaning up

The important thing is to get rid of most of those spurious updatedb cron scripts. You might choose to run updatedb daily, weekly, or only when you choose to run it; but you probably don't want five different scripts running two different versions of updatedb at different times. The packages obviously aren't cleaning up after themselves, so let's do a little manual cleanup.

That find.slocate script looks suspicious. In fact, if you run dpkg -S find.notslocate, you find out that it doesn't belong to any package -- not only should the .dpkg-old version not be there, neither should the other one! So out they go.

As for slocate and mlocate, it's important to know that the two packages can coexist: installing mlocate doesn't remove slocate or vice versa. A clean Hardy install should have only mlocate; upgrades from Gutsy are more likely to have a broken slocate.

Having both packages probably isn't what you want. So pick one, and remove or disable the other. If mlocate is what you want, apt-get purge slocate and just make sure that /etc/cron.*/slocate disappears. If you decide you want slocate, it's a little trickier since the slocate package is broken; but you can fix it by creating an empty /etc/updatedb.conf so updatedb.slocate won't fail.

Tags: linux, boot, ubuntu, install
[ 20:48 Apr 29, 2008    More linux/install | permalink to this entry ]

Autologin changes again for Hardy's upstart

Seems like each new Ubuntu release makes a few gratuitous changes to the syntax of system files. Today's change involves autologin, controlled by the "upstart" system (here's what I wrote about the previous syntax for autologin under upstart).

The /usr/bin/loginscript still hasn't changed, and this still works:

#! /bin/sh
/bin/login -f yourusername

But the syntax has changed a little for the getty line in /etc/event.d/tty1: respawn is now on its own line (I don't know if that matters -- I still can't find any documentation on this file's syntax, though I found a new upstart page that links to some blog entries illustrating how upstart can be used to start system daemons like dbus). And the getty now needs an exec before it. Like this:

respawn
exec /sbin/getty -n -l /usr/bin/loginscript 38400 tty1

Tags: linux, ubuntu, boot
[ 14:27 Apr 22, 2008    More linux | permalink to this entry ]

Upgrading from Ubuntu Gutsy to Hardy

I finally had a moment to upgrade my desktop to Ubuntu's "Hardy Heron". I followed the same procedure as when I went from feisty to gutsy:

1. cp -ax / /hardy
2. cp -ax /dev/.static/dev/* /hardy/dev/
3. Fix up files like /hardy/etc/fstab and /boot/grub/menu.lst
4. Reboot into the newly copied gutsy
5. do-release-upgrade -d

It took an hour or two to pull down all the files, followed by a long interval of occasionally typing Y or N, and then I was ready to start cleaning up some of the packages I'd noticed flying by that I didn't want. Oops! I couldn't remove or install anything with apt-get, because: dpkg --configure -a
But I couldn't dpkg --configure -a because several packages were broken.

The first broken package was plucker, which apparently had failed to install any files. Its postinstall script was failing because it had no files to operate on; and then I couldn't do anything further with it because apt-get wouldn't do anything until I did a dpkg --reconfigure -a

I finally got out of that by dpkg -P plucker; then after several more dpkg --reconfigure -a rounds I was eventually able to apt-get install plucker (which installed just fine the second time).

But apt still wasn't happy, because it wanted to run the trigger for initramfs-tools, which wouldn't run because it wanted kernel modules for some specific kernel version in /lib/modules. I didn't have any kernel modules because I'm not running Ubuntu's kernel (I'm stuck on 2.6.23 because bug 10118 makes all 2.6.24 variants unable to sync with USB Palm devices). But I couldn't remove initramfs-tools because udev (along with a bunch of other less important packages) depends on it. I finally found my way out of that by removing /var/lib/dpkg/triggers/initramfs-tools. I reported it as bug 220094.

Update: I forgot to mention one important thing I hit both on this machine and earlier, on the laptop: /usr/bin/play (provided by the "sox" package) no longer works because it now depends on a zillion separate libraries. apt-get install libsox-fmt-all to get all of them.

Tags: linux, ubuntu, debian, install
[ 20:02 Apr 20, 2008    More linux/install | permalink to this entry ]

A minimal xorg.conf

Dave has been experimenting with xorg configuration lately -- trying to figure out why the latest Xorg no longer supports 1600x1200 on his monitor. (I've looked for bug reports and found gazillions of them, all blaming it on the video card but involving three different makes of video card, so color me skeptical.)

Anyway, part of this has involved taking out parts of his /etc/X11/xorg.conf file to see which parts might be causing the problem, and he's found something interesting.

What do you suppose is the minimal useful xorg.conf file? You might suppose, oh, screen and monitor sections, an input section for the keyboard and another one for a generic mouse, and that might be all you need ... right?

Okay, try it. Let's start with a really minimal file -- nothing -- and gradually add sections. To try it, make a backup of your current xorg.conf, then zero out the file:

cd /etc/X11
mv xorg.conf xorg.conf.sav
cp /dev/null xorg.conf

Now exit X if you hadn't already, and start it up again (or let gdm do it for you). Be prepared to do repairs from the console in case X doesn't start up: e.g. sudo cp /etc/X11/xorg.conf.bak /etc/X11/xorg.conf

What happened?

In my case, on the laptop running Hardy beta, X starts right up and looks just the same as it did before.

xorg.conf -- who needs it?

A specious question, of course, which has a perfectly good answer: anyone who needs a resolution other than whatever xorg picks as the default; anyone with additional hardware, like a wacom tablet; anyone who wants customizations like XkbOptions = ctrl:nocaps. There are lots of reasons to have an xorg.conf. But it's fun to know that at least on some machines, it's possible to run without one.

Update: turns out this is part of Ubuntu's new BulletProof X feature. It doesn't work on other distros or older versions. Thanks to James D for the tip.

Tags: linux, X11
[ 10:25 Apr 10, 2008    More linux | permalink to this entry ]

k3b breaks without hal

I burned a CD for the Ubuntu hardy beta alternate installer. I used k3b since that's been a good, fairly reliable burning app with a well designed UI -- I've been using it for years despite not running a KDE desktop. I selected "Burn CD Image", reduced the speed (burning apps are always wildly optimistic about speed, with the result that they create a lot of coasters) and checked the box for "verify contents after burning".

The burn went fine, and k3b ejected the CD, then sucked it back in again for the verification stage. At that point k3b started spewing lots of errors to the terminal, things like "/dev/hdd: READ 10 failed!" and "Failed to init HAL context!" repeated many times.

How annoying! k3b has added a new dependency on hal, and although it can burn a CD just fine, without hal it then forgets where the CD drive was so it can read the CD back in to verify it.

Fortunately dd /dev/cdrw | md5sum worked fine to verify that the burn was correct. I guess it's time to investigate other CD burning programs.

Tags: linux, hal
[ 10:04 Apr 10, 2008    More linux | permalink to this entry ]

Ubuntu "Hardy Heron"

On a lunchtime bird walk on Monday I saw one blue heron and at least five green herons (very unusuual to see so many of those). Maybe that helped prepare me for installing the latest Ubuntu beta, "Hardy Heron", Monday afternoon.

I was trying the beta primarily in the hope that it would fix a serious video out regression that appeared in Gutsy (the current Ubuntu) in January. My beloved old Vaio SR17 laptop can't switch video signals on the fly like some laptops can; I've always needed to boot it with an external monitor or projector connected. But as long as it saw a monitor at boot time, it would remember that state through many suspend cycles, so I could come out of suspend, plug in to a projector and be ready to go. But beginning some time in late January, somehow Gutsy started doing something that turned off the video signal when suspending. To talk to a projector, I could reboot with the projector connected (I hate making an audience watch that! and besides, it takes away the magic). I also discovered that switching to one of the alternate consoles, then back (ctl-alt-F2 ctl-alt-F7) got a signal going out on the video port -- but I found out the hard way, in front of an audience, that it was only a 640x480 signal, not the 1024x768 signal I expected. Not pretty! I could either go back to Feisty ... or try upgrading to Hardy.

I've already written about the handy debootstrap lightweight install process I used. (I did try the official Hardy "alternate installer" disk first, but after finishing package installation it got into a spin lock trying to configure kernel modules, so I had to pull the plug and try another approach.)

This left me with a system that was very minimal indeed, so I spent the next few hours installing packages, starting with tcsh, vim (Ubuntu's minimal install has something called vim, but it's not actually vim so you tend to get lots of errors about parsing your .vimrc until you install the real vim), acpi and acpi-support (for suspending), and the window system: xorg and friends. To get xorg, I started with:

apt-get install xserver-xorg-video-savage xbase-clients openbox xloadimage xterm

Then there was the usual exercise of aptitude search font and installing everything on that list that seemed relevant to European languages (I don't really need to scroll through dozens of Tamil, Thai, Devanagari and Bangla fonts every time I'm looking for a fancy cursive in GIMP).

But I hit a problem with that pretty early on: turns out most of the fonts I installed weren't actually showing up in xlsfonts, xfontsel, gtkfontsel, or, most important, the little xlib program I'm using for a talk I need to give in a couple weeks. I filed it as bug 212669, but kept working on it, and when a clever person on #ubuntu+1 ("redwhitewaldo") suggested I take a look at the x-ttcidfont-conf README, that gave me enough clue to get me the rest of the way. Turns out there's a Debian bug with the solution, and the workaround is easy until the Ubuntu folks pick up the update.

I hit a few other problems, like the PCMCIA/udev problem I've described elsewhere ... but mostly, my debootstrapped Hardy Heron is working quite well.

And in case you're wondering whether Hardy fixed the video signal problem, I'm happy to say it does. Video out is working just fine.

Tags: linux, ubuntu, install, fonts, vaio
[ 18:31 Apr 07, 2008    More linux/install | permalink to this entry ]

Debootstrap

Some time ago, I wished for a simple Linux "Tarball installer", something that could install a minimal install of a Linux distribution onto an existing partition or directory, skipping all the flaky and error-prone hardware-guessing that installers do.

It turns out Debian (and therefore also Ubuntu) has had this for years, and it's totally cool. It's called debootstrap.

Some folks on the #ubuntu+1 channel told me about it, and I found a nice clear howto article on how to use it for Debian. It works just the same for Ubuntu.

First, get the .deb package for the debootstrap you want to use. Here's debootstrap for Ubuntu Hardy Heron. Install it with dpkg -i. Then run it, giving it the name of the system you want to install and the directory (or mounted partition) where you want to install it. Like this: debootstrap hardy /mnt/hda3

That's all! It fetches the files it needs from the online repositories. It takes no time at all -- this really is a minimal system.

Then you need to do some fiddling to turn it into a bootable system. That includes (all paths relative to the newly installed filesystem unless otherwise stated):

• Set up etc/fstab to list the fileystems on the disk, and to mount / from the filesystem you just installed
• Define the hostname in etc/hostname
• Set up a grub boot stanza in /boot/grub/menu.lst (that's /boot on the current system, which should be the same as /boot in the new fstab you just created). Use whatever kernel you were using for your old system, for now.

Now you're read to reboot into the new system. Of course, since this is a very minimal system, you have a lot more work to do. Hardly anything is installed, and nothing has been configured for you. Some things may be challenging (for example, as I write this, X is installed but most of the fonts aren't showing up properly, which may be a bug in Hardy).

Anyway, you can get a good start by mounting your old system's root directory and copying some starter files from there, starting with these:

• Set up your important configuration files: /etc/network/interfaces, /etc/hosts, /etc/resolv.conf, /etc/passwd etc.
• edit /etc/apt/sources.list to include restricted universe multiverse
• Install a kernel package if you're using distro kernels
• Install vim if you're a vim user -- remember, ubuntu comes with something called vim that isn't really vim.
• Create users and homedirs and such
• Install all the other stuff you want -- X, gimp/gtk, development tools, editors, shells -- all that stuff that makes the system feel like home. You're on your own there, so have fun!

Tags: linux, debian, installer
[ 12:54 Apr 06, 2008    More linux/install | permalink to this entry ]

Handling a network card automatically: Calling ifup from udev

I'm experimenting with Ubuntu's "Hardy Heron" beta on the laptop, and one problem I've hit is that it never configures my network card properly.

The card is a cardbus 3Com card that uses the 3c59x driver. When I plug it in, or when I boot or resume after a suspend, the card ends up in a state where it shows up in ifconfig eth0, but it isn't marked UP. ifup eth0 says it's already up; ifdown eth0 complains error: SIOCDELRT: No such process but afterward, I can run ifup eth0 and this time it works. I've made an alias, net, that does sudo ifdown eth0; sudo ifup eth0. That's silly -- I wanted to fix it so it happened automatically.

Unfortunately, there's nothing written anywhere on debugging udev. I fiddled a little with udevmonitor and udevtest /class/net/eth0 and it looked like udev was in fact running the ifup rule in /etc/udev/rules.d/85-ifupdown.rules, which calls: /sbin/start-stop-daemon --start --background --pid file /var/run/network/bogus --startas /sbin/ifup -- --allow auto $env{INTERFACE} So I tried running that by hand (with $env{INTERFACE} being eth0) and, indeed, it didn't bring the interface up.

But that suggested a fix: how about adding --force to that ifup line? I don't know why the card is already in a state where ifup doesn't want to handle it, but it is, and maybe --force would fix it. Sure enough: that worked fine, and it even works when resuming after a suspend.

I filed bug 211955 including a description of the fix. Maybe there's some reason for not wanting to use --force in 85-ifupdown (why wouldn't you always want to configure a network card when it's added and is specified as auto and allow-hotplug in /etc/network/interfaces?) but if so, maybe someone will suggest a better fix.

Tags: linux, ubuntu, udev, net
[ 13:41 Apr 04, 2008    More linux | permalink to this entry ]

Synaptics and USB mouse simultaneously

A month or so back, I spent some time fiddling with the options for the Synaptics touchpad driver. The Alps (not Synaptics) trackpad on my laptop has always worked okay with just the standard PS/2 mouse driver, but in recent kernels it's become overly sensitive to taps, registering spurious clicks when I'm in the middle of typing a word (so suddenly I'm typing in a completely different window without knowing it).

I eventually got it working. I tried various options, but here's what I settled on:

Section "InputDevice"
        Identifier      "Trackpad"
        Driver          "synaptics"
        Option          "SHMConfig"             "true"
        Option          "SendCoreEvents"        "true"
        Option          "Device"                "/dev/psaux"
        Option          "Protocol"              "auto-dev"
        Option          "MinSpeed"              "0.5"
        Option          "MaxSpeed"              "0.75"
# AccelFactor defaults to .0015 -- synclient -l to check
        Option          "TouchpadOff"           "2"
        Option          "Emulate3Buttons"       "true"
EndSection

Section "InputDevice"
        Identifier      "Configured Mouse"
        Driver          "mouse"
        Option          "CorePointer"
        Option          "Device"                "/dev/input/mice"
        Option          "Protocol"              "ExplorerPS/2"
        Option          "ZAxisMapping"          "4 5"
        Option          "Emulate3Buttons"       "true"
EndSection

Life was groovy (I thought). Fast forward to LCA, a few days before my talk, when I decide to verify that I can run my USB mouse and the slide-advancing presentation gizmo through a hub off the single USB port. Quel surprise: the USB mouse doesn't work at all!

I didn't really need a mouse for that presentation (it was on GIMP scripting, not GIMP image editing) so I put it on the back burner, and came back to it when I got home. As I suspected, the USB mouse was working fine if I commented out the Synaptics entry from xorg.conf; it just couldn't run both at the same time.

A little googling led me to the answer, in a thread called Can't use Synaptics TouchPad and USB Mouse -- it wasn't the first google hit for synaptics "xorg.conf" usb mouse, so perhaps this entry will help its google-fu. The important part I was missing was in the "ServerLayout" section:

        InputDevice     "Trackpad"              "AlwaysCore"
        InputDevice     "Configured Mouse"      "CorePointer"

Adding "AlwaysCore" and "CorePointer" parts was what did the trick. Thanks to "finferflu" who posted the right answer in the thread.

Tags: linux, X11
[ 21:54 Feb 05, 2008    More linux | permalink to this entry ]

Gutsy's persistent net rules don't persist

I use wireless so seldom that it seems like each time I need it, it's a brand new adventure finding out what has changed since the last time to make it break in a new and exciting way.

This week's wi-fi adventure involved Ubuntu's current "Gutsy Gibbon" release and my prism54 wireless card. I booted the machine, switched to the right (a href="http://shallowsky.com/linux/networkSchemes.html">network scheme, inserted the card, and ... no lights. ifconfig -a showed the card on eth1 rather than eth0.

After some fiddling, I ejected the card and re-inserted it; now ifconfig -a showed it on eth2. Each time I inserted it, the number incremented by one.

Ah, that's something I remembered from Debian Etch -- a problem with the udev "persistent net rules" file in /etc/udev.

Sure enough, /etc/udev/70-persistent-net.rules had two entries for the card, one on eth1 and the other on eth2. Ejecting and re-inserting added another one for eth3. Since my network scheme is set up to apply to eth0, this obviously wouldn't work.

A comment in that file says it's generated from 75-persistent-net-generator.rules. But unfortunately, the rules uesd by that file are undocumented and opaque -- I've never been able to figure out how to make a change in its behavior. I fiddled around for a bit, then gave up and chose the brute force solution:

• Remove /etc/udev/75-persistent-net-generator.rulesa
• Edit /etc/udev/70-persistent-net.rules to give the device the right name (eth1, eth0 or whatever).

And that worked fine. Without 75-persistent-net-generator.rules getting in the way, the name seen in 70-persistent-net.rules works fine and I'm able to use the network.

The weird thing about this is that I've been using Gutsy with my wired network card (a 3com) for at least a month now without this problem showing up. For some reason, the persistent net generator doesn't work for the Prism54 card though it works fine for the 3com. A scan of the Ubuntu bug repository reveals lots of other people hitting similar problems on an assortment of wireless cards; bug 153727 is a fairly typical report, but the older bug 31502 (marked as fixed) points to a likely reason this is apparently so common on wireless cards -- apparently some of them report the wrong MAC address before the firmware is loaded.

Tags: linux, ubuntu, udev, net
[ 18:02 Dec 23, 2007    More linux | permalink to this entry ]

Smart Wrapping with Greedy and Non-Greedy Regular Expressions

I had a chance to spend a day at the AGU conference last week. The American Geophysical Union is a fabulous conference -- something like 14,000 different talks over the course of the week, on anything related to earth or planetary sciences -- geology, solar system astronomy, atmospheric science, geophysics, geochemistry, you name it.

I have no idea how regular attendees manage the information overload of deciding which talks to attend. I wasn't sure how I would, either, but I started by going through the schedule for the day I'd be there, picking out a (way too long) list of potentially interesting talks, and saving them as lines in a file.

Now I had a file full of lines like:

1020      U22A    MS 303  Terrestrial Impact Cratering: New Insights Into the Cratering Process From Geophysics and Geochemistry II

Fine, except that I couldn't print out something like that -- printers stop at 80 columns. I could pass it through a program like "fold" to wrap the long lines, but then it would be hard to scan through quickly to find the talk titles and room numbers. What I really wanted was to wrap it so that the above line turned into something like:

1020      U22A    MS 303  Terrestrial Impact Cratering: New Insights
                          Into the Cratering Process From Geophysics
                          and Geochemistry II

But how to do that? I stared at it for a while, trying to figure out whether there was a clever vim substitute that could handle it. I asked on a couple of IRC channels, just in case there was some amazing Linux smart-wrap utility I'd never heard of. I was on the verge of concluding that the answer was no, and that I'd have to write a python script to do the wrapping I wanted, when Mikael emitted a burst of line noise:
%s/\(.\{72\}\)\(.*\)/\1^M^I^I^I\2/

Only it wasn't line noise. Seems Mikael just happened to have been reading about some of the finer points of vim regular expressions earlier that day, and he knew exactly the trick I needed -- that .\{72\}, which matches lines that are at least 72 characters long. And amazingly, that expression did something very close to what I wanted.

Or at least the first step of it. It inserts the first line break, turning my line into

1020      U22A    MS 303  Terrestrial Impact Cratering: New Insights
                          Into the Cratering Process From Geophysics and Geochemistry II

but I still needed to wrap the second and subsequent lines.

But that was an easier problem -- just do essentially the same thing again, but limit it to only lines starting with a tab. After some tweaking, I arrived at exactly what I wanted:

%s/^\(.\{,65\}\) \(.*\)/\1^M^I^I^I\2/

%g/^^I^I^I.\{58\}/s/^\(.\{,55\}\) \(.*\)/\1^M^I^I^I\2/

I had to run the second line two or three times to wrap the very long lines.

Devdas helpfully translated the second one into English: "You have 3 tabs, followed by 58 characters, out of which you match the first 55 and put that bit in $1, and the capture the remaining in $2, and rewrite to $1 newline tab tab tab $2."

Here's a more detailed breakdown:

Line one:

%	Do this over the whole file
s/	Begin global substitute
^	Start at the beginning of the line
\(	Remember the result of the next match
.\{,65\}_	Look for up to 65 characters with a space at the end
\) \(	End of remembered pattern #1, skip a space, and start remembered pattern #2
.*\)	Pattern #2 includes everything to the end of the line
/	End of matched pattern; begin replacement pattern
\1^M	Insert saved pattern #1 (the first 65 lines ending with a space) followed by a newline
^I^I^I\2	On the second line, insert three tabs then saved pattern #2
/	End replacement pattern

Line two:

%g/	Over the whole file, only operate on lines with this pattern
^^I^I^I	Lines starting with three tabs
.\{58\}/	After the tabs, only match lines that still have at least 58 characters (this guards against wrapping already wrapped lines when it's run repeatedly)
s/	Begin global substitute
^	Start at the beginning of the line
\(	Remember the result of the next match
.\{,55\}	Up to 55 characters
\) \(	End of remembered pattern #1, skip a space, and start remembered pattern #2
.*\)	Pattern #2 includes everything to the end of the line
/	End of matched pattern; begin replacement pattern
\1^M	The first pattern (up to 55 chars) is one line
^I^I^I\2	Three tabs then the second pattern
/	End replacement pattern

Greedy and non-greedy brace matches

The real key is those curly-brace expressions, \{,65\} and \{58\} -- that's how you control how many characters vim will match and whether or not the match is "greedy". Here's how they work (thanks to Mikael for explaining).

The basic expression is {M,N} -- it means between M and N matches of whatever precedes it. (Vim requires that the first brace be escaped -- \{}. Escaping the second brace is optional.) So .{M,N} can match anything between M and N characters but "prefer" N, i.e. try to match as many as possible up to N. To make it "non-greedy" (match as few as possible, "preferring" M), use .{-M,N}

You can leave out M, N, or both; M defaults to 0 while N defaults to infinity. So {} is short for {0,∞} and is equivalent to *, while {-} means {-0,∞}, like a non-greedy version of *.

Given the string: one, two, three, four, five

,.\{},	matches , two, three, four,
,.\{-},	matches , two,
,.\{5,},	matches , two, three, four,
,.\{-5,},	matches , two, three,
,.\{,2},	matches nothing
,.\{,7},	matches , two,
,.\{5,7},	matches , three,

Of course, this syntax is purely for vim; regular expressions are unfortunately different in sed, perl and every other program. Here's a fun table of regexp terms in various programs.

Tags: linux, editors, regexp
[ 11:44 Dec 20, 2007    More linux/editors | permalink to this entry ]

XFCE dukes it out with Gnome; users lose

Looking for a volume control that might me installed on mom's XFCE4-based Xubuntu desktop, I tried running xfce4-mixer.

The mixer came up fine -- but after I exited, I discovered that my xchat had gone all wonky. None of my normal key bindings worked, my cursor was blinking, and the fonts used for tabs was about half its normal size. Over in my Firefox window, key bindings were also affected.

I've seen this sort of thing happen before with Gnome apps, and had found a way to solve it using gconf-editor. That app was not installed, so I installed it and discovered that it didn't help.

I tried killing the running gconfd-2, removing .gconf/ and .gconfd/ from my home directory, then removing the four gnome directories (.gnome/, .gnome2/, .gnome2_private/, and .gnome_private/). Nothing helped xchat (though Firefox did return to normal).

After much flailing and annoying people by restarting xchat repeatedly, it turned out the problem was that xfce-mixer had started a daemon called xfce-mcs-manager, which is like gconf, only different. Like gconf, it mucks with settings of all running gtk programs without asking first. It runs simultaneously with gconf, but overrides gconf, which in turn overrides the values set in ~/.gtkrc-2.0.

Killing xfce-mcs-manager caused my running xchat to revert to its normal settings.

... Well, *almost* revert. A few key bindings didn't get reset, as I discovered when I hit a ctrl-W to erase the last word and found myself disconnected from the channel. Another xchat restart, with xfce-mcs-manager not running, fixed that.

Aside from the ever-present issue of "Where do I look when some unfriendly program decides to change the settings in running applications?" (which begs the question, "What genius thought it would be a good idea to give any random app like a volume control the power to change settings in every other gtk application currently running on the system? And do they have their medications adjusted better now?") there's another reason this is interesting.

See, if an arbitrary app like xfce-mcs-manager can send a message to xchat to change key bindings like ctrl-W ... then maybe I could write a program that could send a similar message telling xchat to cancel those compiled-in bindings like ctrl-F and ctrl-L, ones that it doesn't allow the user to change. If I could get something like that working, I could use a standard xchat -- I'd no longer need to patch the source and build my own.

Tags: linux, gnome
[ 20:12 Dec 14, 2007    More linux | permalink to this entry ]

Bug fixes? Why would we bother to ship bug fixes?

(A culture of regressions, part 2)

I've been running on Ubuntu's latest, "Gutsy gibbon", for maybe a month now. Like any release, it has its problems that I've needed to work around. Like many distros, these problems won't be fixed before the next release. But unlike other distros, it's not just lack of developer time; it turns out Ubuntu's developers point to an official policy as a reason not to fix bugs.

Take the case of the aumix bug. Aumix just plain doesn't work in gutsy. It prints, "aumix: SOUND_MIXER_READ_DEVMASK" and exits.

This turns out to be some error in the way it was compiled. If you apt-get the official ubuntu sources, build the package and install it yourself, it works fine. So somehow they got a glitch during the process of building it, and produced a bad binary.

(Minor digression -- does that make this a GPL violation? Shipping sources that don't match the distributed binary? No telling what sources were used to produce the binary in Gutsy. Not that anyone would actually want the sources for the broken aumix, of course.)

It's an easy fix, right? Just rebuild the binary from the source in the repository, and push it to the servers.

Apparently not. A few days ago, Henrik Nilsen Omma wrote in the bug:

This bug was nominated for Gutsy but does currently not qualify for a 7.10 stable release update (SRU) and the nomination is therefore declined. According the the SRU policy, the fix should already be deployed and tested in the current development version before an update to the stable releases will be considered. [ ... ] See: https://wiki.ubuntu.com/StableReleaseUpdates.

Of course, I clicked on the link to receive enlightenment. Ubuntu's Stable Release page explains

Users of the official release, in contrast, expect a high degree of stability. They use their Ubuntu system for their day-to-day work, and problems they experience with it can be extremely disruptive. Many of them are less experienced with Ubuntu and with Linux, and expect a reliable system which does not require their intervention.

by way of explaining the official Ubuntu policy on updates:

Stable release updates will, in general, only be issued in order to fix high-impact bugs. Examples of such bugs include:

• Bugs which may, under realistic circumstances, directly cause a security vulnerability
• Bugs which represent severe regressions from the previous release of Ubuntu
• Bugs which may, under realistic circumstances, directly cause a loss of user data

Clearly aumix isn't a security vulnerability or a loss of user data. But I could make a good argument that a package that doesn't work ... ever ... for anyone ... constitutes a severe regression from the previous version of that package.

Ubuntu apparently thinks that users get used to packages not working, and grow to like it. I guess that if you actually fixed packages that you broke, that would be disruptive to users of the stable release.

I'm trying to picture these Ubuntu target users, who embrace regressions and get upset when something that doesn't work at all gets fixed so that it works as it did in past releases. I can't say I've ever actually met a user like that myself. But evidently the Ubuntu Updates Team knows better.

Update: I just have to pass along Dave's comment: "When an organization gets to the point where it spends more energy on institutional processes for justifying not fixing something than on just fixing it -- it's over."

Update: Carla Schroder has also written about this.

Tags: linux, ubuntu, audio
[ 10:21 Dec 07, 2007    More linux | permalink to this entry ]

More Tips on International Input

With what I learned last week, I've been able to type accented characters into GTK apps such as xchat, and a few other apps such as emacs. That's nice -- but I was still having trouble reading accented characters in mutt, or writing them in vim to send through mutt (darn terminal apps).

The biggest problem was the terminal. I was using urxvt, but it turns out that urxvt won't let me type any nonascii characters. It just ignores my multi-key sequences, or prints a space instead of the character I wanted. I have no idea why, but switching to plain ol' xterm solved that problem. Of course, I had to make sure that I was using a font that supported the characters I wanted (ISO 8859-1 or 8859-15 or something similar), which leaves out my favorite terminal font (Schumacher Clean bold), but Bitstream Vera Sans Mono bold is almost as readable.

Of course, it's important to have your locale variables set appropriately. There are several locale variables:

LC_CTYPE

Which encodings to use for typing and displaying characters.

LC_MESSAGES

Which translations to use, in programs that offer them.

LC_COLLATE

How to sort alphabetically (this one also affects whether ls groups capitalized filenames first).

LC_ALL

Overrides any of the others.

LANG

The default, in case none of the other variables is set.

There are a few others which control very specific features like time, numbers, money, addresses and paper size: type locale to see all of them.

Once I switched to xterm, I was able to set either LANG or LC_CTYPE to either en_US.UTF-8 or en_US.ISO-8859-1. I set LC_COLLATE and LANG or LC_MESSAGES to C, so that I get the default (usually US) translations for programs and so that ls groups all the capitalized files first.

Along the way, I learned about yet another way to type accented characters.

setxkbmap -model pc104 -layout us -variant intl

switches to an international layout, at which point typing certain punctuation (like ' or ~) is assumed to be a prefix key. So instead of typing [Multi] ~ n, I can just type ~ n. The catch: it makes it harder to type quotes or tildes by themselves (you have to type a space after the quote or tilde).

Even faster, the international layout also offers shortcuts to many common characters with the "AltGr" key, which I'd heard about for years but never knew how to enable. AltGr is the right alt key, and typing, say, AltGr followed by n gives an ñ. You can see a full map at Wikipedia (AltGr characters are blue, quote prefixes are red).

To get back to a US non-international layout:

setxkbmap -model pc104 -layout us

Of course, these aren't the only keyboard layouts to choose from -- there are lots, plus you can define your own. And I was going to write a little bit about that, except it turns out they've changed it all around again since I last did that two years ago (don't you love the digital world?). So that will have to wait for another time.

But the place to start exploring is /usr/share/X11/xkb. The file symbols/us contains the definitions for those US keyboards, and I believe it's included via the files in the rules directory, probably rules/base, base.xml and base.lst. From there you're on your own. But the standard layouts probably follow the ones in the Wikipedia article on keyboard layouts

Tags: linux, i18n, keyboard
[ 15:48 Dec 01, 2007    More linux | permalink to this entry ]

Backing up a file system

I upgraded my system to the latest Ubuntu, "Gutsy Gibbon", recently. Of course, it's always best to make a backup before doing a major upgrade. In this case, the goal was to back up my root partition to another partition on the same disk and get it working as a bootable Ubuntu, which I could then upgrade, saving the old partition as a working backup. I'll describe here a couple of silly snags I hit, to save you from making the same mistakes.

Linux offers lots of ways to copy filesystems. I've used tar in the past, with a command like (starting in /gutsy): tar --one-file-system -cf - / | tar xvf - > /tmp/backup.out but cp seemed like an easier way, so I want to try it.

I mounted my freshly made backup partition as /gutsy and started a cp -ax /* /gutsy (-a does the right thing for permissions, owner and group, and file type; -x tells it to stay on the original filesystem). Count to ten, then check what's getting copied. Whoops! It clearly wasn't staying on the original filesystem.

It turned out my mistake was that /*. Pretty obvious in hindsight what cp was doing: for each entry in / it did a cp -ax, staying on the filesystem for that entry, not on the filesystem for /. So /home, /boot, /proc, etc. all got copied. The solution was to remove the *: cp -ax / /gutsy.

But it wasn't quite that simple. It looked like it was working -- a long wait, then cp finished and I had what looked like a nice filesystem backup. I adjusted /gutsy/etc/fstab so that it would point to the right root, set up a grub entry, and rebooted. Disaster! The boot hung right after Attached scsi generic sg0 type 0 with no indication of what was wrong.

Rebooting into the old partition told me that what's supposed to happen next is: * Setting preliminary keymap... But the crucial error message was actually several lines earlier: Warning: unable to open an initial console. It hadn't been able to open /dev/console.

Now, in the newly copied filesystem, there was no /dev/console: in fact, /dev was empty. Nothing had been copied because /dev is a virtual file system, created by udev.

But it turns out that the boot process needs some static devices in /dev, before udev has created anything. Of course, once udev's virtual filesystem has been mounted on /dev, you can no longer read whatever was in /dev on the root partition in order to copy it somewhere else. But udev nicely gives you access to it, in /dev/.static/dev. So what I needed to do to get my new partition booting was: cp -ax /dev/.static/dev/ /gutsy/dev/ With that done, I was able to boot into my new filesystem and upgrade to Gutsy.

Tags: linux, ubuntu, backups
[ 22:48 Nov 30, 2007    More linux | permalink to this entry ]

Typing accented characters (for ignorant 'murricans)

Happy Thanksgiving, everyone! Today's holiday tip involves how to type international characters.

For the online Spanish class I've been taking, so far I've been able to manage without having to type characters like ñ or á. Usually, if I need one I can find it in one of the class examples, copy it, and paste it wherever I need it. But obviously that would be tedious if I needed to type much.

I hacked up a quickie workaround: a python script that shows a set of buttons, one for each accented character I'm likely to need. Clicking a button copies that character to the clipboard, so I can now paste via mouse middleclick or ctrl-V. (I'm sure that sounds pathetic to those of you who type accented characters every day, but it's not something most US English speakers need to do. And besides, now I know how to access the X clipboard from Python-GTK -- hooray for learning new things from procrastination projects!)

Anyway, Mikael Magnusson took pity on me and explained in simple language how to use the X "Multi key" to type these characters the right way (well, a right way, anyway). Since all the online instructions I've seen have been rather complicated, here are the simple instructions for any of my fellow US monolingists who'd like to expand their horizons:

First, choose a key for the "Multi key" that you're not using for anything else. A lot of people use one of the Alt or Windows keys, but I use both of those already. What I don't use is the Menu key (that little key down by the right Ctrl key, at least on my keyboard) since not many Linux apps support it anyway.

Find the keycode for that key, by firing up xev and typing the key. For my Menu key, the keycode is 117.

Now type:

xmodmap -e "keycode 117 = Multi_key"

Now you're ready to type a sequence like: [Menu] ~ n to type an n-tilde, [Menu] ' a for an accented a, or [menu] ? ? for the upside-down question mark, in any app that supports those characters.

Of course, you don't want to type that xmodmap command every time you log in, so to make it permanent, put this in your .Xmodmap (you're on your own for figuring out whether your X environment reads .Xmodmap automatically or whether you need to tell it to run xmodmap .Xmodmap when X starts up):

keycode 117 = Multi_key

I have one final useful international input tidbit to offer: how to type Unicode characters by number. Hold ctrl+shift+U, then release U but keep holding the other two while you type a numeric sequence. (This may only work in gtk apps.) For instance, try this: hold down ctrl and shift, then type: u 2 6 6 c. Cool, huh? You can use the "gucharmap" program to find other neat sequences (hint: View->By Unicode Block otherwise you'll never find anything).

Now it's time to check the turkey. Have a good day, everyone!

Tags: linux, i18n, keyboard
[ 16:03 Nov 22, 2007    More linux | permalink to this entry ]

Proposal: A Simpler Linux Installer

I spent a couple of fruitless hours today trying to install PCLinuxOS, a well-reviewed new Linux distro, on my Vaio. I got lots of help from the nice folks on the IRC channel, and tried lots of different approaches, but no dice -- their Live CD won't boot because it doesn't grok PCMCIA CDROM drives, and they have no other installation method besides using the live CD.

Which brings me to today's question: Why do Linux distros have installers at all?

That probably sounds like a silly question. Of course you need an installer to get the system onto your disk ... don't you?

Well, yes and no. You could make it a lot simpler than anyone currently does.

What if you distributed a Linux distro as a filesystem image? Make it tar, zip, CD iso or whatever format you like -- but provide the user with a tree of files that, when copied onto a hard drive, can boot as a running Linux.

Set up this minimal installation filesystem so that when you boot into it, you get a commandline (X hasn't been configured yet) and a set of scripts that make it easy to go the rest of the way. From your running minimal system, you can configure X, set up networking, install more programs from the distro repositories (or even from a CD image), and do all the rest of the machine-specific configuration that an installer does.

The key is that this is all happening from a running system, not from some cobbled-together installer kernel or live CD.

If you have a problem with any step, you still have a basic running system, and tools to fix the problem. You avoid the usual loop:

• Run installer
• Spend 20 minutes answering questions
• Spend 45 minutes waiting for installer
• Discover it failed
• Start over with slightly different parameters

If your X configuration fails, try running X configuration again -- no need to do another install from the beginning. If it doesn't see your network card -- ditto. Since this debugging all happens from a normal running Linux, you can use the normal Linux tools you're used to, not some busybox-based installer.

This model would be much more hardware agnostic than current installers:

• You can install on systems with weirdo graphics cards;
• You can install on systems that need special drivers for the network card or other hardware;
• You can install on systems with no CDROM or an external CDROM;
• You can install even if you don't have access to a CD burner.

Another advantage is that it makes it very easy to make a customized version of your distro: just take the basic system image, change the part that needs changing and tar it up again.

Some distros have gone a little way with this, with an installer that gives you a starter system, then scripts to download the rest -- but I've never seen one that made the minimal system available as a filesystem image, with easy instructions on going to the next step.

What about the people who aren't already running Linux or aren't comfortable writing a filesystem image to a partition?

No problem. They get a CD image with a very simple installer -- it handles the partitioning, copies the minimal install to the partition, and updates grub. It's as prone to hardware compatibility issues as any installer (though far less so than a live CD is) but it's still better than the current model, because it won't be trying to configure hardware until the user reboots into a working minimal system.

Of course, Live CDs are still cool -- on machines where they actually work -- for showing Linux to people not ready to commit to an install. But don't tie your installer to that. Give people a simpler way to install, one that's fast and straightforward and hardware agnostic and easy to understand or customize.

The tarball installer. An idea whose time has come. Now if I could just persuade the distros to offer it.

Update: a couple of people told me about Dynebolic, a distro that apparently does just that -- you install by copying a directory on the CD onto your partition, or rsyncing from their site. Nice!

Tags: linux
[ 22:59 Nov 14, 2007    More linux | permalink to this entry ]

Tried bash, went back to tcsh

I've been a tcsh user for many years. Back in the day, there were lots of reasons for preferring csh to sh, mostly having to do with command history. Most of those reasons are long gone -- modern bash and tcsh are vastly improved from those early shells, and borrow from each other, so the differences are fairly minor.

Back in July, I solved the last blocker that had been keeping me off bash, so I put some effort into migrating all my .cshrc settings into a .bashrc so I could give bash a fair shot. It almost won; but after four months, I've switched back to tcsh, due mostly to a single niggling bash bug that I just can't seem to solve. After all these years, the crucial difference is still history. Specifically, history amnesia: bash has an annoying habit of forgetting history commands just when I most want them back.

Say I type some longish command. After it runs, I hit return a couple of times, wait a while, do a couple of other things, then decide I want to call that command back from history so I can run something similar, maybe with the filename changed or a different flag. I ctrl-P or up-arrow ... and the command isn't there!

If I type history at this point, I'll see most of my command history ... with an empty line in place of the line I was hoping to repeat. The command is gone. My only option is to remember what I typed, and type it all again.

Nobody seems to know why this happens, and it's sporadic, doesn't happen every time. Friends have been able to reproduce it, so it's not just me or my weird settings. It drives me batty. It wouldn't be so bad except it always seems to happen on the tricky commands that I really didn't want to retype.

It's too bad, because otherwise I had bash nicely whipped into shape, and it does have some advantages over tcsh. Some of the tradeoffs:

tcsh wins

• Totally reliable history: commands never disappear.
• History tab completion: typing !a<TAB> expands to the last command that started with a. In bash, I have to type !a:p to see the command, then !! to execute it.
• When I tab-complete a file and there are multiple matches, tcsh shows them, or at least beeps (depending on configuration). In bash I have to hit a second tab just in case there might be matches.
• When I tab-complete a directory, tcsh adds the / automatically. (Arguable. I find I want the / roughly 3/4 of the time.)
• tcsh doesn't drop remote connections if I suspend (with ~^Z). bash drops me within a minute or two, regardless of settings like $TMOUT. Bash users tell me I could solve this by using screen, but that seems like a heavyweight workaround when tcsh "just works".
• tcsh sees $REMOTEHOST and $DISPLAY automatically when I ssh. bash doesn't: ssh -X helps, but I still need some tricky code in .bash_profile.
• aliases can have arguments, e.g. alias llth 'ls -laFt \!* | head' In bash these have to be functions, which means they don't show up typing "which" or "alias".
• Prompt settings are more flexible -- there are options like %B for bold. In bash you have to get the terminal type and use the ansi color escape sequances, which don't include bold.
• Easier command editing setup -- behaviors like getting word-erase to stop at punctuation don't involve chasing through multiple semi-documented programs, and the answer doesn't vary with version.
• Documentation -- tcsh's is mostly in man tcsh, bash's is scattered all over man pages for various helper programs. And it's hard to google for bash help because "bash" as a keyword gets you lots of stuff not related to shells.

Of course, you bash users, set me straight if I missed out on some bash options that would have solved some of these problems. And especially if you have a clue about the evil disappearing history commands!

bash wins

• You don't have to rehash every time you add a program or change your path. That's a real annoyance of tcsh, and I could understand a person used to bash rejecting tcsh on this alone.
• History remembers entire multi-line commands, and shows them with semicolons when you arrow back through history. Very nice. tcsh only remembers the first line and you have to retype the rest.
• Functions: although I don't like having to use them instead of aliases, they're certainly powerful and nice to have.

Of course, bash and tcsh aren't the only shells around. From what I hear, zsh blends the good features of bash and tcsh. One of these days I'll try it and see.

Tags: linux, shells, bash, csh
[ 21:58 Nov 07, 2007    More linux | permalink to this entry ]

Custom beep sounds

It's always amazed me that Linux doesn't let you customize the system beep. You can call xset b volume pitch duration to make it beep higher or lower, or you can turn it off or switch to "visual bell"; but there's nothing like the way most other OSes let you customize it to any sound you want. (Some desktops let you customize the desktop alerts, but that doesn't do anything about the beeping you get from vim, or emacs, or bash, or a hundred other programs that run in terminals.)

Today someone pointed me toward a Gentoo wiki page that led me to Fancy Beeper Daemon. This is a small kernel module that adds a device, /dev/beep, which outputs a byte every time there's a beep. You can write a very simple daemon (several samples in Python are included with the module) which reads /dev/beep and plays the sound of your choice.

I downloaded beep-2.6.15+.tar.gz, but it needed one tweak to build it under 2.6.23-rc3: I needed to add

#include <linux/sched.h>

among the includes at the beginning of beep.c. After that, it compiled and installed just fine. Since it's a kernel module, it works in consoles as well as under X.

/dev/beep is created with owner and group root, and readable/ writable only by owner and group, so to test it I had to chmod 666 /dev/beep to run the daemon. I fixed that by making a file in /etc/udev/rules.d called 41-beep.rules, containing:

KERNEL=="beep", GROUP="audio"

Finally, based on the nice samples that came with the module, I wrote my own very simple Python daemon, playbeepd, that uses aplay.

Lots of fun! I don't know how much I'll use it, but it's good to have the option of custom beep sounds.

Tags: linux, audio, kernel
[ 20:47 Sep 08, 2007    More linux | permalink to this entry ]

Widescreen Monitor, Intel Graphics on Ubuntu

On a seemingly harmless trip to Fry's, my mother got a look at the 22-inch widescreen LCD monitors and decided she had to have one. (Can't blame her ... I've been feeling the urge myself lately.)

We got the lovely new monitor home, plugged it in, configured X and discovered that the screen showed severe vertical banding. It was beautiful at low resolutions, but whenever we went to the monitor's maximum resolution of 1680x1050, the bands appeared.

After lots of testing, we tentatively pinned the problem down to the motherboard. It turns out ancient machines with 1x AGP motherboards can't drive that many pixels properly, even if the video card is up to the job. Who knew?

Off we trooped to check out new computers. We'd been hinting for quite some time that it might be about time for a new machine, and Mom was ready to take the plunge (especially if it meant not having to return that beautiful monitor).

We were hoping to find something with a relatively efficient Intel Core 2 processor and Intel integrated graphics: I've been told the Intel graphics chip works well with Linux using open source drivers. (Mom, being a person of good taste, prefers Linux, and none of us wanted to wrestle with the proprietary nvidia drivers). We found a likely machine at PC Club. They were even willing to knock $60 off the price since she didn't want Windows.

But that raised a new problem. During our fiddling with her old machine, we'd tried burning a Xubuntu CD, to see if the banding problem was due to the old XFree86 she was running. Installing it hadn't worked: her CD burner claimed it burned correctly, but the resulting CD had errors and didn't pass verification. So we needed a CD burned. We asked PC Club when buying the computer whether we might burn the ISO to CD, but apparently that counts as a "data transfer" and their minimum data transfer charge is $80. A bit much.

No problem -- a friend was coming over for dinner that night, and he was kind enough to bring his Mac laptop ... and after a half hour of fiddling, we determined that his burner didn't work either (it gave a checksum error before starting the burn). He'd never tried burning a CD on that laptop.

What about Kinko's? They have lots of data services, right? Maybe they can burn an ISO. So we stopped at Kinko's after dinner. They, of course, had never heard of an ISO image and had no idea how to burn one on their Windows box. Fearing getting a disk with a filesystem containing one file named "xubuntu-7.04-alternate-i386.iso", we asked if they had a mac, since we knew how to burn an ISO there. They did, though they said sometimes the CD burner was flaky. We decided to take the risk.

Burning an ISO on a mac isn't straightforward -- you have to do things in exactly the right order. It took some fast talking to persuade them of the steps ("No, it really won't work if you insert the blank CD first. Yes, we're quite sure") and we had to wait a long time for Kinko's antivirus software to decide that Xubuntu wasn't malware, but 45 minutes and $10 later, we had a disc.

And it worked! We first set up the machine in the living room, away from the network, so we had to kill aptitude update when the install hung installing "xubuntu-desktop" at 85% (thank goodness for alternate consoles on ctl-alt-F2) but otherwise the install went just fine. We rebooted, and Xubuntu came up ... at 1280x1024, totally wrong. Fiddling with the resolution in xorg.conf didn't help; trying to autodetect the monitor with dpkg-reconfigure xorg crashed the machine and we had to power cycle.

Back to the web ... turns out that Ubuntu "Feisty" ships with a bad Intel driver. Lots of people have hit the problem, and we found a few elaborate workarounds involving installing X drivers from various places, but nothing simple. Well, we hadn't come this far to take all the hardware back now.

First we moved the machine into the computer room, hooked up networking and reinstalled xubuntu with a full network, just in case. The resolution was still wrong. Then, with Dave in the living room calling out steps off a web page he'd found, we began the long workaround process.

"First," Dave suggested, reading, "check the version of xserver-xorg-video-intel. Let's make sure we're starting with the same version this guy is."

dpkg -l xserver-xorg-video-intel ... "Uh, it isn't installed," I reported. I tried installing it. "It wants to remove xserver-xorg-video-i810." Hmm! We decided we'd better do it, since the rest of the instructions depended on having the intel, not i810, driver.

And that was all it needed! The intel driver autodetected the monitor and worked fine at 1680x1050.

So forget the elaborate instructions for trying X drivers from various sources. The problem was that xubuntu installed the wrong driver: the i810 driver instead of the more generic intel driver. (Apparently that bug is fixed for the next Ubuntu release.)

With that fix, it was only a few more minutes before Mom was happily using her new system, widescreen monitor and all.

Tags: linux, X11, ubuntu
[ 13:23 Aug 25, 2007    More linux | permalink to this entry ]