Shallow Thoughts : : linux

Akkana's Musings on Open Source, Science, and Nature.

Tue, 22 Jun 2010

Helpful CUPS error messages, decoded

Another in the continuing series of "This isn't documented anywhere and Google searches aren't helpful":

Dave's printer was failing to print on Ubuntu Lucid. The only failure mode: pages came out with the single printed line,

Unable to open the initial device, quitting.

What a great, helpful error message! Thanks, CUPS!

Web searching found lots of people using HP printers, using various versions of the HPLIP software to installer newer drivers and otherwise reconfigure their HP setups.

Only problem: this printer isn't an HP. It's a Brother HL2070N, which has given very good service and, until now, worked flawlessly with every OS we've tried including Linux.

The solution? It turns out the problem really is HPLIP -- even if the printer isn't an HP. What this message really meant was:

HPLIP isn't installed, and Ubuntu's CUPS refuses to work without it.

apt-get install hplip hplip-data got the printer working again.

Wouldn't it be nice if CUPS bothered to print error messages that gave some hint of the real problem? Ideally, visible on the computer to the user, rather than on the printed page, so you don't have to waste 25 pages of dead tree while you try to narrow down the problem?

Update: After further testing, we have established that a standard Gnome-based Ubuntu Lucid machine needs hplip and hplip-data installed, while a Lucid machine without Gnome needs those two plus hpijs. Or you can get around needing any of them by ignoring CUPS' recommendation for which driver to use, and choosing the Gutenprint driver in the CUPS configuration screens.

A reader asked me if we had checked the CUPS error log. On one machine, the log file was virtually empty; on another, there actually were some lines about hpijs (nothing about hplip), intermixed with a lot of debug chatter and a large number of errors that were fairly clearly unrelated to anything we were doing.

Tags: , ,
[ 21:46 Jun 22, 2010    More linux | permalink to this entry ]

Fri, 18 Jun 2010

Use "date" to show time abroad

While I was in Europe, Dave stumbled on a handy alias on his Mac to check the time where I was: date -v +10 (+10 is the offset from the current time). But when he tried to translate this to Linux, he found that the -v flag from FreeBSD's date program wasn't available on the GNU date on Linux.

But I suggested he could do the same thing with the TZ environment variable. It's not documented well anywhere I could find, but if you set TZ to the name of a time zone, date will print out the time for that zone rather than your current one.

So, for bash:

$ TZ=Europe/Paris date  # time in Paris
$ TZ=GB date            # time in Great Britain
$ TZ=GMT-02 date        # time two timezones east of GMT
or for csh:
% ( setenv TZ Europe/Paris; date)
% ( setenv TZ GB; date)
% ( setenv TZ GMT-02; date)

That's all very well. But when I tried

% ( setenv TZ UK; date)
% ( setenv TZ FR; date)
they gave the wrong time, even though Wikipedia's list of time zones seemed to indicate that those abbreviations were okay.

The trick seems to be that setting TZ only works for abbreviations in /usr/share/zoneinfo/, or maybe in /usr/share/zoneinfo/posix/. If you give an abbreviation, like UK or FR or America/San_Francisco, it won't give you an error, it'll just print GMT as if that was what you had asked for.

So this trick is useful for printing times abroad -- but if you want to be safe, either stick to syntaxes like GMT-2, or make a script that checks whether your abbreviation exists in the directory before calling date, and warns you rather than just printing the wrong time.

Tags: , , ,
[ 13:04 Jun 18, 2010    More linux/cmdline | permalink to this entry ]

Sun, 13 Jun 2010

Enabling system beeps on Ubuntu Lucid

Update: though the rest of this article is still useful in explaining how to un-blacklist the pcspkr module, unfortunately that module works very erratically. Sometimes you'll get a beep, sometimes not. So this article may be a good start but it still doesn't explain why Ubuntu's kernels have such a flaky pcspkr module.

For years I've used Ubuntu with my own kernels rather than the kernels Ubuntu provides. I have several reasons: home-built kernels boot a lot faster (when I say a lot, I mean like saving 30 seconds off a one-minute boot) and offer more control over options. But a minor reason is that Ubuntu kernels generally don't support the system beep, so for example there's no way to tell in vim when you get out of insert mode. (In the past I've sometimes used the excellent fancy beeper module to play sounds, but I don't always want that.)

On Ubuntu's latest "Lucid Lynx", I'm using their kernel (so far). The Ubuntu kernel team has made huge improvements in boot time and number of modules loaded, so it's much more efficient than past kernels. But it did leave me without a beeper.

modprobe pcspkr failed to do anything except print the enigmatic:

WARNING: All config files need .conf: /etc/modprobe.d/00local, it will be ignored in a future release.
modprobe -v pcspkr (verbose) was no help -- it printed install /bin/true which didn't make anything clearer.

To get my beep back, I had to do two things:

First, edit /etc/modprobe.d/blacklist.conf and comment out the line blacklisting pcspeakr. It looks like this:

# ugly and loud noise, getting on everyone's nerves; this should be done by a
# nice pulseaudio bing (Ubuntu: #77010)
blacklist pcspkr
(They don't seem to be concerned about anyone who doesn't run Pulse, or about the various other bugs involved -- there's quite a laundry list in bug 486154.)

Secomd. pcspkr was blacklisted a second time in a different way, in that file so confusingly alluded to by the warning. /etc/modprobe.d/00local was apparently left over from a previous version of Ubuntu, and never removed by any upgrade script, and consisted of this:

install pcspkr /bin/true

Aha! So that's why modprobe -v pcspkr printed install /bin/true -- because that's all it was doing instead of loading the module like I'd asked.

So rm /etc/modprobe.d/00local was the second step, and once I'd done that, modprobe pcspkr loaded the module and gave me my system beep.

Tags: , ,
[ 13:02 Jun 13, 2010    More linux/kernel | permalink to this entry ]

Sun, 09 May 2010

The new udev in Lucid

Ubuntu's latest release, 10.04 "Lucid Lynx", really seems remarkably solid. It boots much faster than any Ubuntu of the past three years, and has some other nice improvements too.

But like every release, they made some pointless random undocumented changes that broke stuff. The most frustrating has been getting my front-panel flash card reader to work under Lucid's new udev, so I could read SD cards from my camera and PDA.

The SD card slot shows up as /dev/sdb, but unless there's a card plugged in at boot time, there's no /dev/sdb1 that you can actually mount.

hal vs udisks

Prior to Lucid, the "approved" way of creating sdb1 was to let hald-addons-storage poll every USB device every so often, to see if anyone has plugged in a card and if so, check its partition table and create appropriate devices.

That's a lot of polling -- and in any case, hald isn't standard on Lucid, and even when it's installed, it sometimes runs and sometimes doesn't. (I haven't figured out what controls whether it decides to run). Hal isn't even supposed to be needed on Lucid -- it's supposed to use devicekit (renamed to) udisks for that.

Except I guess they couldn't quite figure out how to get udisks working in time, so they patched things together so that on Gnome systems, hald does the same old polling stuff -- and on non Gnome systems, well, maybe it does and maybe it doesn't. And maybe you can't read your camera cards. Oh well!

udev rules

But on systems prior to Lucid there was another way: make a udev rule to create sdb1 through sdb15 every time. I have an older article on setting up udev rules for multicard readers, but none of my old udev rules worked on Lucid.

After many rounds of udevadm info -a -p /block/sdb and udevadm test /block/sdb, service udev restart, and many reboots, I finally found a rule that worked.

Create a /etc/udev/rules.d/71-multicard-reader.rules file containing the following:

# Create all devices for multicard reader:
KERNEL=="sd[b-g]", SUBSYSTEMS=="usb", ATTRS{idVendor}=="1d6b", ATTRS{idProduct}=="0002", OPTIONS+="all_partitions,last_rule"

Replace the 1d6b and 0002 with the vendor and product of your own device, as determined with udevadm info -a -p /block/sdb ... and don't be tempted to use the vendor and device ID you get from lsusb, because those are different.

What didn't work that used to? String matches. Some of them. For example, this worked:

KERNEL=="sd[b-g]", SUBSYSTEMS=="scsi", ATTRS{model}=="*SD*", NAME{all_partitions}="sdcard"
but these didn't:
KERNEL=="sd[b-g]", SUBSYSTEMS=="scsi", ATTRS{model}=="*SD*Reader*", NAME{all_partitions}="sdcard"
KERNEL=="sd[a-g]", SUBSYSTEMS=="scsi", ATTRS{model}=="USB SD Reader   ", NAME{all_partitions}="cardsd"

Update: The first of those two lines does indeed work now, whereas it didn't when I was testing. It's possible that this has something to do with saving hardware states and needing an extra udevadm trigger, as suggested in Alex's Changes in Ubuntu Lucid to udev.

According to udevadm info, the model is "USB SD Reader " (three spaces at the end). But somehow "*SD*" matches this while "*SD*Reader*" and the exact string do not. Go figure.

Numeric order

I'd like to have this rule run earlier, so it runs before /lib/udev/rules.d/60-persistent-storage.rules and could use OPTIONS+="last_rule" to keep the persistent storage rules from firing (they run a lot of unnecessary external programs for each device). But if I rename the rule from 71-multicard-reader.rules to 59-, it doesn't run at all. Why? Shrug. It's not like udevadm test will tell me.

Other things I love (not) about the new udev

Tags: , , , ,
[ 20:51 May 09, 2010    More linux/kernel | permalink to this entry ]

Wed, 05 May 2010

The luxury of understanding the problem

On a Linux list, someone was having trouble with wireless networking, and someone else said he'd had a similar problem and solved it by reinstalling Kubuntu from scratch. Another poster then criticised him for that: "if the answer is reinstall, you might as well downgrade to Windows.", and later added, "if "we should understand a problem, and *then* choose a remedy to match."

As someone who spends quite a lot of time trying to track down root causes of problems so that I can come up with a fix that doesn't involve reinstalling, I thought that was unfair. Here is how I replied on the list (or you can go straight to the mailing list version):

I'm a big fan of understanding the root cause of a problem and solving it on that basis. Because I am, I waste many days chasing down problems that ought to "just work", and probably would "just work" if I gave in and installed a bone stock Ubuntu Gnome desktop with no customizations. Modern Linux distros (except maybe Gentoo) are written with the assumption that you aren't going to change anything -- so reverting to the original (reinstalling) will often fix a problem.

Understanding this stuff *shouldn't* take days of wasted time -- but it does, because none of this crap has decent documentation. With a lot of the underlying processes in Linux -- networking, fonts, sound, external storage -- there are plenty of "Click on the System Settings menu, then click on ... here's a screenshot" howtos, but not much "Then the foo daemon runs the /etc/acpi/bar.sh script, which calls ifconfig with these arguments". Mostly you have to reverse-engineer it by running experiments, or read the source code.

Sometimes I wonder why I bother. It may be sort of obsessive-compulsive disorder, but I guess it's better than washing my hands 'til they bleed, or hoarding 100 cats. At least I end up with a nice customized system and more knowledge about how Linux works. And no cat food expenses.

But don't get on someone's case because he doesn't have days to waste chasing down deep understanding of a system problem. If you're going to get on someone's case, go after the people who write these systems and then don't document how they actually work, so people could debug them.

Tags: , ,
[ 18:37 May 05, 2010    More linux | permalink to this entry ]

Thu, 15 Apr 2010

Function keys on an Apple keyboard, on Linux

Quick tip from Dave, passed along to someone else trying to use an Apple keyboard on Linux:

On Linux, for some reason Apple keyboards' function keys don't work by default. Most of them try to run special functions instead, like volume up/down or play/pause.

But you can get normal function keys by talking to the kernel module that drives the keyboard:

echo 2 >  /sys/module/hid_apple/parameters/fnmode

This will only last until shutdown, so put that line in /etc/rc.local or a similar place so it runs every time you boot.

Here's an Ubuntu help page on Apple Keyboards with more information and other tricks.

Tags: , , ,
[ 15:03 Apr 15, 2010    More linux/kernel | permalink to this entry ]

Thu, 01 Apr 2010

Quick tip: User-mounted filesystems with "exec"

Second time I've run into this -- time to write it down.

Trying to run debootstrap to install the new Ubuntu beta, I kept hitting a snag right at the beginning: debootstrap kept saying the filesystem was mounted with the wrong permissions, and it couldn't create an executable file or a device.

I have lines for each of my spare filesystems in /etc/fstab, like this:

/dev/sda4  /lucid  ext3  relatime,user,noauto  0   0
That way, if I'm booted into one OS but I want to check a file from another, I can mount it without needing sudo, just by typing mount /lucid.

Not being able to create executable files means it's mounted with the noexec flag. I checked another machine and saw that it was using lines like

/dev/sda4  /lucid  ext3  exec,user,noauto  0   0

I added the "exec," to fstab, unmounted and remounted ... and it was still mounted with noexec.

Turns out on some Linux versions, making a filesystem user mountable turns off exec even if you've specified it explicitly. You have to add the exec after the user.

But that still didn't make debootstrap happy, because it couldn't create a device. That's a separate fstab option, dev, and user implies nodev.

So here's the fstab entry that finally worked:

/dev/sda4  /lucid  ext3  relatime,user,exec,dev,noauto  0   0

Tags: , , ,
[ 22:07 Apr 01, 2010    More linux | permalink to this entry ]

Mon, 29 Mar 2010

Non-greedy regular expressions to clean up crappy autogenerated HTML

I maintain the websites for several clubs. No surprise there -- anyone with even a moderate knowledge of HTML, or just a lack of fear of it, invariably gets drafted into that job in any non-computer club.

In one club, the person in charge of scheduling sends out an elaborate document every three months in various formats -- Word, RTF, Excel, it's different each time. The only regularity is that it's always full of crap that makes it hard to turn it into a nice simple HTML table.

This quarter, the formats were Word and RTF. I used unrtf to turn the RTF version into HTML -- and found a horrifying page full of lines like this:

<body><font size=3></font><font size=3><br>
</font><font size=3></font><b><font size=4></font></b><b><font size=4><table border=2>
</font></b><b><font size=4><tr><td><b><font size=4><font face="Arial">Club Schedule</font></font></b><b><font size=4></font></b><b><font size=4></font></b></td>
<font size=3></font><font size=3><td><font size=3><b><font face="Arial">April 13</font></b></font><font size=3></font><font size=3><br>
</font><font size=3></font><font size=3><b></b></font></td>
I've put the actual page content in bold; the rest is just junk, mostly doing nothing, mostly not even legal HTML, that needs to be eliminated if I want the page to load and display reasonably.

I didn't want to clean up that mess by hand! So I needed some regular expressions to clean it up in an editor. I tried emacs first, but emacs makes it hard to try an expression then modify it a little when the first try doesn't work, so I switched to vim.

The key to this sort of cleanup is non-greedy regular expressions. When you have a bad tag sandwiched in the middle of a line containing other tags, you want to remove everything from the <font through the next > -- but no farther, or else you'll delete real content. If you have a line like

<td><font size=3>Hello<font> world</td>
you only want to delete through the <font>, not through the </td>.

In general, you make a regular expression non-greedy by adding a ? after the wildcard -- e.g. <font.*?>. But that doesn't work in vim. In vim, you have to use \{M,N} which matches from M to N repetitions of whatever immediately precedes it. You can also use the shortcut \{-} to mean the same thing as *? (0 or more matches) in other programs.

Using that, I built up a series of regexp substitutes to clean up that unrtf mess in vim:

:%s/<\/\{0,1}font.\{-}>//g
:%s/<b><\/b>//g
:%s/<\/b><b>//g
:%s/<\/i><i>//g
:%s/<td><\/td>/<td><br><\/td>/g
:%s/<\/\{0,1}span.\{-}>//g
:%s/<\/\{0,1}center>//g

That took care of 90% of the work, leaving me with hardly any cleanup I needed to do by hand. I'll definitely keep that list around for the next time I need to do this.

Tags: , , ,
[ 22:02 Mar 29, 2010    More linux/editors | permalink to this entry ]