Shallow Thoughts : tags : install

Updating Debian under a chroot

Debian's Unstable ("Sid") distribution has been terrible lately. They're switching to a version of X that doesn't require root, and apparently the X transition has broken all sorts of things in ways that are hard to fix and there's no ETA for when things might get any better.

And, being Debian, there's no real bug system so you can't just CC yourself on the bug to see when new fixes might be available to try. You just have to wait, try every few days and see if the system

That's hard when the system doesn't work at all. Last week, I was booting into a shell but X wouldn't run, so at least I could pull updates. This week, X starts but the keyboard and mouse don't work at all, making it hard to run an upgrade. has been fixed.

Fortunately, I have an install of Debian stable ("Jessie") on this system as well. When I partition a large disk I always reserve several root partitions so I can try out other Linux distros, and when running the more experimental versions, like Sid, sometimes that's a life saver. So I've been running Jessie while I wait for Sid to get fixed. The only trick is: how can I upgrade my Sid partition while running Jessie, since Sid isn't usable at all?

I have an entry in /etc/fstab that lets me mount my Sid partition easily:

/dev/sda6 /sid ext4 defaults,user,noauto,exec 0 0

So I can type mount /sid as myself, without even needing to be root.

But Debian's apt upgrade tools assume everything will be on /, not on /sid. So I'll need to use chroot /sid (as root) to change the root of the filesystem to /sid. That only affects the shell where I type that command; the rest of my system will still be happily running Jessie.

Mount the special filesystems

That mostly works, but not quite, because I get a lot of errors like permission denied: /dev/null.

/dev/null is a device: you can write to it and the bytes disappear, as if into a black hole except without Hawking radiation. Since /dev is implemented by the kernel and udev, in the chroot it's just an empty directory. And if a program opens /dev/null in the chroot, it might create a regular file there and actually write to it. You wouldn't want that: it eats up disk space and can slow things down a lot.

The way to fix that is before you chroot: mount --bind /dev /sid/dev which will make /sid/dev a mirror of the real /dev. It has to be done before the chroot because inside the chroot, you no longer have access to the running system's /dev.

But there is a different syntax you can use after chrooting:

mount -t proc proc proc/
mount --rbind /sys sys/
mount --rbind /dev dev/

It's a good idea to do this for /proc and /sys as well, and Debian recommends adding /dev/pts (which must be done after you've mounted /dev), even though most of these probably won't come into play during your upgrade.

Mount /boot

Finally, on my multi-boot system, I have one shared /boot partition with kernels for Jessie, Sid and any other distros I have installed on this system. (That's somewhat hard to do using grub2 but easy on Debian though you may need to turn off auto-update and Debian is making it harder to use extlinux now.) Anyway, if you have a separate /boot partition, you'll want it mounted in the chroot, in case the update needs to add a new kernel. Since you presumably already have the same /boot mounted on the running system, use mount --bind for that as well.

So here's the final set of commands to run, as root:

mount /sid
mount --bind /proc /sid/proc
mount --bind /sys /sid/sys
mount --bind /dev /sid/dev
mount --bind /dev/pts /sid/dev/pts
mount --bind /boot /sid/boot
chroot /sid

And then you can proceed with your apt-get update, apt-get dist-upgrade etc. When you're finished, you can unmount everything with one command:

umount --recursive /sid

Some helpful background reading:

• Change root on the Arch Linux wiki: thorough and readable, like most Arch Wiki pages, and applicable to any distro, not just Arch
• chroot on the Debian wiki: A good discussion of using debootstrap to install a new version of Debian, but less good as an explanation of chroot
• mount dev, proc, sys in a chroot environment? discussion on SuperUser

Tags: linux, debian, chroot, install, boot
[ 11:43 Feb 05, 2016    More linux/install | permalink to this entry | ]

How To Satisfy Debian Dependencies Without Installing The Stupid Package

(This is a guest post by David North.)

Debian developers tend to get overzealous in their dependency lists, probably to avoid constant headaches from fringe cases whose favorite programs fail because they also need some obscure library or package support (and yes, I'm talking to you, Ubuntu). But what if you don't want some goofy dependency (and the cascade of other crap it pulls in?)

As a small aside, aptitude/apt-get hold <pkg> is terrific if you just want to keep a package at a pre-horkage level, but for some obcure reason you can't "hold" a package that isn't installed. So that won't work as of 11/2012.

You can however generate an equivalent package with a higher version number and install it, which naturally blocks the offending package. Even better, the replacement package need do nothing at all other than satisfy the apt database. Even better, the whole thing is incredibly simple.

First install the "equivs" package. This will deliver two programs:

• equivs-control
• equivs-build

Officially you should start with 'equivs-control <:pkgname>' which will create a file 'pkgname' in the current directory. Inside are various fields but you only need eight and can simply delete the rest. Here's approximately what you should end up with for a fictional package "pkgname":

Section: misc
Priority: optional
Standards-Version: 3.9.2

Package: pkgname
Version: 1:42
Maintainer: Your Name <your@email.address>
Architecture: all
Description: fake pkgname to block a dumb dependency

The first three lines are just boilerplate, though you may have to increment the standards-version at some point if you reuse the file. No changes are needed now.

The pkgname does actually have to match the name of the package you want to block. The version must be higher than that of the target package. Maintainer need not be you, but it's a good idea to at least use a name you recognize as yourself. Architecture can be left as "all" unless you're doing something extra tricky. Description is not necessary but a good idea; put your notes here.

The only trick is the version. Note the 1:42 structure here. The first number is the "epoch" in debian-speak, and may or may not be used. In practice I've never seen an epoch greater than one, so I suggest using either 1 or 2 here rather than just leaving it blank. You can see the epoch number in a package when you use aptitude show <pkgname>. The version is the number immediately after the colon, and for safety's sake should be considerably larger than the version you're trying to block (to avoid future updates). I like to use "42" for obvious reasons unless the actual package version is too close. Factoid: if no "epoch" is indicated debian will assume epoch 0, which will not show up as a zero in a .deb (or in aptitude show) but rather as a blank. The version number will have no colon in this event.

Having done this, all you need do is issue the command 'equivs-build path-to-pkgname' (preferably from the same directory) and you get a fake deb to install with dpkg -i. Say goodbye to the dependency.

One more trick: once you have your file <pkgname> with the Eight Important Fields, you can pretty much skip using equivs-control. All it does is make the initial text file, and it will be easier to edit the one you already have with a new package name (and rename the file at the same time). Note, however, this handy file will not necessarily be useful on other debian-based systems or later installs, so running equivs-control after a big upgrade or moving to another distro is very good practice. If you compare the files and they have the same entries, great. If not, use the new ones.

Tags: linux, debian, ubuntu, install
[ 11:50 Nov 14, 2012    More linux/install | permalink to this entry | ]

Making an Ubuntu live USB stick persistent

I wrote a few days ago about my multi-distro Linux live USB stick. Very handy!

But one thing that bugs me about live distros: they're set up with default settings and don't have a lot of the programs I want to use. Even getting a terminal takes quite a lot of clicks on most distros. If only they would save their settings!

It's possible to make a live USB stick "persistent", but not much is written about it. Most of what's written tells you to create the USB stick with usb-creator -- a GUI app that I've tried periodically for the past two years without ever once succeeding in creating a bootable USB stick.

Even if usb-creator did work, it wouldn't work with a multi-boot stick like this one, because it would want to overwrite the whole drive. So how does persistence really work? What is usb-creator doing, anyway?

How persistence works: Casper

The best howto I've found on Ubuntu persistence is LiveCD Persistence. But it's long and you have to wade through a lot of fdisk commands and similar arcana. So here's how to take your multi-distro stick and make at least one of the installs persistent.

Ubuntu persistence uses a package called casper which overlays the live filesystem with the contents of another filesystem. Figuring out where it looks for that filesystem is the key.

Casper looks for its persistent storage in two possible places: a partition with the label "casper-rw", and a file named "casper-rw" at the root of its mounted partitions.

So you could make a separate partition labeled "casper-rw", using your favorite partitioning tool, such as gparted or fdisk. But if you already have your multi-distro stick set up as one big partition, it's just as easy to create a file. You'll have to decide how big to make the file, based on the size of your USB stick.

I'm using a 4G stick, and I chose 512M for my persistent partition:

$ dd if=/dev/zero of=/path/to/casper-rw bs=1M count=512

Be patient: this step takes a while.

Next, create a filesystem inside that file. I'm not sure what the tradeoffs are among various filesystem types -- no filesystem is optimized for being run as a loopback file read from a vfat USB stick that was also the boot device. So I flipped a coin and used ext3:

$ mkfs.ext3 /path/to/casper-rw
/path/to/casper-rw is not a block special device.
Proceed anyway? (y,n) y

One more step: you need to add the persistent flag to your boot options. If you're following the multi-distro USB stick tutorial I linked to earlier, that means you should edit boot/grub/grub.cfg on the USB stick, find the boot stanza you're using for Ubuntu, and make the line starting with linux look something like this:

    linux (loop)/casper/vmlinuz boot=casper iso-scan/filename=$isofile quiet splash noprompt persistent --

Now write the stick, unmount it, and try booting your live install.

Testing: did it work?

The LiveCD/Persistence page says persistent settings aren't necessarily saved for the default "ubuntu" user, so it's a good idea to make a new user. I did so.

Oops -- about that Ubuntu new user thing

But at least in Ubuntu Oneiric: there's a problem with that. If you create a user, even as class Administrator (and of course you do want to be an Administrator), it doesn't ask you for a password. If you now log out or reboot, your new user should be saved -- but you won't be able to do anything with the system, because anything that requires sudo will prompt you for your nonexistent password. Even attempting to set a password will prompt you for the nonexistent password.

Apparently you can "unlock" the user at the time you create it, and then maybe it'll let you set a password. I didn't know this beforehand, so here's how to set a password on a locked user from a terminal:

$ sudo passwd username

For some reason, sudo will let you do this without prompting for a password, even though you can't do anything administrative through the GUI.

Testing redux

Once you're logged in as your new user, try making some changes. Add and remove some items from the unity taskbar. Install a couple of packages. Change the background.

Now try rebooting. If your casper-rw file worked, it should remember your changes.

When you're not booted from your live USB stick, you can poke around in the filesystem it uses by mounting it in "loopback" mode. Plug the stick into a running Linux machine, mount it the usb stick, then mount it with

$ sudo mount -o loop /path/to/casper-rw /mnt

/path/to is wherever you mounted your usb stick -- e.g. /media/whatever. With the file mounted in loopback mode, you should be able to adjust settings or add new files without needing to boot the live install -- and they should show up the next time you use the live install.

My live Ubuntu Oneiric install is so much more fun to use now!

Tags: ubuntu, linux, install, grub
[ 15:41 Oct 28, 2011    More linux/install | permalink to this entry | ]

Creating a multi-distro Linux Live USB stick

Linux live USB sticks (flash drivers) are awesome. You can carry them anywhere and give a demo of Linux on anyone's computer, any time. But how do you keep track of them? Especially since USB sticks don't have any place to write a label. How do you remember that the shiny blue stick is the one with Ubuntu Oneiric, the black one has Ubuntu Lucid, the other blue one that's missing its top is Debian ... and so forth. It's impossible! Plus, such a waste -- you can hardly buy a flash drive smaller than 4G these days, and then you go and devote it to a 700Mb ISO designed to fit on a CD. Silly.

The answer: get one big USB stick and put lots of distros on it, using grub to let you choose at boot time.

To create my stick, I followed the easy instructions at HOWTO: Booting LiveCD ISOs from USB flash drive with Grub2. I found that tutorial quite simple, so I'm not going to duplicate the instructions there. I used the non-LUA version, since my grub on Ubuntu Natty didn't seem to support LUA. Basically you run grub-install to the stick, create a directory called iso where you stick all your ISO files, then create a grub.cfg with magic incantations to boot each ISO.

Ah, wait ... magic incantations? The tutorial is missing one important part: what if you want to use an ISO that isn't already mentioned in the tutorial? If Ubuntu's entry is
linux (loop)/casper/vmlinuz boot=casper iso-scan/filename=$isofile quiet splash noprompt -- and Parted Magic's is
linux (loop)/pmagic/bzImage iso_filename=$isofile edd=off noapic load_ramdisk=1 prompt_ramdisk=0 rwnomce sleep=10 loglevel=0 then you know there's some magic going on there.

I knew I needed at least the Ubuntu "alternate installer", since it allows installing a command-line system without the Unity desktop, and Debian Squeeze, since that's currently the most power-efficient Linux for laptops, in addition to the distros mentioned in the tutorial. How do you figure out what to put in those grub.cfg lines? Here's how to figure it out from the ISO file. I'll use the Debian Squeeze ISO as an example.

Step 1: mount the ISO file.

$ sudo mount -o loop /pix/boot/isos/debian-6.0.0-i386-netinst.iso /mnt

Step 2: find the kernel

$ ls /mnt/*/vmlinuz /mnt/*/bzImage
/mnt/install.386/vmlinuz

Step 3: find the initrd. It might have various names, and might or might not be compressed, but the name will almost always start with init.

$ ls /mnt/*/vmlinuz /mnt/*/init*
/mnt/install.386/initrd.gz

Unmount the ISO file.

$ umount /mnt

The trick in steps 2 and 3 is that nearly all live ISO images put the kernel and initrd a single directory below the root. If you're using an ISO that doesn't, you may have to search more deeply (try /mnt/*/*).

In the case of Debian Squeeze, now I have the two filenames: /install.386/vmlinuz and /install.386/initrd.gz. (I've removed the /mnt part since that won't be there when I'm booting from the USB stick.) Now I can edit boot/grub/grub.cfg and make a boot stanza for Debian:

menuentry "Debian Squeeze" {
    set isofile="/boot/isos/debian-6.0.0-i386-netinst.iso"

    loopback loop $isofile 
    linux (loop)/install.386/vmlinuz iso_filename=$isofile quiet splash noprompt --
    initrd (loop)/install.386/initrd.gz
}

Here's the entry for the Ubuntu alternate installer:

menuentry "Oneiric 11.10 alternate" {
    set isofile="/boot/isos/ubuntu-11.10-alternate-i386.iso"
 
    loopback loop $isofile 
    linux (loop)/install/vmlinuz iso_filename=$isofile
    initrd (loop)/install/initrd.gz
}

It sounds a little convoluted, I know -- but you only have to do it once, and then you have this amazing keychain drive with every Linux distro on it you can think of. Amaze your friends!

Tags: linux, install, ubuntu, debian, grub
[ 22:21 Oct 25, 2011    More linux/install | permalink to this entry | ]

Ubuntu "Natty Narwhal" on the ExoPC

Intel hosted a MeeGo developer camp on Friday where they gave out ExoPC tablets for developers, and I was lucky enough to get one.

Intel is making a big MeeGo push -- they want lots of apps available for this platform, so they're trying to make it as easy as possible for develoeprs to make new apps for their AppUp store.

Meego looks fun -- it's a real Unix under the hood, with a more or less mainstream kernel and a shell. I'm looking forward to developing for it; in theory it can run Python programs (using Qt or possibly even gtk for the front end) as well as C++ Qt apps. Of course, I'll be writing about MeeGo developing once I know more about it; for now I'm still setting up my development environment.

But on a lazy Saturday, I thought it would be fun to see if the new Ubuntu 11.04, "Natty Narwhal", can run on the ExoPC. Natty's whizzy new "Unity" interface (actually not new, but much revamped since the previous Ubuntu release) is rumoured to be somewhat aimed at tablets with touchscreens. How would it work on the ExoPC?

Making a bootable Ubuntu USB stick

The first step was to create a bootable USB stick with Ubuntu on it. Sadly, this is not as easy as on Fedora or SuSE. Ubuntu is still very CD oriented, and to make a live USB stick you need to take an ISO intended for a CDROM then run a program that changes it to make it bootable from USB.

There are two programs for this: usb-creator and unetbootin. In the past, I've had zero luck getting these programs to work except when running under a Gnome desktop on the same version of Ubuntu I was trying to install. Maybe it would be better this time.

I tried usb-creator-gtk first, since that seems to be the one Ubuntu pushes most. It installed without too many extra dependencies -- it did pull in several PolicyKit libraries like libpolkit-backend-1-0 and libpolkit-gobject-1-0. When I ran it, it saw the USB stick right away, and I chose the ubuntu-11.04-desktop-i386.iso file I'd downloaded. But the Make Startup Disk button remained blank. I guess I needed to click the Erase Disk button first. So I did -- and was presented with an error dialog that said:

org.freedesktop.DBus.Error.ServiceUnknown: The name org.freedesktop.PolicyKit1 was not provided by any service files

Dave (who's wrestled with this problem more than I have) suggested maybe it wanted a vfat partition on the USB stick. So I quit usb-creator-gtk, used gparted to make the stick into a single vfat partition, and restarted usb-creator-gtk. Now everything was un-greyed -- so I clicked Make Startup Disk and was immediately presented with another dialog:

Installation failed

No clue about what went wrong or why. Okay, on to unetbootin.

When I ran unetbootin, it gave me a helpful dialog that "unetbootin must be run as root." Then it proceeded to show its window anyway. I can read, so I quit and ran it again as root. I chose the iso file, clicked OK -- and it worked! In a minute or two I had a bootable Ubuntu USB stick.

(Update: unetbootin is better than usb-creator for another reason: you can use it to burn CDs other than the default live desktop CD -- like if you want to burn the "alternate installer" ISO so you can install server systems, use RAID partitions, etc.)

Booting on the ExoPC

Natty booted up just fine! I inserted the USB stick, powered on, leapt for the XXX button that shows the boot menu and told it to boot from the stick. Natty booted quite fast, and before long I was in the Unity desktop, and, oddly, it started off in a banshee screen telling me I didn't have any albums installed. I dismissed banshee ...

... at which point I found I couldn't actually do much without a keyboard. I couldn't sign on to our wi-fi since I couldn't type the password, and I didn't have any local files installed. But wait! I had an SD card with some photos on it, and Ubuntu recognized it just fine and popped up a file browser.

But I wanted to try net access. I borrowed Dave's Mac USB keyboard to type in the WPA password. It worked fine, and soon I was signed on to wi-fi and happily browsing the web.

"onboard" keyboard

What about an onscreen keyboard, though? I found one, called "onboard". It's installed by default. Unfortunately, I couldn't find a way to run it without a keyboard. Unity has a "+" button that took me to a window with a text field labeled Search Applications, but you have to type something there before it will show you any applications. I couldn't find any way to get a list of applications without a keyboard.

With a keyboard, I was able to find a terminal app, from which I was able to run onboard. It's tiny! Far too small for me to type on a capacitive display, even with my tiny fingers. It has no man page, but it does have a --help argument, by which I was able to discover the -s argument: onboard -s 900x300 did nicely. It's ugly, but I can live with that. Now if I can figure out how to make a custom Unity launcher for that, I'll be all set.

Unity on tablets -- not quite there yet

With onboard running, I gave Dave back his keyboard, and discovered a few other problems. I couldn't scroll in the file browser window: the scrollbar thumb is only a few pixels wide, too narrow to hit with a finger on a touchscreen, and the onboard keyboard has no up/down arrows or Page Up/Down. I tried dragging with two fingers, but no dice.

Also, when I went back to that Unity Search Applications screen, I discovered it takes up the whole screen, covering the onscreen keyboard, and there's no way to move it so I can type.

Update: forgot to mention that Unity, for all its huge Playskool buttons, has a lot of very small targets that are hard to hit with a finger. It took me two or three tries to choose the wi-fi icon on the top bar rather than the icon to the left or right of it, and shutdown is similarly tricky.

So Natty's usability on tablets isn't quite there. Still, I'm impressed at how easy it was to get this far. I didn't expect it to boot, run and be so usable without any extra work on my part. Very cool!

And no, I won't be installing Natty permanently on the ExoPC. I got this tablet for MeeGo development and I'm not welching on that deal. But it's fun to know that it's so easy to boot Ubuntu when I want to.

Tags: exopc, ubuntu, tablet, boot, install
[ 13:46 Apr 30, 2011    More linux/install | permalink to this entry | ]

Installing Debian Squeeze

Over the weekend I tried installing Debian's new release, "Squeeze", on my Vaio TX650 laptop.

I used a "net install" CD, the one that installs only the bare minimum then goes to the net for anything else. I used Expert mode, because I needed to set a static IP address and keep it from overwriting my grub configuration.

Most of the install went smoothly -- until I got to the last big step near the end, "Select and install software", where it froze at 1%.

A little web searching (on another machine) gave me the hint that the Debian installer prints a log on the fourth console, Ctrl-Alt-F4. Checking that log made the problem clear: aptitude was complaining about packages without a proper GPG signature -- type Yes to continue without verifying signatures. But since this was running inside the installer, there's no place to type Yes -- that Ctrl-Alt-F4 console is merely displaying messages, not accepting input, and the installer doesn't accept any input for aptitude.

Fortunately, "Select and install software" isn't crucial to the net install process. I don't actually know what software it would have installed -- it never asked me to choose any -- but without it, you should still have a working minimal Debian on the disk. So I made another console on Ctrl-Alt-F2, ran ps aux, found that aptitude was the highest numbered process running, and killed it. Upon returning to the installer (Ctrl-Alt-F1), I was able to skip "Select and install software", finish the install process and reboot.

Upon rebooting, I logged in as root and ran apt-get update. It complained about GPG errors; but now I could do something about it. I ran apt-get upgrade and confirmed that I wanted to proceed even without verifying package signatures. When that was over, the problem was fixed: a subsequent apt-get update ran without errors.

This ISO was downloaded (from the kernel.org mirror, I believe) a few days after the official release. I'm told that Debian changes the keys at the last minute before a release; perhaps the new keys don't make it into the ISO images on all the mirrors. Or maybe they just messed up with the Squeeze release.

Anyway, it was fairly easily solved, but seemed like a disappointing and silly problem. A web search found lots of people people hitting this problem; it's a shame that the installer can't run aptitude in a mode where it won't prompt and hang up the whole install.

Alas, it's probably all academic anyway, since suspend/resume doesn't work. It freezes on resume, with a black screen -- another common Debian problem, judging by what I see on the net. I'm a bit surprised, since every other distro I've tried has suspended the Vaio beautifully. But after hours of messing with it over the weekend, I ran out of time and conceded defeat.

Tags: install, debian, linux
[ 22:49 Mar 22, 2011    More linux/install | permalink to this entry | ]

Making CapsLock equal Control in Debian Squeeze

It's time for another installment of "Where have the control/capslock adjustments migrated to?" This time it's for the latest Debian release, "Squeeze".

Ever since they stopped making keyboards with the control key to the left of the A, I've remapped my CapsLock key to be another Control key. I never need CapsLock, but I use Control constantly all day while editing text. Some people prefer to swap Control and CapsLock.

But the right way to do that changes periodically. For the last few years, since Ubuntu Intrepid, you could set XKbOptions for Control and Capslock in /etc/default/console-setup. But that no longer works in Squeeze.

It turns out Squeeze introduced a new file, /etc/default/keyboard, so any keyboard options previously had in console-setup need to move to keyboard. For me, that's these lines:

XKBMODEL="pc104"
XKBLAYOUT="us"
XKBVARIANT=""
XKBOPTIONS="ctrl:nocaps,compose:menu,terminate:ctrl_alt_bksp"

though I suspect only the last line matters.

This wasn't well covered on the web. There aren't many howtos covering Squeeze yet, but I found the hint I needed in a terse Debian IRCbot factoid: Factoid capslock says

For console-setup, append ",ctrl:nocaps" to the value of XKBOPTIONS within /etc/default/console-setup (/etc/default/keyboard on Squeeze).

That factoid assumes you already have XKBOPTIONS set; as shipped, it's empty, so skip that initial comma.

I was going to conclude with a link to the documentation on XKBOPTIONS, or XKbOptions as it was capitalized in xorg.conf ... but there doesn't seem to be any. It's not in any of the Xorg man pages like xorg.conf(5) where I expected to find it; nor can I find anything on the web beyond howtos like this one from people who have figured out a few specific options. Anyone know?

Tags: install, debian, linux
[ 12:54 Mar 20, 2011    More linux/install | permalink to this entry | ]

The new udev in Lucid

Ubuntu's latest release, 10.04 "Lucid Lynx", really seems remarkably solid. It boots much faster than any Ubuntu of the past three years, and has some other nice improvements too.

But like every release, they made some pointless random undocumented changes that broke stuff. The most frustrating has been getting my front-panel flash card reader to work under Lucid's new udev, so I could read SD cards from my camera and PDA.

The SD card slot shows up as /dev/sdb, but unless there's a card plugged in at boot time, there's no /dev/sdb1 that you can actually mount.

hal vs udisks

Prior to Lucid, the "approved" way of creating sdb1 was to let hald-addons-storage poll every USB device every so often, to see if anyone has plugged in a card and if so, check its partition table and create appropriate devices.

That's a lot of polling -- and in any case, hald isn't standard on Lucid, and even when it's installed, it sometimes runs and sometimes doesn't. (I haven't figured out what controls whether it decides to run). Hal isn't even supposed to be needed on Lucid -- it's supposed to use devicekit (renamed to) udisks for that.

Except I guess they couldn't quite figure out how to get udisks working in time, so they patched things together so that on Gnome systems, hald does the same old polling stuff -- and on non Gnome systems, well, maybe it does and maybe it doesn't. And maybe you can't read your camera cards. Oh well!

udev rules

But on systems prior to Lucid there was another way: make a udev rule to create sdb1 through sdb15 every time. I have an older article on setting up udev rules for multicard readers, but none of my old udev rules worked on Lucid.

After many rounds of udevadm info -a -p /block/sdb and udevadm test /block/sdb, service udev restart, and many reboots, I finally found a rule that worked.

Create a /etc/udev/rules.d/71-multicard-reader.rules file containing the following:

# Create all devices for multicard reader:
KERNEL=="sd[b-g]", SUBSYSTEMS=="usb", ATTRS{idVendor}=="1d6b", ATTRS{idProduct}=="0002", OPTIONS+="all_partitions,last_rule"

Replace the 1d6b and 0002 with the vendor and product of your own device, as determined with udevadm info -a -p /block/sdb ... and don't be tempted to use the vendor and device ID you get from lsusb, because those are different.

What didn't work that used to? String matches. Some of them. For example, this worked:

KERNEL=="sd[b-g]", SUBSYSTEMS=="scsi", ATTRS{model}=="*SD*", NAME{all_partitions}="sdcard"

but these didn't:

KERNEL=="sd[b-g]", SUBSYSTEMS=="scsi", ATTRS{model}=="*SD*Reader*", NAME{all_partitions}="sdcard"
KERNEL=="sd[a-g]", SUBSYSTEMS=="scsi", ATTRS{model}=="USB SD Reader   ", NAME{all_partitions}="cardsd"

Update: The first of those two lines does indeed work now, whereas it didn't when I was testing. It's possible that this has something to do with saving hardware states and needing an extra udevadm trigger, as suggested in Alex's Changes in Ubuntu Lucid to udev.

According to udevadm info, the model is "USB SD Reader " (three spaces at the end). But somehow "*SD*" matches this while "*SD*Reader*" and the exact string do not. Go figure.

Numeric order

I'd like to have this rule run earlier, so it runs before /lib/udev/rules.d/60-persistent-storage.rules and could use OPTIONS+="last_rule" to keep the persistent storage rules from firing (they run a lot of unnecessary external programs for each device). But if I rename the rule from 71-multicard-reader.rules to 59-, it doesn't run at all. Why? Shrug. It's not like udevadm test will tell me.

Other things I love (not) about the new udev

• I love how if you give the udevadm info arguments in the wrong order, -p -a, it means something else and gives an error message.
• I love how udevadm test doesn't actually test the same rules udev will use, so it's completely unrelated to anything.
• I love the complete lack of documentation on things like string matching and how the numeric order is handled.
• I love how you can't match both the device name (a string) and the USB IDs in the same rule, because one is SUBSYSTEMS=="scsi" and the other is SUBSYSTEMS=="usb".
• Finally, I love how there's no longer any way to test udev rules on a running system -- if you want it to actually create new devices, you have to reboot for each new test. service udev restart and udevadm control --reload-rules don't touch existing devices. Gives me that warm feeling like maybe I'm not missing out on the full Windows experience by using Linux.

Tags: linux, ubuntu, kernel, udev, install
[ 21:51 May 09, 2010    More linux/kernel | permalink to this entry | ]

Creating a Linux Live USB stick: Big win for Fedora

Three times now I've gotten myself into a situation where I was trying to install Ubuntu and for some reason couldn't burn a CD. So I thought hey, maybe I can make a bootable USB image on this handy thumb drive here. And spent the next three hours unsuccessfully trying to create one. And finally gave up, got in the car and went to buy a new CD burner or find someone who could burn the ISO to a CD because that's really the only way you can install or run Ubuntu.

There are tons of howtos on the web for creating live USB sticks for Ubuntu. Almost all of them start with "First, download the CD image and burn it to a CD. Now, boot off the CD and ..."

The few that don't discuss apps like usb-creator-gtk or unetbootin tha work great if you're burning the current Ubuntu Live CD image from a reasonably current Ubuntu machine, but which fail miserably in every other case (wildly pathological cases like burning the current Ubuntu alternate installer CD from the last long-term-support version of Ubuntu. I mean, really, should that be so unusual?)

Tonight, I wanted a bootable USB of Fedora 12. I tried the Ubuntu tools already mentioned, but usb-creator-gtk won't even try with an image that isn't Ubuntu, and unetbootin wrote something but the resulting stick didn't boot.

I asked on the Fedora IRC channel, where a helpful person pointed me to this paragraph on copying an ISO image with dd.

Holy mackerel! One command:

dd if=Fedora-12-i686-Live.iso of=/dev/sdf bs=8M

and in less than ten minutes it was ready. And it booted just fine!

Really, Ubuntu, you should take a look at Fedora now and then. For machines that are new enough, USB boot is much faster and easier than CD burning -- so give people an easy way to get a bootable USB version of your operating system. Or they might give up and try a distro that does make it easy.

Tags: linux, install, fedora, ubuntu
[ 23:01 Mar 27, 2010    More linux/install | permalink to this entry | ]

Fedora 11 not encouraging so far

Noticing that every article I write ends up including a section on "This doesn't work under Ubuntu; here's a link to the year-old bug with a patch and several workarounds," I decided to try Fedora 11 and see if it was any better.

I downloaded and burned the latest netinst CD ISO and booted it on the Atom machine. It greeted me with this prompt:

    1.
    2.
Select CD-ROM boot type: 

I am not getting warm fuzzies about Fedora being the solution to the Linux distro problem.

What do you think? Should I choose 1. or 2. ?

Tags: linux, install, fedora
[ 21:09 Aug 23, 2009    More linux/install | permalink to this entry | ]

Upgrading without risk

Someone asked on a mailing list whether to upgrade to a new OS release when her current install was working so well. I thought I should write up how I back up my old systems before attempting a risky upgrade or new install.

On my disks, I make several relatively small partitions, maybe 15G or so (pause to laugh about what I would have thought ten or even five years ago if someone told me I'd be referring to 15G as "small"), one small shared /boot partition, a swap partition, and use the rest of the disk for /home or other shared data.

Now you can install a new release, like 9.04, onto a new partition without risking your existing install.

If you prefer upgrading rather than running the installer, you can do that too. I needed a jaunty (9.04) install to test whether a bug was fixed. But my intrepid (8.10) is working fine and I know there are some issues with jaunty, so I didn't want to risk the working install. So from Intrepid, I copied the whole root partition over to one of my spare root partitions, sda5:

mkfs.ext3 /dev/sda5
mkdir /jaunty
mount /dev/sda5 /jaunty
cp -ax / /jaunty

(that last step takes quite a while: you're copying the whole system.)

Now there are a couple of things you have to do to make that /jaunty partition work as a bootable install:

1. /dev on an ubuntu system isn't a real file system, but something magically created by the kernel and udev. But to boot, you need some basic stuff there. When you're up and running, that's stored in /dev/.static, so you can copy it like this: cp -ax /dev/.static/dev/ /jaunty/

2. /etc/fstab on the system you just created points to the wrong root partition, so you have to fix that. As root, edit /etc/fstab in your favorite editor (e.g. sudo vim /etc/fstab or whatever) and find the line for the root filesystem -- the one where the second entry on the line is /. It'll look something like this:

# /dev/sda1
UUID=f7djaac8-fd44-672b-3432-5afd759bc561  /  ext3  relatime,errors=remount-ro  0 1

The easy fix is to change that to point to your new disk partition:

# jaunty is now on /dev/sda5
/dev/sda5  /  ext3  relatime,errors=remount-ro  0 1

If you want to do it the "right", ubuntu-approved way, with UUIDs, you can get the UUID of your disk this way:

ls -l /dev/disk/by-uuid/ | grep sda5

Take the UUID (that's the big long hex number with the dashes) and put it after the UUID= in the original fstab line.

While you're editing /etc/fstab, be sure to look for any lines that might mount /dev/sda5 as something other than root and delete them or comment them out.

The following section describes how to update grub1. Since most distros now use grub2, it is out of date. With any luck, you can run update-grub and it will notice your new partition; but you might want to fiddle with entries in /etc/grub.d to label it more clearly.

Now you should have a partition that you can boot into and upgrade. Now you just need to tell grub about it. As root, edit /boot/grub/menu.lst and find the line that's booting your current kernel. If you haven't changed the file yourself, that's probably right after a line that says:

## ## End Default Options ##

It will look something like this:

title           Ubuntu 8.10, kernel 2.6.27-11-generic
uuid            f7djaac8-fd44-672b-3432-5afd759bc561
kernel          /vmlinuz-2.6.27-11-generic root=UUID=f7djaac8-fd44-672b-3432-5afd759bc561 ro
initrd          /initrd.img-2.6.27-11-generic

Make a copy of this whole stanza, so you have two identical copies, and edit one of them. (If you edit the first of them, the new OS it will be the default when you boot; if you're not that confident, edit the second copy.) Change the two UUIDs to point to your new disk partition (the same UUID you just put into /etc/fstab) and change the Title to say 9.04 or Jaunty or My Copy or whatever you want the title to be (this is the title that shows up in the grub menu when you first boot the machine).

Now you should be able to boot into your new partition. Most things should basically work -- certainly enough to start a do-release-upgrade without risking your original install.

Tags: linux, ubuntu, install
[ 10:44 May 13, 2009    More linux/install | permalink to this entry | ]

Upgraded to Intrepid: X keyboard options and losing network after suspend

I finally got around to upgrading to the current Ubuntu, Intrepid Ibex. I know Intrepid has been out for months and Jaunty is just around the corner; but I was busy with the run-up to a couple of important conferences when Intrepid came out, and couldn't risk an upgrade. Better late than never, right?

The upgrade went smoothly, though with the usual amount of babysitting, watching messages scroll by for a couple of hours so that I could answer the questions that popped up every five or ten minutes. Question: Why, after all these years of software installs, hasn't anyone come up with a way to ask all the questions at the beginning, or at the end, so the user can go have dinner or watch a movie or sleep or do anything besides sit there for hours watching messages scroll by?

XKbOptions: getting Ctrl/Capslock back

The upgrade finished, I rebooted, everything seemed to work ... except my capslock key wasn't doing ctrl as it should. I checked /etc/X11/xorg.conf, where that's set ... and found the whole file commented out, preceded by the comment:

# commented out by update-manager, HAL is now used

Oh, great. And thanks for the tip on where to look to get my settings back. HAL, that really narrows it down.

Google led me to a forum thread on Intrepid xorg.conf - input section. The official recommendation is to run sudo dpkg-reconfigure console-setup ... but of course it doesn't allow for options like ctrl/capslock. (It does let you specify which key will act as the Compose key, which is thoughtful.)

Fortunately, the release notes give the crucial file name: /etc/default/console-setup. The XKBOPTIONS= line in that file is what I needed.

It also had the useful XKBOPTIONS="compose:menu" option left over from my dpkg-configure run. I hadn't known about that before; I'd been using xmodmap to set my multi key. So my XKBOPTIONS now looks like: "ctrl:nocaps,compose:menu".

Fixing the network after resume from suspend

Another problem I hit was suspending on my desktop machine. It still suspended, but after resuming, there was no network. The problem turned out to lie in /etc/acpi/suspend.d/55-down-interfaces.sh. It makes a list of interfaces which should be brought up and down like this:

IFDOWN_INTERFACES="`cat /etc/network/run/ifstate | sed 's/=.*//'`"
IFUP_INTERFACES="`cat /etc/network/run/ifstate`"

However, there is no file /etc/network/run/ifstate, so this always fails and so /etc/acpi/resume.d/62-ifup.sh fails to bring up the network.

Google to the rescue again. The bad thing about Ubuntu is that they change random stuff so things break from release to release. The good thing about Ubuntu is a zillion other people run it too, so whatever problem you find, someone has already written about. Turns out ifstate is actually in /var/run/network/ifstate now, so making that change in /etc/acpi/suspend.d/55-down-interfaces.sh fixes suspend/resume. It's bug 295544, fixed in Jaunty and nominated for Intrepid (I just learned about the "Nominate for release" button, which I'd completely missed in the past -- very useful!) Should be interesting to see if the fix gets pushed to Intrepid, since networking after resume is completely broken without it.

Otherwise, it was a very clean upgrade -- and now I can build the GIMP trunk again, which was really the point of the exercise.

Tags: ubuntu, linux, install, X11, networking
[ 18:28 Mar 11, 2009    More linux/install | permalink to this entry | ]

How often does updatedb really need to run?

Since updating to Hardy, I've been getting mail from Anacron:

/etc/cron.weekly/slocate:
slocate: fatal error: load_file: Could not open file: /etc/updatedb.conf: No such file or directory

That's the script that updates the database for locate, Linux's fast find system. I figured I must have screwed something up when I moved that slocate cron script from cron.daily to cron.weekly (because I hate having my machine slow to a crawl as soon as I boot it in the morning, and it doesn't bother me if the database doesn't necessarily have files added in the last day or two).

But after talking to some other folks and googling for Ubuntu bugs, I discovered I wasn't the only one getting that mail, and there was already a bug covering it. Comparing my setup with another Hardy user's, I found that the file slocate was failing to find, /etc/updatedb.conf, belongs to a different package, mlocate. If mlocate is installed, then slocate's cron script works; otherwise, it doesn't. Sounds like slocate should have a dependency that pulls in mlocate, no?

But wait, what do these two packages do? Let's try a little aptitude search locate:

p   dlocate                         - fast alternative to dpkg -L and dpkg -S   
p   kio-locate                      - kio-slave for the locate command          
i   locate                          - maintain and query an index of a directory
p   mlocate                         - quickly find files on the filesystem based
i   slocate                         - Secure replacement of findutil's locate   

Okay, forget the first two, but we have locate, mlocate, and slocate. How do they relate?

Worse, if I install mlocate (so slocate will work) and then look in my cron directories, it turns out I now have, count 'em, five different cron scripts that run updatedb. They are:

In cron.daily:

locate: 72 lines! but a lot of that is comments and pruning, and a lot of fiddling to figure out what version of the kernel is running to see whether it can pass any advanced flags when it tries to renice the process. In the end it calls updatedb.findutils (note no full path, though it uses a full path when it checks for it earlier in the script).

slocate: A much simpler but unfortunately buggy 20 lines. It checks for /etc/updatedb.conf, runs it if it exists, fiddles with ionice, checks again for /etc/updatedb.conf, and based on whether it finds it, runs either /usr/bin/slocate -u or /usr/bin/slocate -u -f proc. The latter path is what was failing and sending root mail every time the script was run.

mlocate: an even slimmer 12 line script, which checks for /usr/bin/updatedb.mlocate and, if it exists, fiddles ionice then runs /usr/bin/updatedb.mlocate.

In cron.weekly:

Two virtually identical scripts called find.notslocate and find.notslocate.dpkg-new, which differ only in dpkg-new having more elaborate ionice options. They both run updatedb. And which updatedb would that be? Probably /usr/bin/updatedb, which links to /etc/alternatives/updatedb, which probably links to either updatedb.mlocate or updatedb.slocate, whichever you've installed most recently. But in either case, it's hard to see why you'd need this script running weekly if you're already running both flavors of updatedb from other scripts cron.daily. And having two copies of the script is just plain wrong (and there was already a bug filed on it). (As long as you're poking around in cron.daily and cron.weekly, check and see if you have any more of these extra dpkg-new or dpkg-old scripts -- they might be slowing down your machine for no reason.)

Further research reveals that mlocate is a new(ish) package intended to replace slocate. (There was a long discussion of that on ubuntu-devel, leading to the replacement of slocate with mlocate very late in the Hardy development cycle. There was also lots of discussion of "tracker", apparently a GUI fast find tool that can only search in the user's home directory.)

What is this mlocate? The m stands for "merge": the advantage of mlocate is that it can merge new results into its existing database instead of replacing the whole thing every time. Sounds good, right? However, the down side is that mlocate apparently can't to purge its database of old files that no longer exist, and these files will clutter up your locate results. Running locate -e will keep them from being printed -- but there seems to be no way to set this permanently, via an environment variable or .locaterc file, nor to tell updatedb.mlocate to clean up its database. So you'll need to alias locate to locate -e if you want sensible behavior. Or go back to slocate. Sigh.

Cleaning up

The important thing is to get rid of most of those spurious updatedb cron scripts. You might choose to run updatedb daily, weekly, or only when you choose to run it; but you probably don't want five different scripts running two different versions of updatedb at different times. The packages obviously aren't cleaning up after themselves, so let's do a little manual cleanup.

That find.slocate script looks suspicious. In fact, if you run dpkg -S find.notslocate, you find out that it doesn't belong to any package -- not only should the .dpkg-old version not be there, neither should the other one! So out they go.

As for slocate and mlocate, it's important to know that the two packages can coexist: installing mlocate doesn't remove slocate or vice versa. A clean Hardy install should have only mlocate; upgrades from Gutsy are more likely to have a broken slocate.

Having both packages probably isn't what you want. So pick one, and remove or disable the other. If mlocate is what you want, apt-get purge slocate and just make sure that /etc/cron.*/slocate disappears. If you decide you want slocate, it's a little trickier since the slocate package is broken; but you can fix it by creating an empty /etc/updatedb.conf so updatedb.slocate won't fail.

Tags: linux, boot, ubuntu, install
[ 21:48 Apr 29, 2008    More linux/install | permalink to this entry | ]

Upgrading from Ubuntu Gutsy to Hardy

I finally had a moment to upgrade my desktop to Ubuntu's "Hardy Heron". I followed the same procedure as when I went from feisty to gutsy:

1. cp -ax / /hardy
2. cp -ax /dev/.static/dev/* /hardy/dev/
3. Fix up files like /hardy/etc/fstab and /boot/grub/menu.lst
4. Reboot into the newly copied gutsy
5. do-release-upgrade -d

It took an hour or two to pull down all the files, followed by a long interval of occasionally typing Y or N, and then I was ready to start cleaning up some of the packages I'd noticed flying by that I didn't want. Oops! I couldn't remove or install anything with apt-get, because: dpkg --configure -a
But I couldn't dpkg --configure -a because several packages were broken.

The first broken package was plucker, which apparently had failed to install any files. Its postinstall script was failing because it had no files to operate on; and then I couldn't do anything further with it because apt-get wouldn't do anything until I did a dpkg --reconfigure -a

I finally got out of that by dpkg -P plucker; then after several more dpkg --reconfigure -a rounds I was eventually able to apt-get install plucker (which installed just fine the second time).

But apt still wasn't happy, because it wanted to run the trigger for initramfs-tools, which wouldn't run because it wanted kernel modules for some specific kernel version in /lib/modules. I didn't have any kernel modules because I'm not running Ubuntu's kernel (I'm stuck on 2.6.23 because bug 10118 makes all 2.6.24 variants unable to sync with USB Palm devices). But I couldn't remove initramfs-tools because udev (along with a bunch of other less important packages) depends on it. I finally found my way out of that by removing /var/lib/dpkg/triggers/initramfs-tools. I reported it as bug 220094.

Update: I forgot to mention one important thing I hit both on this machine and earlier, on the laptop: /usr/bin/play (provided by the "sox" package) no longer works because it now depends on a zillion separate libraries. apt-get install libsox-fmt-all to get all of them.

Tags: linux, ubuntu, debian, install
[ 21:02 Apr 20, 2008    More linux/install | permalink to this entry | ]

Ubuntu "Hardy Heron"

On a lunchtime bird walk on Monday I saw one blue heron and at least five green herons (very unusuual to see so many of those). Maybe that helped prepare me for installing the latest Ubuntu beta, "Hardy Heron", Monday afternoon.

I was trying the beta primarily in the hope that it would fix a serious video out regression that appeared in Gutsy (the current Ubuntu) in January. My beloved old Vaio SR17 laptop can't switch video signals on the fly like some laptops can; I've always needed to boot it with an external monitor or projector connected. But as long as it saw a monitor at boot time, it would remember that state through many suspend cycles, so I could come out of suspend, plug in to a projector and be ready to go. But beginning some time in late January, somehow Gutsy started doing something that turned off the video signal when suspending. To talk to a projector, I could reboot with the projector connected (I hate making an audience watch that! and besides, it takes away the magic). I also discovered that switching to one of the alternate consoles, then back (ctl-alt-F2 ctl-alt-F7) got a signal going out on the video port -- but I found out the hard way, in front of an audience, that it was only a 640x480 signal, not the 1024x768 signal I expected. Not pretty! I could either go back to Feisty ... or try upgrading to Hardy.

I've already written about the handy debootstrap lightweight install process I used. (I did try the official Hardy "alternate installer" disk first, but after finishing package installation it got into a spin lock trying to configure kernel modules, so I had to pull the plug and try another approach.)

This left me with a system that was very minimal indeed, so I spent the next few hours installing packages, starting with tcsh, vim (Ubuntu's minimal install has something called vim, but it's not actually vim so you tend to get lots of errors about parsing your .vimrc until you install the real vim), acpi and acpi-support (for suspending), and the window system: xorg and friends. To get xorg, I started with:

apt-get install xserver-xorg-video-savage xbase-clients openbox xloadimage xterm

Then there was the usual exercise of aptitude search font and installing everything on that list that seemed relevant to European languages (I don't really need to scroll through dozens of Tamil, Thai, Devanagari and Bangla fonts every time I'm looking for a fancy cursive in GIMP).

But I hit a problem with that pretty early on: turns out most of the fonts I installed weren't actually showing up in xlsfonts, xfontsel, gtkfontsel, or, most important, the little xlib program I'm using for a talk I need to give in a couple weeks. I filed it as bug 212669, but kept working on it, and when a clever person on #ubuntu+1 ("redwhitewaldo") suggested I take a look at the x-ttcidfont-conf README, that gave me enough clue to get me the rest of the way. Turns out there's a Debian bug with the solution, and the workaround is easy until the Ubuntu folks pick up the update.

I hit a few other problems, like the PCMCIA/udev problem I've described elsewhere ... but mostly, my debootstrapped Hardy Heron is working quite well.

And in case you're wondering whether Hardy fixed the video signal problem, I'm happy to say it does. Video out is working just fine.

Tags: linux, ubuntu, install, fonts, vaio
[ 19:31 Apr 07, 2008    More linux/install | permalink to this entry | ]

Debootstrap

Some time ago, I wished for a simple Linux "Tarball installer", something that could install a minimal install of a Linux distribution onto an existing partition or directory, skipping all the flaky and error-prone hardware-guessing that installers do.

It turns out Debian (and therefore also Ubuntu) has had this for years, and it's totally cool. It's called debootstrap.

Some folks on the #ubuntu+1 channel told me about it, and I found a nice clear howto article on how to use it for Debian. It works just the same for Ubuntu.

First, get the .deb package for the debootstrap you want to use. Here's debootstrap for Ubuntu Hardy Heron. Install it with dpkg -i. Then run it, giving it the name of the system you want to install and the directory (or mounted partition) where you want to install it. Like this: debootstrap hardy /mnt/hda3

That's all! It fetches the files it needs from the online repositories. It takes no time at all -- this really is a minimal system.

Then you need to do some fiddling to turn it into a bootable system. That includes (all paths relative to the newly installed filesystem unless otherwise stated):

• Set up etc/fstab to list the fileystems on the disk, and to mount / from the filesystem you just installed
• Define the hostname in etc/hostname
• Set up a grub boot stanza in /boot/grub/menu.lst (that's /boot on the current system, which should be the same as /boot in the new fstab you just created). Use whatever kernel you were using for your old system, for now.

Now you're read to reboot into the new system. Of course, since this is a very minimal system, you have a lot more work to do. Hardly anything is installed, and nothing has been configured for you. Some things may be challenging (for example, as I write this, X is installed but most of the fonts aren't showing up properly, which may be a bug in Hardy).

Anyway, you can get a good start by mounting your old system's root directory and copying some starter files from there, starting with these:

• Set up your important configuration files: /etc/network/interfaces, /etc/hosts, /etc/resolv.conf, /etc/passwd etc.
• edit /etc/apt/sources.list to include restricted universe multiverse
• Install a kernel package if you're using distro kernels
• Install vim if you're a vim user -- remember, ubuntu comes with something called vim that isn't really vim.
• Create users and homedirs and such
• Install all the other stuff you want -- X, gimp/gtk, development tools, editors, shells -- all that stuff that makes the system feel like home. You're on your own there, so have fun!

Tags: linux, debian, install
[ 13:54 Apr 06, 2008    More linux/install | permalink to this entry | ]