Building and installing a new kernel for a SheevaPlug

I have several hardware-control projects that need a little more horsepower than an Arduino; I was hoping the Sheevaplug might be the answer. The specs on these babies say they have GPIO lines, though there isn't much support or documentation for using them.

But one thing that's clear: to use them, you need to compile your own kernel. None of them come with a kernel that enables /sys/class/gpio, i2c or any of the other methods you might need to read and set bits. So the first step in any plug GPIO adventure involves building a kernel and installing it.

No problem, right? I've built a kazillion Linux kernels on x86 machines, and I've cross-compiled for the Arduino and other platforms. How hard can it be to build an arm kernel and install it on a plug?

As it turns out, quite hard. Building it is one thing; getting a bootable kernel installed onto the plug's flash is quite another. There are dozens of howtos scattered all over the net, each of them saying "Well, the standard instructions don't work, so here's how to really do it", and of course they're all different. And none of them worked for me. So here's Yet Another Tutorial on Building and installing a new kernel for a SheevaPlug. I'll be cross-compiling on a Linux box. You can also build a kernel directly on the plug (apparently it takes about an hour), but that would be a different tutorial.

Preliminaries: Unbricking

Flashing a kernel that doesn't boot will leave your plug "bricked", unusable. Therefore, before you even try building your own kernel, you should make sure you can un-brick it.

This is harder than it sounds. In theory you can get ready-to-go installs of Debian, Ubuntu and Android, and you can use scripts like ESIA, the SheevaPlug Installer or the GuruPlug Installer to flash them to the plug.

In practice, none of those scripts worked for me, every plug is different, and some, like GuruPlugs, may be hopeless. I still haven't managed to unbrick my GuruPlug, let alone install a homebuilt kernel on it.

So find a setup that will unbrick your plug before you bother building a new kernel for it. Be prepared to spend a while downloading several different setups and trying different methods. And cross your fingers. If you find, like me with the GuruPlug, that you have a plug that you just can't unbrick, you can tell yourself that it probably would have happened eventually anyway.

If one of the above scripts work for you, rejoice! and skip ahead to Building a New Kernel.

Otherwise, don't give up. None of them worked for me either. Go download a standard Debian kernel, initrd and rootfs and proceed to the Install from USB on a Bricked Plug section at the end of this article.

Building a New Kernel

Once you're sure you can unbrick your plug, you're ready to build a kernel.

Download the kernel

You can use a mainline Linux kernel -- whatever the current version is at Kernel.org. Or you can use Marvell's "Orion" kernel. I used a mainline kernel, 2.6.35.2. Download and unpack it.

Get a cross-compiler

First you need a cross compiler. There are at least three around, none of them available in distro repositories. I used the one from CodeSourcery.

By default, CodeSourcery installs to a directory under your homedir, ~/CodeSourcery. You will need to specify this location when you build the kernel. Using ~ doesn't work -- some of the kernel build scripts break in confusing ways -- so either move the directory somewhere close to where you're building your kernel (e.g. ../CodeSourcery) or reference it by full pathname, /home/yourname/CodeSourcery.

Reference it how? This is something on which most howtos are rather vague. You need a CROSS_COMPILE= argument that will be prepended to the name of the various compiler binaries. In other words, if you've installed to ../CodeSourcery, use CROSS_COMPILE=../CodeSourcery/Sourcery_G++_Lite/bin/arm-none-eabi- and when make looks for gcc, it will look for ../CodeSourcery/Sourcery_G++_Lite/bin/arm-none-eabi-gcc.

You'll also need another binary: UBoot's mkimage program. On Ubuntu you can get this with apt-get install uboot-mkimage ... or you can try building mkimage from source (I haven't tried that myself).

Configure the kernel and enable ubi

make ARCH=arm kirkwood_defconfig

This creates a default .config file with the architecture, etc. set up correctly. Then, if you need to change anything -- and presumably you do, since otherwise why are you putting yourself through this? -- run

make ARCH=arm menuconfig

Do not forget the ARCH=arm here! If you build a lot of kernels it's easy to forget and just type make menuconfig. If you do that, menuconfig will trash all that nice kirkwood configuration and default back to whatever architecture you're using to build (probably x86 and you'll end up with a kernel that doesn't boot.

The root filesystem on the Ionics plug I'm using for this is Ubifs. Ubifs is not enabled in the default kirkwood install, so you need it. First enable UBI, under drivers->MTD, then enable UBIFS, under Filesystems->Misc filesytems

Build the kernel

make -j4 ARCH=arm CROSS_COMPILE=../CodeSourcery/Sourcery_G++_Lite/bin/arm-none-eabi- uImage
make -j4 ARCH=arm CROSS_COMPILE=../CodeSourcery/Sourcery_G++_Lite/bin/arm-none-eabi- modules
make -j4 ARCH=arm CROSS_COMPILE=../CodeSourcery/Sourcery_G++_Lite/bin/arm-none-eabi- INSTALL_MOD_PATH=modules_install modules_install
cd modules_install; tar czvf ../modules.tar.gz . ; cd ..

Assuming this all worked, you have a kernel! If your plug is up, running and unbricked, installing it may be fairly easy.

Install on a working plug

In theory, if you copy (using the network or USB) your kernel and modules over to a running plug, you should be able to flash it like this:

nandwrite -p /dev/mtd0 /boot/uImage

(This should perhaps be /dev/mtd1 on some systems. Use cat /proc/mtd to see which mtd device has what.)

In practice, that won't work and your plug won't boot. Go figure. But at least you can unpack the modules:

cd /
tar xvf /wherever/modules.tar.gz

Now go ahead and try that kernel you just flashed. When your plug refuses to boot with "Bad magic number", proceed to the next section.

But wait! Since I wrote that, there are some new instructions: lots of good info in this thread on Installing sheeva-with-linux kernels in Debian.

By the way, Sheeva Uboot Tools looks really useful for finding out about your uboot environment from within a running plug.

Install or unbrick from uBoot using a USB stick

This procedure assumes you have a serial/JTAG connection to the plug, so you can talk to uBoot.

Before proceeding, power the plug, connect to it and hit a key in time to stop in the uboot prompt. Now type:

printenv

and copy everything it just printed to a file on your local machine. You may want this later.

Check whether mainlineLinux is set to yes. If it's not, you'll probably want

setenv mainlineLinux yes

for anything except the original kernel that came with the plug. You should only have to set that once.

Now copy the kernel and modules to a fat- or vfat-formatted USB stick:

cp arch/arm/boot/uImage modules.tar.gz /stick

You may need an initrd as well.

Plug the USB stick into the plug.

Now comes the tricky part, because it varies between models. On some plugs, like the Ionics PlugComputer Plus, you can type:

run recover2

and it will find the USB stick, load the kernel and initrd into flash, reboot from them, then reload the root filesystem. You can hit Enter to stop the reloading of the root filesystem, if all you need is the kernel.

Great -- What if that doesn't work? Or if you don't have recover2 defined at all?

Unfortunately, every plug seems to have a different set of boot commands. Go look at that printenv output you saved earlier. Anything that starts with recover or boot is potentially relevant. Here's what those look like on the Ionics Plus:

bootargs_root=ubi.mtd=1 root=ubi0:rootfs rootfstype=ubifs
mtdpartitions=mtdparts=orion_nand:0x400000@0x100000(uImage),0x1fb00000@0x500000(rootfs)
real_bootcmd=setenv bootargs $(bootargs_console) $(mtdpartitions) $(bootargs_root); nand read.e 0x00800000 0x00100000 0x00400000; bootm 0x00800000
bootargs_console=console=ttyS0,115200
recover1=setenv mainlineLinux yes; setenv arcNumber 2097; setenv bootcmd run recover2; saveenv; reset
recover2=run recover3; setenv bootcmd $(real_bootcmd); saveenv; setenv bootargs $(bootargs_console) $(mtdpartitions) root=/dev/ram0 rw ramdisk=0x01100000,8M install_type=nand; bootm 0x00800000 0x01100000
recover3=run recover4; nand erase clean 0x00100000 0x00400000; nand write.e 0x00800000 0x00100000 0x00400000
recover4=usb start; fatload usb 0 0x00800000 uImage; fatload usb 0 0x01100000 initrd
arcNumber=2097
run_diag=no
bootargs=console=ttyS0,115200 mtdparts=orion_nand:0x400000@0x100000(uImage),0x1fb00000@0x500000(rootfs) ubi.mtd=1 root=ubi0:rootfs rootfstype=ubifs
ethaddr=00:26:DB:00:03:EA
filesize=1CAE6D
bootcmd=setenv bootargs $(bootargs_console) $(mtdpartitions) $(bootargs_root); nand read.e 0x00800000 0x00100000 0x00400000; bootm 0x00800000

Note that when you run recover2 on this plug, what you're really doing is running recover3 first. recover3 runs recover4, which loads the kernel and initrd from USB. Then recover3 writes those values to flash and passes control back to recover2, which sets up the boot command and actually boots. Whew!

If you get rid of all those separate variables, here's what the Ionics is doing, and something very like this sequence may work for you:

(recover4)
  usb start
  fatload usb 0 0x00800000 uImage
  fatload usb 0 0x01100000 initrd
(recover3)
  nand erase clean 0x00100000 0x00400000
  nand write.e 0x00800000 0x00100000 0x00400000
(recover2)
  setenv bootcmd $(real_bootcmd)
  saveenv
  setenv bootargs $(bootargs_console) $(mtdpartitions) root=/dev/ram0 rw ramdisk=0x01100000,8M install_type=nand
  bootm 0x00800000 0x01100000

Note that real_bootcmd is setenv bootargs $(bootargs_console) $(mtdpartitions) $(bootargs_root); nand read.e 0x00800000 0x00100000 0x00400000; bootm 0x00800000

Some likely tripping points here:

In recover2, the kernel boots with root=/dev/ram0 ramdisk=0x01100000,8M. But bootargs_root has ubi.mtd=1 root=ubi0:rootfs rootfstype=ubifs. This is because recover2 is trying to do a one-time boot from to the initrd it's just read in from USB (I think), while on reboot you'll want the kernel to find the ubifs root filesystem. When you're trying to boot your own kernel, without an initrd, you'll probably want the ubi option, not the ramdisk option.

So when you're booting your real kernel, not just unbricking, you might want to skip run recover2 and instead do something like:

run recover3
setenv bootcmd $(real_bootcmd)
saveenv
bootm 0x00800000 0x01100000

The difference is you're skipping the setenv bootargs step which would set root=/dev/ram0.

But wait! If your kernel doesn't need an initrd, you can skip that part too. So this is really all you need, assuming your bootcmd is already set right:

usb start
fatload usb 0 0x00800000 uImage
nand erase clean 0x00100000 0x00400000
nand write.e 0x00800000 0x00100000 0x00400000
bootm 0x00800000 0x01100000

I suspect that nand write.e could be shortened, but I haven't found documentation on what the numbers mean yet.

... or use tftp from uBoot

My guruplug seems to have broken USB -- it won't recognize a USB stick. You can tell by running usb start and checking whether it sees a USB storage device.

In the absence of USB, uBoot can get a kernel using tftp. Set up tftp on your server (hint: on Ubuntu, the tftp package has broken dependencies; use apt-get install xinetd tftpd then follow these tftp instructions; or tftpd-hpa or atftpd might work).

Now it gets a little confusing: I got these instructions from somewhere but I'm not convinced they're correct. The addresses don't seem to agree with the addresses used by recover4 in the previous section. but the uBoot doc are a bit sparse ( Uboot at DULG, U-Boot Command Line Interface and a PDF U-Boot Reference Manual).

Anyway, at the uBoot prompt, type something like:

setenv ipaddr 192.168.1.something
setenv serverip 192.168.1.somethingelse
tftp 0x6400000 uImage
nand erase 0x0 0x100000
nand write.e 0x6400000 0x0 0x100000
reset

It might also help, instead of reset, to try bootm 0x6400000.

Someone else reports luck with the simpler:

tftpboot 0x100000 uImage
bootm 0x100000

Any of these work? Woohoo! You're done!

If you have your new kernel booting, rejoice! You can worry about what to do about all those proprietary drivers, like the network and wi-fi chips that aren't in the Debian kernel, later.

But my wi-fi doesn't work any more!

Right. There's some trick to getting the "libertas" driver working. I don't know what the trick is. Please tell me if you know!

Wait, what about those GPIO lines?

Ha! Are you kidding?

I have no idea. At this point, I'm happy to have a kernel that boots at all. I think I'll stick to Arduinos for controlling hardware lines, and use plug computers only for higher-level tasks.