Shallow Thoughts : tags : virtualization

Running Windows 10 under QEMU, Update: Debian Changed OVMF

In October I wrote about making a Windows 10 that Boots off a USB Stick, From Linux.

A Debian update today or yesterday (Merry Christmas!) broke that and I spent a few hours today chasing that down.

There's a package called ovmf that puts BIOS/firmware related files in /usr/share/OVMF/. The command I used in the earlier article included the flag -bios /usr/share/OVMF/OVMF_CODE.fd but as of today, -bios apparently doesn't work any more with any of the files there.

Read more ...

QEMU Windows Guest: Easy File Sharing with QEMU's built-in SMB

Last year I wrote about my efforts to share files in QEMU between a Linux host and Windows guest. Someone in the comments pointed me to WinFSP and WinSSHFS, and I was able to get file sharing working that way, after installing both those packages on the Windows guest.

But I recently found a much easier way, using QEMU's built-in SMB handler.

Read more ...

Create a Windows 10 that Boots off a USB Stick, from Linux

In 2019, I wrote about struggling to get any sort of Windows booting off an external USB stick, in order to Install Lenovo Firmware Packaged as a .exe on a Linux Machine. I ended up needing to borrow a real Windows machine and install Rufus on it.

In 2023, things are much better. Aki at atkdinosaurus has written a clear, concise tutorial on that topic: How to create a Windows 10 installation on a USB stick in UEFI mode. I love that it's all command-line, so you can duplicate the steps exactly.

Read more ...

Shared Folder in QEMU Between Linux Host and Windows Guest

Update: I have found a much easier way, using QEMU's built-in Samba. See QEMU Windows Guest: Sharing Files with the Host.

Unexpectedly, one of the hardest parts of Migrating a VirtualBox Windows Virtual Machine to qemu/kvm/virt-manager was finding a way to exchange files between Linux and Windows. In virtualbox, setting up a shared folder is trivial. In QEMU, not so much.

Read more ...

Virt-Manager Beginner Tips

A couple of small tips on QEMU/KVM/VirtManager that I picked up while migrating my Windows 10 virtual machine from VirtualBox, for use once you get virt-manager running and migrate your VirtualBox VMs to virt-manager/QEMU:

Read more ...

Migrating a VirtualBox Windows Virtual Machine to QEMU/KVM/virt-manager

A month ago I wrote about Getting virt-manager Running on Debian. The ultimate goal of this was to migrate my Windows 10 install from VirtualBox to QEMU, because VirtualBox is becoming increasingly difficult to install on Linux, especially on Debian, which has removed VirtualBox from Bookworm (testing) and there are indications that it might be removed from Sid (unstable) as well. I gather there's something unsavory about the license now that Oracle owns it, but I haven't been following the details.

Anyway, after getting virt-manager running, I'd been putting off the rest of the migration out of a suspicion that there lay dragons. I was right: it took several days of struggling, but I now have Windows 10 working under virt-manager and qemu/kvm. Here's how.

Read more ...

Getting virt-manager Running on Debian

A conversation that happens every so often on a Linux chat channel:

newbie:	Which is easier for virtualization, KVM/qemu or virtualbox?
enthusiast:	KVM/qemu is really easy if you use virt-manager.
me:	If you're running a full Gnome desktop, maybe. I've tried to use virt-manager several times, with virt-manager enthusiasts on this channel helping, and never got it going.

That happened again a few weeks ago, and one of the virt-manager enthusiasts on the channel wanted to help me track down the problems. Since I didn't have anything much going on, I agreed, and kept at it instead of giving up after the first few iterations.

It took about 45 minutes of fiddling, installing more packages, web searching on the error messages and discussing them on IRC, then fiddling some more, getting a little further with each package I installed. In the end, I did get virt-manager running.

Here's the list of packages I had to install, as well as adding myself to the groups kvm, libvirt and libvirt-qemu:

apt install virt-manager libvirt-daemon qemu qemu-kvm libvirt0 libvirt-bin virt-manager bridge-utils libvirt-daemon-system qemu-system-x86 qemu-utils dnsmasq gir1.2-spiceclientgtk-3.0

Part of the problem, apparently, is that Debian's virt-manager package isn't set up to require on all the other packages it needs to run. They might be there in the "recommends" and "suggests", but I don't install those by default, since they tend to pull in all sorts of silly bloatware I'll never want. With most packages, the "depends" are all that's needed to use a package in its basic form, and "recommends" and "suggests" are for optional extra features. But if you install just virt-manager without at least some of the suggested and/or recommended packages, it won't run at all.

I haven't run any real virtual machines yet under virt-manager, but I think it's working now. At least I'm to the point where I can boot from a Debian installer ISO and see the initial screen.

Emulating Raspbian on your Linux x86/amd64 System

I was planning to teach a class on Raspberry Pis, and I wanted to start with the standard Raspbian image, update it, and add some programs like GIMP that we'd use in the class. And I wanted to do that just once, before burning the image to a bunch of different SD cards. Is there a way to do that on my regular Linux box, with its nice fast processor and disk?

Why yes, there is, and it's pretty easy. But there are a lot of unclear or misleading tutorials out there, so I hope this is a bit simpler and easier to follow.

I got most of this from a tutorial that no longer seems to be available (but I'll include the link in case it comes back): Solar Staker/RPI Image/Creation.

I've tested this on Ubuntu 19.10, Debian Stretch and Buster; the instructions should be pretty general except for the name of the loopback mount. Commands you type are in bold; the rest is the output you should see. $ is your normal shell prompt and # is a root prompt.

Required Packages

You'll need kpartx, qemu and some supporting packages. On Debian or Ubuntu:

$ sudo apt install kpartx qemu binfmt-support qemu-user-static

You've probably already downloaded a Raspbian SD card image.

Set Up the Loopback Devices

kpartx can read the Raspbian ISO image and split it into the two filesystems it would have if you wrote it to an SD card. It turns the filesystems into loopback devices you can mount like regular filesystems.

$ sudo kpartx -av 2019-09-26-raspbian-buster-lite.img
add map loop10p1 (253:1): 0 524288 linear 7:10 8192
add map loop10p2 (253:2): 0 3858432 linear 7:10 532480

Make a note of those loopback device names. They may not always be loop10p1 and loop10p2, but they'll probably always end in 1 and 2. 1 is the Raspbian /boot filesystem, 2 is the Raspbian root.

(Optional) Check and Possibly Resize the Raspbian Filesystem

Make sure that the Raspbian filesystem is intact, and that it's a reasonable size.

In practice, I didn't find this made any difference (everything was fine to begin with), but it doesn't hurt to make sure.

$ sudo e2fsck -f /dev/mapper/loop10p2
e2fsck 1.45.3 (14-Jul-2019)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
rootfs: 44367/120720 files (0.3$ non-contiguous), 292014/482304 blocks

$ sudo resize2fs /dev/mapper/loop10p2
resize2fs 1.45.3 (14-Jul-2019)
The filesystem is already 482304 (4k) blocks long.  Nothing to do!

Mount the Loopback Filesystems

You're ready to mount the two filesystems. A Raspbian SD card image contains two filesystems.

Partition 1 is a small vfat /boot filesystem containing the kernel and some other files it needs for booting, plus the two important configuration files cmdline.txt and config.txt.

Partition 2 is the Raspbian root filesystem in ext4 format. The Raspbian root includes an empty /boot directory; mount the root first, then mount the Raspbian boot partition on Raspbian's /boot:

$ sudo mkdir /mnt/pi_image
$ sudo mount /dev/mapper/loop10p2 /mnt/pi_image
$ sudo mount /dev/mapper/loop10p1 /mnt/pi_image/boot

Prepare for Chroot

You're going to chroot to the Raspbian filesystem. Chroot limits the filesystem you can access, so when you type /, instead of your host filesystem's / you'll see the root of the Raspbian filesystem, /mnt/pi_image. That means you won't have access to your host system's /usr/bin, any more.

But qemu needs /usr/bin/qemu-arm-static to be able to emulate ARM binaries in user mode. So copy that to the Raspbian filesystem so you'll still be able to access it after the chroot:

$ sudo cp /usr/bin/qemu-arm-static /mnt/pi_image/usr/bin

Chroot to the Raspbian System

$ sudo chroot /mnt/pi_image /bin/bash
root:/#

Now you're running in Raspbian's root filesystem. All the binaries in your path (e.g. /bin/ls, /bin/bash) are ARM binaries, but if you try to run them, qemu will see qemu-arm-static and run the program as though you're on an actual Raspberry Pi.

Run Stuff!

Now you can run Raspbian commands.

# file /bin/ls
/bin/ls: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 3.2.0, BuildID[sha1]=67a394390830ea3ab4e83b5811c66fea9784ee69, stripped
#
# /bin/ls
bin   dev  home  lost+found  mnt  proc	run   srv  tmp	var
boot  etc  lib	 media	     opt  root	sbin  sys  usr

# file /bin/cat
/bin/cat: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 3.2.0, BuildID[sha1]=2239a192f2f277bd1a4892e39a41eba97266b91f, stripped
#
# cat /etc/issue
Raspbian GNU/Linux 10 \n \l

You can even install packages or update the whole system:

# apt update
Get:1 http://raspbian.raspberrypi.org/raspbian buster InRelease [15.0 kB]
Get:2 http://archive.raspberrypi.org/debian buster InRelease [25.2 kB]
Get:3 http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages [13.0 MB]
Get:4 http://archive.raspberrypi.org/debian buster/main armhf Packages [259 kB]
Fetched 13.3 MB in 20s (652 kB/s)
Reading package lists... Done

# apt dist-upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following NEW packages will be installed:
  busybox initramfs-tools initramfs-tools-core klibc-utils libklibc linux-base
  pigz
The following packages will be upgraded:
  dhcpcd5 e2fsprogs file firmware-atheros firmware-brcm80211 firmware-libertas
  firmware-misc-nonfree firmware-realtek libcom-err2 libext2fs2 libmagic-mgc
  libmagic1 libraspberrypi-bin libraspberrypi-dev libraspberrypi-doc
  libraspberrypi0 libss2 libssl1.1 libxml2 libxmuu1 openssh-client
  openssh-server openssh-sftp-server openssl raspberrypi-bootloader
  raspberrypi-kernel raspi-config rpi-eeprom rpi-eeprom-images ssh sudo
  wpasupplicant
32 upgraded, 7 newly installed, 0 to remove and 0 not upgraded.
Need to get 133 MB of archives.
After this operation, 3192 kB of additional disk space will be used.
Do you want to continue? [Y/n]

Pretty neat! Although you're not actually running Raspbian, you can run Raspbian executables with the Raspbian root filesystem mounted as though you were actually running on your Raspberry Pi.

Cleaning Up

When you're done with the chroot, just exit that shell (Ctrl-D or exit). If you want to undo everything else afterward:

$ sudo rm /mnt/pi_image/usr/bin/qemu-arm-static

$ sudo umount /mnt/pi_image/boot
$ sudo umount /mnt/pi_image

$ sudo kpartx -dv /dev/loop0
$ sudo losetup -d /dev/loop0

$ sudo rmdir /mnt/pi_image

Limitations

Keep in mind you're not really running Raspbian. You never booted the Raspbian kernel, and you can't test things that depend on Raspbian's init system, like whether networking works, let alone running the Raspbian X desktop or accessing GPIO pins. This is an ARM emulator, not a Raspberry Pi emulator.

More details

If you want to read more about qemu user mode and how it lets you run binaries from other architectures, I recommend these links:

• Transparently running binaries from any architecture in Linux with QEMU and binfmt_misc
• Running ARM programs under linux (without starting QEMU VM!)