My new Lenovo Carbon X1 Gen 7 has one irritating problem: the trackpad
sometimes disappears, flooding dmesg with messages like
"i2c_designware i2c_designware.1: controller timed out".
Once this happens, the only fix is to reboot.
Lenovo has a fix -- new trackpad firmware -- but unlike their BIOS
updates, which are installable from Linux, device firmware updates
are distributed as Windows EXE files that require running Windows
on the bare metal, leaving Linux users out in the cold.
Ironic, since Lenovo is so popular among Linux users and is a member
of
the Linux
Firmware Service, and the CX1 is supposedly Ubuntu certified.
Those Linux users on the forums who managed to install the firmware
update raved about it, saying that indeed it solved their problem. But
finding a way to to install it led me on a not-so-merry four-day quest.
Here's how I installed the firmware, in the end:
Make a Windows to Go using Rufus on a Real Windows Box
Back up anything you don't want to lose, because you never know.
Borrow a real Windows box. I tried many times
using Windows inside VirtualBox and QEmu on top of Linux,
but it never worked.
Find a USB stick or SD card, 16G or larger. Actually, find a bunch
of them: this process is incredibly finicky about the stick you use
and the only way you find out is that it doesn't work and you
have to try again (see below).
Use Rufus to create a Windows to Go image. The alternative
is to make a Windows installer; that won't work, because you can't
run anything useful from the installer, and you don't want to
actually install Windows, or you wouldn't be in this fix
in the first place.
Be patient: creating a W2G image takes several hours.
Click on Rufus' log file button (it's the rightmost of
four obscure icons down near the lower left of the Rufus window;
it has a mouseover tooltip) at any time to see what's happening;
if things don't go right you might at least get some idea why.
When the W2G stick is finished (whew!), move it to your Linux machine
and mount its second partition (/dev/sdb2 or whatever).
This will tell you it wasn't properly unmounted and it's fixing it,
giving you a heart attack about whether Linux is going to change the
filesystem in some way that makes it fail after you waited all that
time creating it.
Copy the firmware .exe to it. Wherever you want; I just put it at
the root of the filesystem. Sync and unmount it.
Boot your computer from the USB stick. This will
take forever and may fail if the phase of the moon is wrong.
If you're lucky and the planets are in alignment, eventually a
Windows installer will come up and ask you a bunch of annoying questions
about language, keyboard, whether you consent to having Microsoft
spy on you in a skillion different ways, name, password, three
security questions, etc. Meanwhile, you're having another heart
attack because does this mean it's going to install Windows to
your real disk on top of Linux? Hopefully not -- at least it
didn't in my case -- but here's where you really want to have that
recent backup.
If you make it all the way through the questions and get a Windows
screen, rejoice! Navigate to wherever you put your exe and run it.
Cross your fingers -- maybe you're done!
If it hangs or bluescreens during boot, or Rufus fails to create
the W2G stick in the first place, try running Rufus again with a
different USB stick.
I think I tried five before finally finding one that worked,
and the successful one (a Transcend SD card in an old Patriot USB
adapter) wasn't the newest, or the fastest, or the largest.
It's a mystery.
Some Approaches that Didn't Work
Before I finally got this working, I wasted four days trying many
other approaches. Many of them sound very clever and reasonable and
ought to work, but they didn't work for me. These include:
Install Windows
PE or the full Windows 10 installer directly onto a USB stick
using qemu using a command like this:
That brought up a window in which the Windows installer booted and
proceeded to install using the USB stick mounted at /dev/sdX as the disk.
At least in theory.
I tried this several times first without the -nic none,
and every time, the installer eventually hit an infinite loop of
"Why did my PC restart?" bluescreens.
Adding the -nic none cured that problem, but I ended up
with a USB stick that wasn't bootable.
This was my favorite method in theory because it's clever and I
understand at least a little what it was supposedly doing. I wish it
had actually worked. Possibly if I knew enough about Windows and UEFI
boot architecture, it might have been possible to patch the stick to
make it bootable.
Various other attempts to make a bootable WinPE -- I liked
the minimal WinPE idea so I kept trying it in different ways, but
I've lost track of all the different methods.
Use vboxmanage to clone the Win10 I already had in Virtualbox
to a USB stick, as suggested
here.
Use FreeDOS. "This program must be run under Win32."
Use Windows 10 inside Virtualbox to run the initial exe to
extract the installer inside, another exe, then run that in
FreeDOS. "This program cannot be run in DOS mode."
Boot from a Win10 installer, use Shift-F10 to get a command prompt
window, and run the exe from that. "The image file is valid, but is
for a machine type other than the current machine."
Use Windows 10 running inside Virtualbox to run Rufus to make a
Windows to Go stick. I had no trouble making a regular USB installer
using Rufus in virtual Windows, but when I tried Win to Go, it failed
every time. Sometimes it failed because it couldn't read the USB stick
properly (even though I had configured Virtualbox to pass it through);
other times it failed because it was confused by the Win10 installer
ISO, which was on a Virtualbox shared drive (because I didn't have
that much spare space in the virtual Windows instance). Apparently
there's something not entirely reliable about Virtualbox's USB handling
and its shared folders, at least for very data transfers.
Use WinToUSB or some such program. There seem to be lots of
different programs with names similar to this, they're all closed
source (Rufus is open source), it's hard to tell which ones are real
and which are malware, and when I did think I'd finally found one from
a reliable source, it just hung when I tried to run it.
Use a backup made with Macrium Reflect on a real Windows box,
and boot from that. "The version of [exename] is not compatible with
the version of Windows you're running. Check your computer's system
information and then contact the software publisher."
So, lots of different ways. Some of them have worked at some time for
someone. Also, I never did try Wine. I don't think Wine would be able
to run the actual exe and update the trackpad firmware (I was afraid
to try it), but it's possible that Rufus in Wine might have been able
to make a Windows To Go stick.
If anyone manages that -- or any other way of getting this to work --
I'd love to hear about it.
I have a new laptop, a birthday present to myself last month.
For once, rather than buying a cut-rate netbook, I decided to
treat myself to a fancy Lenovo Carbon X1 with an up-to-date processor
and lots of RAM.
Since I have way more resources than I'm used to, I decided I'd try
installing a full Ubuntu and not trying to pare it down to a super
lightweight system. I'm still running the lightweight, fast, highly
configurable Openbox window manager instead of a full Gnome desktop:
Openbox does just what I tell it and no more, and doesn't surprise me
with random redesigns. But I did let Ubuntu install some system
utilities I've always avoided in the past, like NetworkManager and
PulseAudio. I decided I'd give them a chance, see if they've gotten
better since I last checked.
They have, though they're still a bit of a hassle to deal with.
NetworkManager can be controlled through nmcli, which is poorly
documented but works okay if you google long enough to find the
proper incancations. PulseAudio gave me a bit more trouble.
The standard GUI for controlling PulseAudio is pavucontrol.
It showed two audio devices: "USB PnP Audio Device Analog Stereo" and
"Built-in Audio Analog Stereo". Turns out the USB PnP option is a
sound card built into the USB hub, a Totu tt-hb003a 11-in-1 USB-C hub
that lets me connect to a charger, external monitor, SD and micro-SD
slots, and extra USB ports without juggling a lot of extra cables.
Pulse assumes -- probably reasonably, though it's wrong in this case
-- that if I have a USB audio device connected, I probably want to use
it in preference to the laptop's built-in audio. That would make sense
if I had external speakers plugged in, but I left all my computer
speakers behind when I moved. I should probably order some speakers.
But meanwhile, I needed to persuade PulseAudio to ignore the hub and
use the laptop's built-in sound system.
Mute/Unmute via the Keyboard
The Lenovo, like most laptops, has a dedicated key for muting, Fn-F1.
It even has a little light on it to show whether it's muted. In
Openbox, pressing Fn-F1 actually muted the sound, and even turned on
the light. This is probably because I'd previously set
key="XF86AudioMute" to run amixer set Master toggle
in .config/openbox/rc.xml, which worked on my Pulse-free
pared-down Debian netbook. The problem is that pressing iFn-F1 again
didn't bring the sound back. Instead, it was unmuting the USB hub's audio.
Clicking "Set as fallback" on the built-in audio in pavucontrol made
no difference.
It turns out that it is virtually impossible to persuade PulseAudio to
use "Built-in Audio" when a "USB PnP Audio Device" is available. I
finally found the secret: in pavucontrol's Configuration tab,
set Profile for the PnP USB device to Off. Now only the
built-in device shows up in the other tabs.
But that amixer command still wasn't unmuting properly, so the next
step was to find a command that would actually unmute. Someone on
#linux suggested pactl set-sink-mute @DEFAULT_SINK@ toggle
and that worked great from the command line. But when I tried to bind it
in Openbox to the XF86AudioMute key, it did nothing. I still
don't understand why not; I wasted a lot of time comparing my
shell environment to openbox's environment and never found the
difference.
Back to web searching, I found an
askbuntu thread suggesting
some Openbox stanzas. In particular, it apparently works better to use
alsamixer rather than pactl. This finally worked for toggling mute:
Partial success! Unfortunately, the volume control commands in that
same askbuntu post, amixer -q -D pulse sset Master 3%+ unmute,
did nothing. I had already noticed that in pavucontrol, the volume
controls didn't work either. In fact, if I started some music playing
and then called up alsamixer, channels like Master and Speaker didn't
do anything; the only channel that affected volume was ALSA PCM. After some
fiddling, I discovered that I had to change Master to PCM and
remove the -D pulse:
I'm sure I'll eventually need to fiddle some more. For one thing,
if I ever want to use audio during a talk (as I did briefly at my
Stonehenge talk earlier this year) I'll need to figure out how to
enable a temporary HDMI sound sink quickly without needing to fiddle
with pavucontrol. But for now, I'm happy to have
the basic laptop volume and mute keys working.
Sometimes I tend to ramble on, and wonder if articles I'm writing are
really too long for a blog post. I try to keep them under about 200
lines, but sometimes a really meaty topic demands more.
It occurred to me to wonder how long a typical Shallow Thoughts post is.
A quick measure is lines, which I can measure this way starting
in the directory where I have the source files for all my past posts:
so if I sort -h (human-readable numbers), it will sort on the first
column and give me a sorted list of all posts in order of size.
The shortest posts, three of them, were only five lines;
the longest was 346 lines.
But what's the distribution of lengths?
I can plot the sorted data easily with gnuplot:
gnuplot -p -e 'plot "/tmp/bloglen.dat"'
or, if I didn't want the temp file, I could have done that all
with one command:
That's kind of interesting. But I was really more interested in seeing
a frequency distribution: do I have a lot more shorter posts, or
longer ones? For that I do need the temp file.
I wasted some time trying to find a way in gnuplot to plot frequency
distribution. The best I found was
set style fill solid
plot '/tmp/bloglen' u ($1):(1) t 'data' smooth frequency w boxes
pause mouse close
(put that in a file and then run gnuplot on that file).
But it's not actually right: the bargraph shows 1 for lots of blog
post lengths that aren't represented in the data.
I finally gave up on gnuplot, having wasted enough time that I could
easily have written a Python script, and did so, which only took a
few minutes.
import matplotlib.pyplot as plt
posts = []
with open('/tmp/bloglen') as fp:
for line in fp:
posts.append(int(line.split()[0]))
plt.hist(posts, bins=max(posts))
plt.show()
Turns out I'm doing pretty well at keeping them under 200 lines.
The vast majority of posts are fairly short, with a peak around 50 lines,
and relatively few exceed 200. Only a couple of outliers get over 300.
I think I'm okay with that. Whether you, the readers, agree --
well, feel free to tell me!
The Mercury transit is over. But we learned some interesting things.
I'd seen Mercury transits before, but this is the first time we had an
H-alpha scope (a little 50mm Coronado PST) in addition to a white light
filter (I had my 102mm refractor set up with the Orion white-light filter).
As egress approached, Dave was viewing in the H-alpha while I was on
the white light scope. When I saw the black-drop effect at third
contact, Mercury was still nowhere near the edge in the H-alpha:
the H-alpha shows more of the solar atmosphere so the sun's image
is noticably bigger.
This was the point when we realized that we should have expected this
and been timing and recording. Alas, it was too late.
Mercury was roughly 60% out in the white light filter -- just past the
point where the "bite" it made in the limb of the sun -- by the time
Dave called out third contact. We guessed it was roughly a minute,
but that could be way off.
For fourth contact, Dave counted roughly 45 seconds between when I
couldn't see Mercury any more and when he lost track of it. This is
pretty rough, because it was windy, seeing was terrible and there
was at least a 15-second slop when I wasn't sure if I could any
indentation in the limb; I'm sure it was at least as hard in the
Coronado, which was running at much lower magnification.
So we had a chance to do interesting science and we flubbed it.
And the next chance isn't til 2032; who knows if we'll still be
actively observing then.
I wanted to at least correlate those two numbers: 45 seconds and
60% of a Mercury radius.
Mercury is about 10" (arcseconds) right now. That was easy to find.
But how fast does it move? I couldn't find anything about that,
searching for terms like mercury transit angular speed OR velocity.
I tried to calculate it with PyEphem but got a number that was orders of
magnitude off. Maybe I'll figure it out for a later article, but I wanted
to get this posted quickly.
I didn't spend much time trying photography. I got a couple afocal snaps with
my pocket digital camera through the white-light scope that worked out pretty well.
I wasn't sure that would work for the Coronado: the image is fairly dim.
The snaps I did get show Mercury, though none of the interesting detail
like faculae and the one tiny prominence that was visible. But the
interesting thing is the color. To the eye, the H-alpha scope image is
a slightly orangy red, but in the digital camera it came out a
startling purplish pink. This may be due to the digital camera's filters
passing some IR, confusing the algorithms that decide how to shift the color.
Of course, I could have adjusted the color in GIMP back to the real color,
but I thought it was more interesting to leave it the hue it came out
of the camera. (I did boost contrast and run an unsharp mask filter, to
make it easier to see Mercury.)
Anyway, fun and unexpectedly edifying! I wish we had another transit
happening sooner than 2032.
Mercury Transit 2006, photo by Brocken Inaglory
Next Monday, November 11, is a transit of Mercury across the sun.
Mercury transits aren't super rare -- not once- or twice-in-a-lifetime
events like
Venus
transits -- but they're not that common, either.
The last Mercury transit was in 2016; the next one won't happen til 2032.
This year's transit isn't ideal for US observers. The transit will
already be well underway by the time the sun rises, at least in the
western US. Here in New Mexico (Mountain time), the sun rises with
Mercury transiting, and the transit lasts until 11:04 MST.
Everybody else, check
timeanddate's
Mercury Transit page for your local times.
Mercury is small, unfortunately, so it's not an easy thing to see
without magnification. Of course, you know that
you should never look at the sun without an adequate filter.
But even if you have safe "eclipse glasses", it may be tough to
spot Mercury's small disk against the surface of the sun.
One option is to take some binoculars and use them to project an image.
Point the big end of the binoculars at the sun, and the small end at
a white surface, preferably leaning so it's perpendicular to the sun.
I don't know if binocular projection will give a big enough image
to show Mercury, so a very smooth and white background, tilted so
it's perpendicular to the sun, will help.
(Don't be tempted to stick eclipse glasses in front of a
binocular or telescope and look through the eyepiece! Stick to
projection unless you have filters specifically intended for
telescopes or binoculars.)
Of course, a telescope with a safe solar filter is the best way to see a
transit. If you're in the Los Alamos area, I hear the Pajarito
Astronomers are planning to set up telescopes at Overlook Park.
They don't seem to have announced it in any of the papers yet, but
I see it listed on the
Pajarito Astronomers
website.
There's also an event planned at the high school where the students
will be trying to time Mercury's passage, but I don't know if
that's open to the public. Elsewhere in the world, check with your local
astronomy club for Mercury transit parties: I'm sure most clubs have
something planned.
I was discussing the transit with a couple of local astronomers earlier
this week, and one of them related it to the search for exoplanets.
One of the main methods of detecting exoplanets is to measure the dimming
of a star's light as a planet crosses its face.
For instance, in
55 Cancri e,
you can see a dimming as the planet crosses the star's face, and a
much more subtle dimming when the planet disappears behind the star.
As Mercury crosses the Sun's face, it blocks some of the sun's light
in the same way. By how much?
The radius of Mercury is 0.0035068 solar radii, and the dimming is
proportional to area so it should be 0.00350682, or
0.0000123, a 0.00123% dimming. Not very much!
But it looks like in the 55 Cancri e case, they're detecting dips
of around .001% -- it seems amazing that you could detect a planet
as small as Mercury this way (and certainly the planet is much bigger
in the case of 55 Cancri e) ... but maybe it's possible.
Anyway, it's fun to think about exoplanets as you watch tiny Mercury
make its way across the face of the Sun.
Wherever you are, I hope you get a chance to look!
I was planning to teach a class on Raspberry Pis, and I wanted to
start with the standard Raspbian image, update it, and add some
programs like GIMP that we'd use in the class. And I wanted to do
that just once, before burning the image to a bunch of different SD cards.
Is there a way to do that on my regular Linux box, with its nice
fast processor and disk?
Why yes, there is, and it's pretty easy. But there are a lot
of unclear or misleading tutorials out there, so I hope this is
a bit simpler and easier to follow.
I got most of this from a tutorial that no longer seems to be available
(but I'll include the link in case it comes back):
Solar Staker/RPI Image/Creation.
I've tested this on Ubuntu 19.10, Debian Stretch and Buster; the
instructions should be pretty general except for the name of the
loopback mount. Commands you type are in bold;
the rest is the output you should see. $ is your normal shell
prompt and # is a root prompt.
Required Packages
You'll need kpartx, qemu and some supporting packages.
On Debian or Ubuntu:
kpartx can read the Raspbian ISO image and split it into the two
filesystems it would have if you wrote it to an SD card.
It turns the filesystems into loopback devices you can mount
like regular filesystems.
Make a note of those loopback device names. They may not always
be loop10p1 and loop10p2, but they'll probably always end in
1 and 2. 1 is the Raspbian /boot filesystem, 2 is the Raspbian root.
(Optional) Check and Possibly Resize the Raspbian Filesystem
Make sure that the Raspbian filesystem is intact, and that it's a
reasonable size.
In practice, I didn't find this made any difference (everything was
fine to begin with), but it doesn't hurt to make sure.
You're ready to mount the two filesystems.
A Raspbian SD card image contains two filesystems.
Partition 1 is a small vfat /boot
filesystem containing the kernel and some other files it needs for
booting, plus the two important configuration files cmdline.txt
and config.txt.
Partition 2 is the Raspbian root filesystem in ext4 format.
The Raspbian root includes an empty /boot directory; mount
the root first, then mount the Raspbian boot partition on
Raspbian's /boot:
$ sudo mkdir /mnt/pi_image
$ sudo mount /dev/mapper/loop10p2 /mnt/pi_image
$ sudo mount /dev/mapper/loop10p1 /mnt/pi_image/boot
Prepare for Chroot
You're going to chroot to the Raspbian filesystem.
Chroot limits the filesystem you can access, so when you type /,
instead of your host filesystem's / you'll see the root of the
Raspbian filesystem, /mnt/pi_image. That means you
won't have access to your host system's /usr/bin, any more.
But qemu needs /usr/bin/qemu-arm-static to be able to emulate ARM
binaries in user mode. So copy that to the Raspbian filesystem so
you'll still be able to access it after the chroot:
Now you're running in Raspbian's root filesystem. All the binaries in
your path (e.g. /bin/ls, /bin/bash) are ARM binaries,
but if you try to run them, qemu will see qemu-arm-static and
run the program as though you're on an actual Raspberry Pi.
Run Stuff!
Now you can run Raspbian commands.
# file /bin/ls
/bin/ls: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 3.2.0, BuildID[sha1]=67a394390830ea3ab4e83b5811c66fea9784ee69, stripped
#
# /bin/ls
bin dev home lost+found mnt proc run srv tmp var
boot etc lib media opt root sbin sys usr
# file /bin/cat
/bin/cat: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 3.2.0, BuildID[sha1]=2239a192f2f277bd1a4892e39a41eba97266b91f, stripped
#
# cat /etc/issue
Raspbian GNU/Linux 10 \n \l
You can even install packages or update the whole system:
# apt update
Get:1 http://raspbian.raspberrypi.org/raspbian buster InRelease [15.0 kB]
Get:2 http://archive.raspberrypi.org/debian buster InRelease [25.2 kB]
Get:3 http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages [13.0 MB]
Get:4 http://archive.raspberrypi.org/debian buster/main armhf Packages [259 kB]
Fetched 13.3 MB in 20s (652 kB/s)
Reading package lists... Done
# apt dist-upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following NEW packages will be installed:
busybox initramfs-tools initramfs-tools-core klibc-utils libklibc linux-base
pigz
The following packages will be upgraded:
dhcpcd5 e2fsprogs file firmware-atheros firmware-brcm80211 firmware-libertas
firmware-misc-nonfree firmware-realtek libcom-err2 libext2fs2 libmagic-mgc
libmagic1 libraspberrypi-bin libraspberrypi-dev libraspberrypi-doc
libraspberrypi0 libss2 libssl1.1 libxml2 libxmuu1 openssh-client
openssh-server openssh-sftp-server openssl raspberrypi-bootloader
raspberrypi-kernel raspi-config rpi-eeprom rpi-eeprom-images ssh sudo
wpasupplicant
32 upgraded, 7 newly installed, 0 to remove and 0 not upgraded.
Need to get 133 MB of archives.
After this operation, 3192 kB of additional disk space will be used.
Do you want to continue? [Y/n]
Pretty neat! Although you're not actually running Raspbian, you can
run Raspbian executables with the Raspbian root filesystem mounted
as though you were actually running on your Raspberry Pi.
Cleaning Up
When you're done with the chroot, just exit that shell (Ctrl-D
or exit).
If you want to undo everything else afterward:
Keep in mind you're not really running Raspbian.
You never booted the Raspbian kernel, and you can't test things
that depend on Raspbian's init system, like whether networking works,
let alone running the Raspbian X desktop or accessing GPIO pins.
This is an ARM emulator, not a Raspberry Pi emulator.
More details
If you want to read more about qemu user mode and how it lets you run
binaries from other architectures, I recommend these links:
![[Length of all blog posts, sorted]](http://shallowsky.com/blog/images/bloglength/bloglen-hist.jpg)
I can plot the sorted data easily with gnuplot:
![[Length of all blog posts, frequency distribution]](http://shallowsky.com/blog/images/bloglength/bloglen-freq.jpg)
Turns out I'm doing pretty well at keeping them under 200 lines.
The vast majority of posts are fairly short, with a peak around 50 lines,
and relatively few exceed 200. Only a couple of outliers get over 300.
![[Mercury transit 11/11/2019]](http://shallowsky.com/images/astro/img_2311c.jpg)
![[Mercury transit 11/11/2019 in H-alpha]](http://shallowsky.com/images/astro/img_2304c.jpg)
![[Mercury Transit 2006, photo by Brocken Inaglory]](https://commons.wikimedia.org/wiki/File:Mercury_transit_1.jpg)
![[binocular projection of a solar eclipse]](http://shallowsky.com/images/annular-eclipse-2012/?pic=img_5283.jpg)
![[RSS feed]](https://shallowsky.com/images/logos/rssicon-25x25.png){kind=small}
![[Twitter icon]](https://shallowsky.com/images/logos/twitter-bird-white-on-blue-25x25.png){kind=small}
![[Mastodon icon]](https://shallowsky.com/images/logos/mastodon.svg){kind=small}
![[envelope]](https://shallowsky.com/images/logos/envelope-25x25.png){kind=small}