Shallow Thoughts

Akkana's Musings on Open Source Computing and Technology, Science, and Nature.

Sun, 21 Oct 2018

How to tell sparrows apart

[Sparrow ID page] I was filing an eBird report the other day, dutifully cataloging the first junco of the year and the various other birds that have been hanging around, when a sparrow flew into my binocular field. A chipping sparrow? Probably ... but this one wasn't so clearly marked.

I always have trouble telling the dang sparrows apart. When I open the bird book, I always have to page through dozens of pages of sparrows that are never seen in this county, trying to figure out which one looks most like what I'm seeing.

I used to do that with juncos, but then I made a local copy of a wonderful comparison photo Bob Walker published a couple years ago on the PEEC blog: Bird of the Week – The Dark-eyed Junco. (I also have the same sort of crib sheet for the Raspberry Pi GPIO pins.) Obviously I needed a similar crib sheet for sparrows.

So I collected the best publically-licensed images I could find on the web, and made Sparrows of Los Alamos County, with comparison images close together so I can check them quickly before the bird flies away.

If you live somewhere else so the Los Alamos County list isn't quite what you need, you're welcome to use the code to make your own version.

Tags: , ,
[ 19:18 Oct 21, 2018    More nature | permalink to this entry | comments ]

Sat, 13 Oct 2018

Tape Rabbit

[Packing tape rabbit] I had to mail a package recently, and finished up a roll of packing tape.

I hadn't realized before I removed the tape roll from its built-in dispenser that packing tape was dispensed by rabbits.

Tags:
[ 20:11 Oct 13, 2018    More humor | permalink to this entry | comments ]

Sun, 23 Sep 2018

Writing Solar System Simulations with NAIF SPICE and SpiceyPy

Someone asked me about my Javascript Jupiter code, and whether it used PyEphem. It doesn't, of course, because it's Javascript, not Python (I wish there was something as easy as PyEphem for Javascript!); instead it uses code from the book Astronomical Formulae for Calculators by Jean Meeus. (His better known Astronomical Algorithms, intended for computers rather than calculators, is actually harder to use for programming because Astronomical Algorithms is written for BASIC and the algorithms are relatively hard to translate into other languages, whereas Astronomical Formulae for Calculators concentrates on explaining the algorithms clearly, so you can punch them into a calculator by hand, and this ends up making it fairly easy to implement them in a modern computer language as well.)

Anyway, the person asking also mentioned JPL's page HORIZONS Ephemerides page, which I've certainly found useful at times. Years ago, I tried emailing the site maintainer asking if they might consider releasing the code as open source; it seemed like a reasonable request, given that it came from a government agency and didn't involve anything secret. But I never got an answer.

[SpiceyPy example: Cassini's position] But going to that page today, I find that code is now available! What's available is a massive toolkit called SPICE (it's all in capitals but there's no indication what it might stand for. It comes from NAIF, which is NASA's Navigation and Ancillary Information Facility).

SPICE allows for accurate calculations of all sorts of solar system quantities, from the basic solar system bodies like planets to all of NASA's active and historical public missions. It has bindings for quite a few languages, including C. The official list doesn't include Python, but there's a third-party Python wrapper called SpiceyPy that works fine.

The tricky part of programming with SPICE is that most of the code is hidden away in "kernels" that are specific to the objects and quantities you're calculating. For any given program you'll probably need to download at least four "kernels", maybe more. That wouldn't be a problem except that there's not much help for figuring out which kernels you need and then finding them. There are lots of SPICE examples online but few of them tell you which kernels they need, let alone where to find them.

After wrestling with some of the examples, I learned some tricks for finding kernels, at least enough to get the basic examples working. I've collected what I've learned so far into a new GitHub repository: NAIF SPICE Examples. The README there explains what I know so far about getting kernels; as I learn more, I'll update it.

SPICE isn't easy to use, but it's probably much more accurate than simpler code like PyEphem or my Meeus-based Javascript code, and it can calculate so many more objects. It's definitely something worth knowing about for anyone doing solar system simulations.

Tags: , ,
[ 16:43 Sep 23, 2018    More programming | permalink to this entry | comments ]

Sun, 16 Sep 2018

Printing Two-Sided from the Command Line

The laser printers we bought recently can print on both sides of the page. Nice feature! I've never had access to a printer that can do that before.

But that requires figuring out how to tell the printer to do the right thing. Reading the man page for lp, I spotted the sides option: print -o sides=two-sided-long-edge. But that doesn't work by itself. Adding -n 2 looked like the way to go, but nope! That gives you one sheet that has page 1 on both sides, and a second sheet that has page 2 on both sides. Because of course that's what a normal person would want. Right.

The real answer, after further research and experimentation, turned out to be the collate=true option:

lp -o sides=two-sided-long-edge -o collate=true -d printername file

Tags: , ,
[ 11:05 Sep 16, 2018    More linux | permalink to this entry | comments ]

Mon, 03 Sep 2018

Raspberry Pi Zero as Ethernet Gadget Part 3: An Automated Script

Continuing the discussion of USB networking from a Raspberry Pi Zero or Zero W (Part 1: Configuring an Ethernet Gadget and Part 2: Routing to the Outside World): You've connected your Pi Zero to another Linux computer, which I'll call the gateway computer, via a micro-USB cable. Configuring the Pi end is easy. Configuring the gateway end is easy as long as you know the interface name that corresponds to the gadget.

ip link gave a list of several networking devices; on my laptop right now they include lo, enp3s0, wlp2s0 and enp0s20u1. How do you tell which one is the Pi Gadget? When I tested it on another machine, it showed up as enp0s26u1u1i1. Even aside from my wanting to script it, it's tough for a beginner to guess which interface is the right one.

Try dmesg

Sometimes you can tell by inspecting the output of dmesg | tail. If you run dmesg shortly after you initialized the gadget (either by plugging the USB cable into the gateway computer, you'll see some lines like:

[  639.301065] cdc_ether 3-1:1.0 enp0s20u1: renamed from usb0
[ 9458.218049] usb 3-1: USB disconnect, device number 3
[ 9458.218169] cdc_ether 3-1:1.0 enp0s20u1: unregister 'cdc_ether' usb-0000:00:14.0-1, CDC Ethernet Device
[ 9462.363485] usb 3-1: new high-speed USB device number 4 using xhci_hcd
[ 9462.504635] usb 3-1: New USB device found, idVendor=0525, idProduct=a4a2
[ 9462.504642] usb 3-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 9462.504647] usb 3-1: Product: RNDIS/Ethernet Gadget
[ 9462.504660] usb 3-1: Manufacturer: Linux 4.14.50+ with 20980000.usb
[ 9462.506242] cdc_ether 3-1:1.0 usb0: register 'cdc_ether' at usb-0000:00:14.0-1, CDC Ethernet Device, f2:df:cf:71:b9:92
[ 9462.523189] cdc_ether 3-1:1.0 enp0s20u1: renamed from usb0

(Aside: whose bright idea was it that it would be a good idea to rename usb0 to enp0s26u1u1i1, or wlan0 to wlp2s0? I'm curious exactly who finds their task easier with the name enp0s26u1u1i1 than with usb0. It certainly complicated all sorts of network scripts and howtos when the name wlan0 went away.)

Anyway, from inspecting that dmesg output you can probably figure out the name of your gadget interface. But it would be nice to have something more deterministic, something that could be used from a script. My goal was to have a shell function in my .zshrc, so I could type pigadget and have it set everything up automatically. How to do that?

A More Deterministic Way

First, the name starts with en, meaning it's an ethernet interface, as opposed to wi-fi, loopback, or various other types of networking interface. My laptop also has a built-in ethernet interface, enp3s0, as well as lo0, the loopback or "localhost" interface, and wlp2s0, the wi-fi chip, the one that used to be called wlan0.

Second, it has a 'u' in the name. USB ethernet interfaces start with en and then add suffixes to enumerate all the hubs involved. So the number of 'u's in the name tells you how many hubs are involved; that enp0s26u1u1i1 I saw on my desktop had two hubs in the way, the computer's internal USB hub plus the external one sitting on my desk.

So if you have no USB ethernet interfaces on your computer, looking for an interface name that starts with 'en' and has at least one 'u' would be enough. But if you have USB ethernet, that won't work so well.

Using the MAC Address

You can get some useful information from the MAC address, called "link/ether" in the ip link output. In this case, it's f2:df:cf:71:b9:92, but -- whoops! -- the next time I rebooted the Pi, it became ba:d9:9c:79:c0:ea. The address turns out to be randomly generated and will be different every time. It is possible to set it to a fixed value, and that thread has some suggestions on how, but I think they're out of date, since they reference a kernel module called g_ether whereas the module on my updated Raspbian Stretch is called cdc_ether. I haven't tried.

Anyway, random or not, the MAC address also has one useful property: the first octet (f2 in my first example) will always have the '2' bit set, as an indicator that it's a "locally administered" MAC address rather than one that's globally unique. See the Wikipedia page on MAC addressing for details on the structure of MAC addresses. Both f2 (11110010 in binary) and ba (10111010 binary) have the 2 (00000010) bit set.

No physical networking device, like a USB ethernet dongle, should have that bit set; physical devices have MAC addresses that indicate what company makes them. For instance, Raspberry Pis with networking, like the Pi 3 or Pi Zero W, have interfaces that start with b8:27:eb. Note the 2 bit isn't set in b8.

Most people won't have any USB ethernet devices connected that have the "locally administered" bit set. So it's a fairly good test for a USB ethernet gadget.

Turning That Into a Shell Script

So how do we package that into a pipeline so the shell -- zsh, bash or whatever -- can check whether that 2 bit is set?

First, use ip -o link to print out information about all network interfaces on the system. But really you only need the ones starting with en and containing a u. Splitting out the u isn't easy at this point -- you can check for it later -- but you can at least limit it to lines that have en after a colon-space. That gives output like:

$ ip -o link | grep ": en"
5: enp3s0:  mtu 1500 qdisc pfifo_fast state DOWN mode DEFAULT group default qlen 1000\    link/ether 74:d0:2b:71:7a:3e brd ff:ff:ff:ff:ff:ff
8: enp0s20u1:  mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000\    link/ether f2:df:cf:71:b9:92 brd ff:ff:ff:ff:ff:ff

Within that, you only need two pieces: the interface name (the second word) and the MAC address (the 17th word). Awk is a good tool for picking particular words out of an output line:

$ ip -o link | grep ': en' | awk '{print $2, $17}'
enp3s0: 74:d0:2b:71:7a:3e
enp0s20u1: f2:df:cf:71:b9:92

The next part is harder: you have to get the shell to loop over those output lines, split them into the interface name and the MAC address, then split off the second character of the MAC address and test it as a hexadecimal number to see if the '2' bit is set. I suspected that this would be the time to give up and write a Python script, but no, it turns out zsh and even bash can test bits:

ip -o link | grep en | awk '{print $2, $17}' | \
    while read -r iff mac; do
        # LON is a numeric variable containing the digit we care about.
        # The "let" is required so LON will be a numeric variable,
        # otherwise it's a string and the bitwise test fails.
        let LON=0x$(echo $mac | sed -e 's/:.*//' -e 's/.//')

        # Is the 2 bit set? Meaning it's a locally administered MAC
        if ((($LON & 0x2) != 0)); then
            echo "Bit is set, $iff is the interface"
        fi
    done

Pretty neat! So now we just need to package it up into a shell function and do something useful with $iff when you find one with the bit set: namely, break out of the loop, call ip a add and ip link set to enable networking to the Raspberry Pi gadget, and enable routing so the Pi will be able to get to networks outside this one. Here's the final function:

# Set up a Linux box to talk to a Pi0 using USB gadget on 192.168.0.7:
pigadget() {
    iface=''

    ip -o link | grep en | awk '{print $2, $17}' | \
        while read -r iff mac; do
            # LON is a numeric variable containing the digit we care about.
            # The "let" is required so zsh will know it's numeric,
            # otherwise the bitwise test will fail.
            let LON=0x$(echo $mac | sed -e 's/:.*//' -e 's/.//')

            # Is the 2 bit set? Meaning it's a locally administered MAC
            if ((($LON & 0x2) != 0)); then
                iface=$(echo $iff | sed 's/:.*//')
                break
            fi
        done

    if [[ x$iface == x ]]; then
        echo "No locally administered en interface:"
        ip a | egrep '^[0-9]:'
        echo Bailing.
        return
    fi

    sudo ip a add 192.168.7.1/24 dev $iface
    sudo ip link set dev $iface up

    # Enable routing so the gadget can get to the outside world:
    sudo sh -c 'echo 1 > /proc/sys/net/ipv4/ip_forward'
    sudo iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
}

Tags: , ,
[ 18:41 Sep 03, 2018    More linux | permalink to this entry | comments ]

Fri, 31 Aug 2018

Raspberry Pi Zero as Ethernet Gadget Part 2: Routing to the Outside World

I wrote some time ago about how to use a Raspberry Pi over USB as an "Ethernet Gadget". It's a handy way to talk to a headless Pi Zero or Zero W if you're somewhere where it doesn't already have a wi-fi network configured.

However, the setup I gave in that article doesn't offer a way for the Pi Zero to talk to the outside world. The Pi is set up to use the machine on the other end of the USB cable for routing and DNS, but that doesn't help if the machine on the other end isn't acting as a router or a DNS host.

A lot of the ethernet gadget tutorials I found online explain how to do this on Mac and Windows, but it was tough to find an example for Linux. The best I found was for Slackware, How to connect to the internet over USB from the Raspberry Pi Zero, which should work on any Linux, not just Slackware.

Let's assume you have the Pi running as a gadget and you can talk to it, as discussed in the previous article, so you've run:

sudo ip a add 192.168.7.1/24 dev enp0s20u1
sudo ip link set dev enp0s20u1 up
substituting your network number and the interface name that the Pi created on your Linux machine, which you can find in dmesg | tail or ip link. (In Part 3 I'll talk more about how to find the right interface name if it isn't obvious.)

At this point, the network is up and you should be able to ping the Pi with the address you gave it, assuming you used a static IP: ping 192.168.7.2 If that works, you can ssh to it, assuming you've enabled ssh. But from the Pi's end, all it can see is your machine; it can't get out to the wider world.

For that, you need to enable IP forwarding and masquerading:

sudo sh -c 'echo 1 > /proc/sys/net/ipv4/ip_forward'
sudo iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

Now the Pi can route to the outside world, but it still doesn't have DNS so it can't get any domain names. To test that, on the gateway machine try pinging some well-known host:

$ ping -c 2 google.com
PING google.com (216.58.219.110) 56(84) bytes of data.
64 bytes from mia07s25-in-f14.1e100.net (216.58.219.110): icmp_seq=1 ttl=56 time=78.6 ms
64 bytes from mia07s25-in-f14.1e100.net (216.58.219.110): icmp_seq=2 ttl=56 time=78.7 ms

--- google.com ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 78.646/78.678/78.710/0.032 ms

Take the IP address from that -- e.g. 216.58.219.110 -- then go to a shell on the Pi and try ping -c 2 216.58.219.110, and you should see a response.

DNS with a Public DNS Server

Now all you need is DNS. The easy way is to use one of the free DNS services, like Google's 8.8.8.8. Edit /etc/resolv.conf and add a line like

nameserver 8.8.8.8
and then try pinging some well-known hostname.

If it works, you can make that permanent by editing /etc/resolv.conf, and adding this line:

name_servers=8.8.8.8

Otherwise you'll have to do it every time you boot.

Your Own DNS Server

But not everyone wants to use public nameservers like 8.8.8.8. For one thing, there are privacy implications: it means you're telling Google about every site you ever use for any reason.

Fortunately, there's an easy way around that, and you don't even have to figure out how to configure bind/named. On the gateway box, install dnsmasq, available through your distro's repositories. It will use whatever nameserver you're already using on that machine, and relay it to other machines like your Pi that need the information. I didn't need to configure it at all; it worked right out of the box.

In the next article, Part 3: more about those crazy interface names (why is it enp0s20u1 on my laptop but enp0s26u1u1i1 on my desktop?), how to identify which interface is the gadget by using its MAC, and how to put it all together into a shell function so you can set it up with one command.

Tags: , ,
[ 15:25 Aug 31, 2018    More linux | permalink to this entry | comments ]

Thu, 23 Aug 2018

Making Sure the Debian Kernel is Up To Date

I try to avoid Grub2 on my Linux machines, for reasons I've discussed before. Even if I run it, I usually block it from auto-updating /boot since that tends to overwrite other operating systems. But on a couple of my Debian machines, that has meant needing to notice when a system update has installed a new kernel, so I can update the relevant boot files. Inevitably, I fail to notice, and end up running an out of date kernel.

But didn't Debian use to have a /boot/vmlinuz that always linked to the latest kernel? That was such a good idea: what happened to that?

I'll get to that. But before I found out, I got sidetracked trying to find a way to check whether my kernel was up-to-date, so I could have it warn me of out-of-date kernels when I log in.

That turned out to be fairly easy using uname and a little shell pipery:

# Is the kernel running behind?
kernelvers=$(uname -a | awk '{ print $3; }')
latestvers=$(cd /boot; ls -1 vmlinuz-* | sort --version-sort | tail -1 | sed 's/vmlinuz-//')
if [[ $kernelvers != $latestvers ]]; then
    echo "======= Running kernel $kernelvers but $latestvers is available"
else
    echo "The kernel is up to date"
fi

I put that in my .login. But meanwhile I discovered that that /boot/vmlinuz link still exists -- it just isn't enabled by default for some strange reason. That, of course, is the right way to make sure you're on the latest kernel, and you can do it with the linux-update-symlinks command.

linux-update-symlinks is called automatically when you install a new kernel -- but by default it updates symlinks in the root directory, /, which isn't much help if you're trying to boot off a separate /boot partition.

But you can configure it to notice your /boot partition. Edit /etc/kernel-img.conf and change link_in_boot to yes:

link_in_boot = yes

Then linux-update-symlinks will automatically update the /boot/vmlinuz link whenever you update the kernel, and whatever bootloader you prefer can point to that image. It also updates /boot/vmlinuz.old to point to the previous kernel in case you can't boot from the new one.

Tags: , ,
[ 20:14 Aug 23, 2018    More linux/kernel | permalink to this entry | comments ]

Fri, 17 Aug 2018

Easy DIY Cellphone Stand

Over the years I've picked up a couple of cellphone stands as conference giveaways. A stand is a nice idea, especially if you like to read news articles during mealtime, but the stands I've tried never seem to be quite what I want. Either they're not adjustable, or they're too bulky or heavy to want to carry them around all the time.

A while back, I was browsing on ebay looking for something better than the ones I have. I saw a few that looked like they might be worth trying, but then it occurred to me: I could make one pretty easily that would work better than anything I'd found for sale.

I started with plans that involved wire and a hinge -- the hinge so the two sides of the stand would fold together to fit in a purse or pocket -- and spent a few hours trying different hinge options.I wasn't satisfied, though. And then I realized: all I had to do was bend the wire into the shape I needed. Voilà -- instant lightweight adjustable cellphone stand.

And it has worked great. I've been using it for months and it's much better than any of the prefab stands I had before.

Bend a piece of wire

[Bent wire]

I don't know where this wire came from: it was in my spare-metal-parts collection. You want something a little thinner than coathanger wire, so you can bend it relatively easily; "baling wire" or "rebar wire" is probably about right.

Bend the tips around

[Tips adjusted to fit your phone's width]

Adjust the curve so it's big enough that your cellphone will fit in the crook of the wires.

Bend the back end down, and spread the two halves apart

[Bend the back end down]

Adjust so it fits your phone

[Instant cellphone stand]

Coat the finished stand with rubberized coating (available at your local hardware store in either dip or spray-on varieties) so it won't slide around on tables and won't scratch anything. The finished product is adjustable to any angle you need -- so you can adjust it based on the lighting in any room -- and you can fold the two halves together to make it easy to carry.

Tags: ,
[ 12:06 Aug 17, 2018    More hardware | permalink to this entry | comments ]