Shallow Thoughts

Akkana's Musings on Open Source Computing and Technology, Science, and Nature.

Sat, 17 Feb 2018

Multiplexing Input or Output on a Raspberry Pi Part 2: Port Expanders

In the previous article I talked about Multiplexing input/output using shift registers for a music keyboard project. I ended up with three CD4021 8-bit shift registers cascaded. It worked; but I found that I was spending all my time in the delays between polling each bit serially. I wanted a way to read those bits faster. So I ordered some I/O expander chips.

[Keyboard wired to Raspberry Pi with two MCP23017 port expanders] I/O expander, or port expander, chips take a lot of the hassle out of multiplexing. Instead of writing code to read bits serially, you can use I2C. Some chips also have built-in pullup resistors, so you don't need all those extra wires for pullups or pulldowns. There are lots of options, but two common chips are the MCP23017, which controls 16 lines, and the MCP23008 and PCF8574p, which each handle 8. I'll only discuss the MCP23017 here, because if eight is good, surely sixteen is better! But the MCP23008 is basically the same thing with fewer I/O lines.

A good tutorial to get you started is How To Use A MCP23017 I2C Port Expander With The Raspberry Pi - 2013 Part 1 along with part 2, Python and part 3, reading input.

I'm not going to try to repeat what's in those tutorials, just fill in some gaps I found. For instance, I didn't find I needed sudo for all those I2C commands in Part 1 since my user is already in the i2c group.

Using Python smbus

Part 2 of that tutorial uses Python smbus, but it doesn't really explain all the magic numbers it uses, so it wasn't obvious how to generalize it when I added a second expander chip. It uses this code:

DEVICE = 0x20 # Device address (A0-A2)
IODIRA = 0x00 # Pin direction register
OLATA  = 0x14 # Register for outputs
GPIOA  = 0x12 # Register for inputs

# Set all GPA pins as outputs by setting
# all bits of IODIRA register to 0

# Set output all 7 output bits to 0

DEVICE is the address on the I2C bus, the one you see with i2cdetect -y 1 (20, initially).

IODIRA is the direction: when you call

bus.write_byte_data(DEVICE, IODIRA, 0x00)
you're saying that all eight bits in GPA should be used for output. Zero specifies output, one input: so if you said
bus.write_byte_data(DEVICE, IODIRA, 0x1F)
you'd be specifying that you want to use the lowest five bits for output and the upper four for input.

OLATA = 0x14 is the command to use when writing data:

bus.write_byte_data(DEVICE, OLATA, MyData)
means write data to the eight GPA pins. But what if you want to write to the eight GPB pins instead? Then you'd use
OLATB  = 0x14 #
bus.write_byte_data(DEVICE, OLATB, MyData)

Likewise, if you want to read input from some of the GPB bits, use

GPIOB  = 0x13
val = bus.read_byte_data(DEVICE, GPIOB)

The MCP23017 even has internal pullup resistors you can enable:

GPPUA  = 0x0c    # Pullup resistor on GPA
GPPUB  = 0x0d    # Pullup resistor on GPB
bus.write_byte_data(DEVICE, GPPUB, inmaskB)

Here's a full example: on GitHub.

Using WiringPi

You can also talk to an MCP23017 using the WiringPi library. In that case, you don't set all the bits at once, but instead treat each bit as though it were a separate pin. That's easier to think about conceptually -- you don't have to worry about bit shifting and masking, just use pins one at a time -- but it might be slower if the library is doing a separate read each time you ask for an input bit. It's probably not the right approach to use if you're trying to check a whole keyboard's state at once.

Start by picking a base address for the pin number -- 65 is the lowest you can pick -- and initializing:

pin_base = 65
i2c_addr = 0x20

wiringpi.mcp23017Setup(pin_base, i2c_addr)

Then you can set input or output mode for each pin:

wiringpi.pinMode(pin_base, wiringpi.OUTPUT)
wiringpi.pinMode(input_pin, wiringpi.INPUT)
and then write to or read from each pin:
wiringpi.digitalWrite(pin_no, 1)
val = wiringpi.digitalRead(pin_no)

WiringPi also gives you access to the MCP23017's internal pullup resistors:

wiringpi.pullUpDnControl(input_pin, 2)

Here's an example in Python: on GitHub, and one in C: MCP23017-wiringpi.c on GitHub.

Using multiple MCP23017s

But how do you cascade several MCP23017 chips?

Well, you don't actually cascade them. Since they're I2C devices, you wire them so they each have different addresses on the I2C bus, then query them individually. Happily, that's easier than keeping track of how many bits you've looped through ona shift register.

Pins 15, 16 and 17 on the chip are the address lines, labeled A0, A1 and A2. If you ground all three you get the base address of 0x20. With all three connected to VCC, it will use 0x27 (binary 111 added to the base address). So you can send commands to your first device at 0x20, then to your second one at 0x21 and so on. If you're using WiringPi, you can call mcp23017Setup(pin_base2, i2c_addr2) for your second chip.

I had trouble getting the addresses to work initially, and it turned out the problem wasn't in my understanding of the address line wiring, but that one of my cheap Chinese breadboard had a bad power and ground bus in one quadrant. That's a good lesson for the future: when things don't work as expected, don't assume the breadboard is above suspicion.

Using two MCP23017 chips with their built-in pullup resistors simplified the wiring for my music keyboard enormously, and it made the code cleaner too. Here's the modified code: on GitHub.

What about the speed? It is indeed quite a bit faster than the shift register code. But it's still too laggy to use as a real music keyboard. So I'll still need to do more profiling, and maybe find a faster way of generating notes, if I want to play music on this toy.

Tags: , ,
[ 15:44 Feb 17, 2018    More hardware | permalink to this entry | comments ]

Tue, 13 Feb 2018

Multiplexing Input or Output on a Raspberry Pi Part 1: Shift Registers

I was scouting for parts at a thrift shop and spotted a little 23-key music keyboard. It looked like a fun Raspberry Pi project.

I was hoping it would turn out to use some common protocol like I2C, but when I dissected it, it turned out there was a ribbon cable with 32 wires coming from the keyboard. So each key is a separate pushbutton.

[23-key keyboard wired to a Raspberry Pi] A Raspberry Pi doesn't have that many GPIO pins, and neither does an Arduino Uno. An Arduino Mega does, but buying a Mega to go between the Pi and the keyboard kind of misses the point of scavenging a $3 keyboard; I might as well just buy an I2C or MIDI keyboard. So I needed some sort of I/O multiplexer that would let me read 31 keys using a lot fewer pins.

There are a bunch of different approaches to multiplexing. A lot of keyboards use a matrix approach, but that makes more sense when you're wiring up all the buttons from scratch, not starting with a pre-wired keyboard like this. The two approaches I'll discuss here are shift registers and multiplexer chips.

If you just want to get the job done in the most efficient way, you definitely want a multiplexer (port expander) chip, which I'll cover in Part 2. But for now, let's look at the old-school way: shift registers.

PISO Shift Registers

There are lots of types of shift registers, but for reading lots of inputs, you need a PISO shift register: "Parallel In, Serial Out." That means you can tell the chip to read some number -- typically 8 -- of inputs in parallel, then switch into serial mode and read all the bits one at a time.

Some PISO shift registers can cascade: you can connect a second shift register to the first one and read twice as many bits. For 23 keys I needed three 8-bit shift registers.

Two popular cascading PISO shift registers are the CD4021 and the SN74LS165. They work similarly but they're not exactly the same.

The basic principle with both the CD4021 and the SN74LS165: connect power and ground, and wire up all your inputs to the eight data pins. You'll need pullup or pulldown resistors on each input line, just like you normally would for a pushbutton; I recommend picking up a few high-value (like 1-10k) resistor arrays: you can get these in SIP (single inline package) or DIP (dual-) form factors that plug easily into a breadboard. Resistor arrays can be either independent two pins for each resistor in the array) or bussed (one pin in the chip is a common pin, which you wire to ground for a pulldown or V+ for a pullup; each of the rest of the pins is a resistor). I find bussed networks particularly handy because they can reduce the number of wires you need to run, and with a job where you're multiplexing lots of lines, you'll find that getting the wiring straight is a big part of the job. (See the photo above to see what a snarl this was even with resistor networks.)

For the CD4021, connect three more pins: clock and data pins (labeled CLK and either Q7 or Q8 on the chip's pinout, pins 10 and 3), plus a "latch" pin (labeled M, pin 9). For the SN74LS165, you need one more pin: you need clock and data (labeled CP and Q7, pins 2 and 9), latch (labeled PL, pin 1), and clock enable (labeled CE, pin 15).

At least for the CD4021, some people recommend a 0.1 uF bypass capacitor across the power/ground connections of each CD4021.

If you need to cascade several chips with the CD4021, wire DS (pin 11) from the first chip to Q7 (pin 3), then wire both chips clock lines together and both chips' data lines together. The SN74LS165 is the same: DS (pin 10) to Q8 (pin 9) and tie the clock and data lines together.

Once wired up, you toggle the latch to read the parallel data, then toggle it again and use the clock pin to read the series of bits. You can see the specific details in my Python scripts: on GitHub and on GitHub.

Some References

For wiring diagrams, more background, and Arduino code for the CD4021, read Arduino ShiftIn. For the SN74LS165, read: Arduino: SN74HC165N, 74HC165 8 bit Parallel in/Serial out Shift Register, or Sparkfun: Shift Registers.

Of course, you can use a shift register for output as well as input. In that case you need a SIPO (Serial In, Parallel Out) shift register like a 74HC595. See Arduino ShiftOut: Serial to Parallel Shifting-Out with a 74HC595 Interfacing 74HC595 Serial Shift Register with Raspberry Pi. Another, less common option is the 74HC164N: Using a SN74HC164N Shift Register With Raspberry Pi

For input from my keyboard, initially I used three CD4021s. It basically worked, and you can see the code for it at (older version, for CD4021 shift registers), on GitHub.

But it turned out that looping over all those bits was slow -- I've been advised that you should wait at least 25 microseconds between bits for the CD4021, and even at 10 microseconds I found there wasa significant delay between hitting the key and hearing the note.I thought it might be all the fancy numpy code to generate waveforms for the chords, but when I used the Python profiler, it said most of the program's time was taken up in time.sleep(). Fortunately, there's a faster solution than shift registers: port expanders, which I'll talk about in Multiplexing Part 2: Port Expanders.

Tags: , ,
[ 12:23 Feb 13, 2018    More hardware | permalink to this entry | comments ]

Fri, 02 Feb 2018

Raspberry Pi Console over USB: Configuring an Ethernet Gadget

When I work with a Raspberry Pi from anywhere other than home, I want to make sure I can do what I need to do without a network.

With a Pi model B, you can use an ethernet cable. But that doesn't work with a Pi Zero, at least not without an adapter. The lowest common denominator is a serial cable, and I always recommend that people working with headless Pis get one of these; but there are a lot of things that are difficult or impossible over a serial cable, like file transfer, X forwarding, and running any sort of browser or other network-aware application on the Pi.

Recently I learned how to configure a Pi as a USB ethernet gadget, which lets you network between the Pi and your laptop using only a USB cable. It requires a bit of setup, but it's definitely worth it.

The Cable

The first step is getting the cable. For a Pi Zero or Zero W, use a standard micro-USB cable: you probably have a bunch of them for charging phones (if you're not an Apple person) and other devices. For a model A or B with full-sized ethernet ports, it's a little tricker: you need a USB A to USB A plug, which is a lot less common. USB A is the most common type of USB plug, the type you usually plug into your computer, so you'll need a cable that has one of those on each end.

Set up the Pi

Setting up the Raspberry Pi end requires editing two files in /boot, which you can do either on the Pi itself, or by mounting the first SD card partition on another machine.

In /boot/config.txt add this at the end:


In /boot/cmdline.txt, at the end of the long list of options but on the same line, add a space, followed by: modules-load=dwc2,g_ether

Set a static IP address

This step is optional. In theory you're supposed to use some kind of .local address that Bonjour (the Apple protocol that used to be called zeroconf, and before that was called Rendezvous, and on Linux machines is called Avahi). That doesn't work on my Linux machine. If you don't use Bonjour, finding the Pi over the ethernet link will be much easier if you set it up to use a static IP address. And since there will be nobody else on your USB network besides the Pi and the computer on the other end of the cable, there's no reason not to have a static address: you're not going to collide with anybody else.

You could configure a static IP in /etc/network/interfaces, but that interferes with the way Raspbian handles wi-fi via wpa_supplicant and dhcpcd; so you'd have USB networking but your wi-fi won't work any more.

Instead, configure your address in Raspbian via dhcpcd. Edit /etc/dhcpcd.conf and add this:

interface usb0
static ip_address=
static routers=
static domain_name_servers=

This will tell Raspbian to use address for its USB interface. You'll set up your other computer to use

Now your Pi should be ready to boot with USB networking enabled. Plug in a USB cable (if it's a model A or B) or a micro USB cable (if it's a Zero), plug the other end into your computer, then power up the Pi.

Setting up a Linux machine for USB networking

The final step is to configure your local computer's USB ethernet to use

On Linux, find the name of the USB ethernet interface. This will only show up after you've booted the Pi with the ethernet cable plugged in to both machines.

ip a
The USB interface will probably start eith en and will probably be the last interface shown.

On my Debian machine, the USB network showed up as enp0s26u1u1. So I can configure it thusly (as root, of course):

ip a add dev enp0s26u1u1
ip link set dev enp0s26u1u1 up
(You can also use the older ifconfig rather than ip: sudo ifconfig enp0s26u1u1 up)

You should now be able to ssh into your Raspberry Pi using the address, and you can make an appropriate entry in /etc/hosts, if you wish.

For a less hands-on solution, if you're using Mac or Windows, try Adafruit's USB gadget tutorial. It's possible that might also work for Linux machines running Avahi. If you're using Windows, you might prefer CircuitBasics' ethernet gadget tutorial.

Happy networking!

Tags: , ,
[ 14:53 Feb 02, 2018    More linux | permalink to this entry | comments ]

Thu, 25 Jan 2018

Tricks for Installing a Laser Printer on Linux in CUPS

(Wherein I rant about how bad CUPS has become.)

I had to set up two new printers recently. CUPS hasn't gotten any better since the last time I bought a printer, maybe five years ago; in fact, it's gotten quite a bit worse. I'm amazed at how difficult it was to add these fairly standard laser printers, both of which I'd researched beforehand to make sure they worked with Linux.

It took me about three hours for the first printer. The second one, a few weeks later, "only" took about 45 minutes ... at which point I realized I'd better write everything down so it'll be faster if I need to do it again, or if I get the silly notion that I might want to print from another computer, like my laptop.

I used the CUPS web interface; I didn't try any of the command-line tools.

Figure out the connection type

In the CUPS web interface, after you log in and click on Administration, whether you click on Find New Printers or Add Printer, you're faced with a bunch of identical options with no clue how to choose between them. For example, Find New Printers with a Dell E310dw connected shows:

Available Printers
  • [Add This Printer] Virtual Braille BRF Printer (CUPS-BRF)
  • [Add This Printer] Dell Printer E310dw (Dell Printer E310dw)
  • [Add This Printer] Dell Printer E310dw (Dell Printer E310dw)
  • [Add This Printer] Dell Printer E310dw (Dell Printer E310dw (driverless))

What is a normal human supposed to do with this? What's the difference between the three E210dw entries and which one am I supposed to choose? (Skipping ahead: None of them.) And why is it finding a virtual Braille BRF Printer?

The only way to find out the difference is to choose one, click on Next and look carefully at the URL. For the three E310dw options above, that gives:

Again skipping ahead: none of those are actually right. Go ahead, try all three of them and see. You'll get error messages about empty PPD files. But while you're trying them, write down, for each one, the URL listed as Connection (something like the dnssd:, lpd: or ipp: URLs listed above); and note, in the driver list after you click on your manufacturer, how many entries there are for your printer model, and where they show up in the list. You'll need that information later.

Download some drivers

Muttering about the idiocy of all this -- why ship empty drivers that won't install? Why not just omit drivers if they're not available? Why use the exact same name for three different printer entries and four different driver entries? -- the next step is to download and install the manufacturer's drivers. If you're on anything but Redhat, you'll probably either need to download an RPM and unpack it, or else google for the hidden .deb files that exist on both Dell's and Brother's websites that their sites won't actually find for you.

It might seem like you could just grab the PPD from inside those RPM files and put it wherever CUPS is finding empty ones, but I never got that to work. Much as I dislike installing proprietary .deb files, for both printers that was the only method I found that worked. Both Dell and Brother have two different packages to install. Why two and what's the difference? I don't know.

Once you've installed the printer driver packages, you can go back to the CUPS Add Printer screen. Which hasn't gotten any clearer than before. But for both the Brother and the Dell, ipp: is the only printer protocol that worked. So try each entry until you find the one that starts with ipp:.

Set up an IP address and the correct URL

But wait, you're not done. Because CUPS gives you a URL like ipp://DELL316BAA.local:631/ipp/print, and whatever that .local thing is, it doesn't work. You'll be able to install the printer, but when you try to print to it it fails with "unable to locate printer".

(.local apparently has something to do with assuming you're running a daemon that does "Bonjour", the latest name for the Apple service discovery protocol that was originally called Rendezvous, then renamed to Zeroconf, then to Bonjour. On Linux it's called Avahi, but even with an Avahi daemon this .local thing didn't work for me. At least it made me realize that I had the useless Avahi daemon running, so now I can remove it.).

So go back to Add Printer and click on Internet Printing Protocol (ipp) under Other network printers and click Continue. That takes you to a screen that suggests that you want URLs like:





None of these is actually right. What these printers want -- at least, what both the Brother and the Dell wanted -- was ipp://printerhostname:631/ipp/print

printerhostname? Oh, did I forget to mention static IP? I definitely recommend that you make a static IP for your printer, or at least add it to your router's DHCP list so it always gets the same address. Then you can make an entry in /etc/hosts for printerhostname. I guess that .local thing was supposed to compensate for an address that changes all the time, which might be a nifty idea if it worked, but since it doesn't, make a static IP and use it in your ipp: URL.

Choose a driver

Now, finally! you can move on to choosing a driver. After you pick the manufacturer, you'll be presented with a list that probably includes at least three entries for your printer model. Here's where it helps if you paid attention to how the list looked before you installed the manufacturer's drivers: if there's a new entry for your printer that wasn't there before, that's the non-empty one you want. If there are two or more new entries for your printer that weren't there before, as there were for the Dell ... shrug, all you can do is pick one and hope.

Of course, once you manage to get through configuration to "Printer successfully added", you should immediately run Maintenance->Print Test Page. You may have to power cycle the printer first since it has probably gone to sleep while you were fighting with CUPS.

All this took me maybe three hours the first time, but it only took me about 45 minutes the second time. Hopefully now that I've written this, it'll be much faster next time. At least if I don't succumb to the siren song of thinking a fairly standard laser printer ought to have a driver that's already in CUPS, like they did a decade ago, instead of always needing a download from the manufacturer.

If laser printers are this hard I don't even want to think about what it's like to install a photo printer on Linux these days.

Tags: , , ,
[ 16:19 Jan 25, 2018    More linux | permalink to this entry | comments ]

Sun, 21 Jan 2018

Reading Buttons from a Raspberry Pi

When you attach hardware buttons to a Raspberry Pi's GPIO pin, reading the button's value at any given instant is easy with GPIO.input(). But what if you want to watch for button changes? And how do you do that from a GUI program where the main loop is buried in some library?

Here are some examples of ways to read buttons from a Pi. For this example, I have one side of my button wired to the Raspberry Pi's GPIO 18 and the other side wired to the Pi's 3.3v pin. I'll use the Pi's internal pulldown resistor rather than adding external resistors.

The simplest way: Polling

The obvious way to monitor a button is in a loop, checking the button's value each time:

import RPi.GPIO as GPIO
import time

button_pin = 18


GPIO.setup(button_pin, GPIO.IN, pull_up_down = GPIO.PUD_DOWN)

    while True:
        if GPIO.input(button_pin):


except KeyboardInterrupt:
    print("Cleaning up")

But if you want to be doing something else while you're waiting, instead of just sleeping for a second, it's better to use edge detection.

Edge Detection

GPIO.add_event_detect, will call you back whenever it sees the pin's value change. I'll define a button_handler function that prints out the value of the pin whenever it gets called:

import RPi.GPIO as GPIO
import time

def button_handler(pin):
    print("pin %s's value is %s" % (pin, GPIO.input(pin)))

if __name__ == '__main__':
    button_pin = 18


    GPIO.setup(button_pin, GPIO.IN, pull_up_down = GPIO.PUD_DOWN)

    # events can be GPIO.RISING, GPIO.FALLING, or GPIO.BOTH
    GPIO.add_event_detect(button_pin, GPIO.BOTH,

    except KeyboardInterrupt:

Pretty nifty. But if you try it, you'll probably find that sometimes the value is wrong. You release the switch but it says the value is 1 rather than 0. What's up?

Debounce and Delays

The problem seems to be in the way RPi.GPIO handles that bouncetime=300 parameter.

The bouncetime is there because hardware switches are noisy. As you move the switch from ON to OFF, it doesn't go cleanly all at once from 3.3 volts to 0 volts. Most switches will flicker back and forth between the two values before settling down. To see bounce in action, try the program above without the bouncetime=300. There are ways of fixing bounce in hardware, by adding a capacitor or a Schmitt trigger to the circuit; or you can "debounce" the button in software, by waiting a while after you see a change before acting on it. That's what the bouncetime parameter is for.

But apparently RPi.GPIO, when it handles bouncetime, doesn't always wait quite long enough before calling its event function. It sometimes calls button_handler while the switch is still bouncing, and the value you read might be the wrong one. Increasing bouncetime doesn't help. This seems to be a bug in the RPi.GPIO library.

You'll get more reliable results if you wait a little while before reading the pin's value:

def button_handler(pin):
    time.sleep(.01)    # Wait a while for the pin to settle
    print("pin %s's value is %s" % (pin, GPIO.input(pin)))

Why .01 seconds? Because when I tried it, .001 wasn't enough, and if I used the full bounce time, .3 seconds (corresponding to 300 millisecond bouncetime), I found that the button handler sometimes got called multiple times with the wrong value. I wish I had a better answer for the right amount of time to wait.

Incidentally, the choice of 300 milliseconds for bouncetime is arbitrary and the best value depends on the circuit. You can play around with different values (after commenting out the .01-second sleep) and see how they work with your own circuit and switch.

You might think you could solve the problem by using two handlers:

    GPIO.add_event_detect(button_pin, GPIO.RISING, callback=button_on,
    GPIO.add_event_detect(button_pin, GPIO.FALLING, callback=button_off,
but that apparently isn't allowed: RuntimeError: Conflicting edge detection already enabled for this GPIO channel.

Even if you look just for GPIO.RISING, you'll still get some bogus calls, because there are both rising and falling edges as the switch bounces. Detecting GPIO.BOTH, waiting a short time and checking the pin's value is the only reliable method I've found.

Edge Detection from a GUI Program

And now, the main inspiration for all of this: when you're running a program with a graphical user interface, you don't have control over the event loop. Fortunately, edge detection works fine from a GUI program. For instance, here's a simple TkInter program that monitors a button and shows its state.

import Tkinter
from RPi import GPIO
import time

class ButtonWindow:
    def __init__(self, button_pin):
        self.tkroot = Tkinter.Tk()

        self.label = Tkinter.Label(self.tkroot, text="????",
                                   bg="black", fg="white")
        self.label.pack(padx=5, pady=10, side=Tkinter.LEFT)

        self.button_pin = button_pin

        GPIO.setup(self.button_pin, GPIO.IN, pull_up_down=GPIO.PUD_DOWN)

        GPIO.add_event_detect(self.button_pin, GPIO.BOTH,

    def button_handler(self, channel):
        if GPIO.input(channel):

if __name__ == '__main__':
    win = ButtonWindow(18)

You can see slightly longer versions of these programs in my GitHub Pi Zero Book repository.

Tags: , , ,
[ 11:32 Jan 21, 2018    More hardware | permalink to this entry | comments ]

Sun, 24 Dec 2017

Saving a transparent PNG image from Cairo, in Python

Dave and I will be giving a planetarium talk in February on the analemma and related matters.

Our planetarium, which runs a fiddly and rather limited program called Nightshade, has no way of showing the analemma. Or at least, after trying for nearly a week once, I couldn't find a way. But it can show images, and since I once wrote a Python program to plot the analemma, I figured I could use my program to generate the analemmas I wanted to show and then project them as images onto the planetarium dome.

[analemma simulation] But naturally, I wanted to project just the analemma and associated labels; I didn't want the blue background to cover up the stars the planetarium shows. So I couldn't just use a simple screenshot; I needed a way to get my GTK app to create a transparent image such as a PNG.

That turns out to be hard. GTK can't do it (either GTK2 or GTK3), and people wanting to do anything with transparency are nudged toward the Cairo library. As a first step, I updated my analemma program to use Cairo and GTK3 via gi.repository. Then I dove into Cairo.

I found one C solution for converting an existing Cairo surface to a PNG, but I didn't have much luck with it. But I did find a Python program that draws to a PNG without bothering to create a GUI. I could use that.

The important part of that program is where it creates a new Cairo "surface", and then creates a "context" for that surface:

surface = cairo.ImageSurface(cairo.FORMAT_ARGB32, *imagesize)

cr = cairo.Context(surface)

A Cairo surface is like a canvas to draw on, and it knows how to save itself to a PNG image. A context is the equivalent of a GC in X11 programming: it knows about the current color, font and so forth. So the trick is to create a new surface, create a context, then draw everything all over again with the new context and surface.

A Cairo widget will already have a function to draw everything (in my case, the analemma and all its labels), with this signature:

    def draw(self, widget, ctx):

It already allows passing the context in, so passing in a different context is no problem. I added an argument specifying the background color and transparency, so I could use a blue background in the user interface but a transparent background for the PNG image:

    def draw(self, widget, ctx, background=None):

I also had a minor hitch: in draw(), I was saving the context as self.ctx rather than passing it around to every draw routine. That means calling it with the saved image's context would overwrite the one used for the GUI window. So I save it first.

Here's the final image saving code:

   def save_image(self, outfile):
        dst_surface = cairo.ImageSurface(cairo.FORMAT_ARGB32,
                                         self.width, self.height)

        dst_ctx = cairo.Context(dst_surface)

        # draw() will overwrite self.ctx, so save it first:
        save_ctx = self.ctx

        # Draw everything again to the new context,
        # with a transparent instead of an opaque background:
        self.draw(None, dst_ctx, (0, 0, 1, 0))  # transparent blue

        # Restore the GUI context:
        self.ctx = save_ctx

        print("Saved to", outfile)

Tags: , , , ,
[ 19:39 Dec 24, 2017    More programming | permalink to this entry | comments ]

Sat, 16 Dec 2017

Homemade Arduino Part 2: a Bare Atmega328 Without a Clock

Playing with the ATtiny85 I was struck by how simple the circuit was. Sure, I'd made a homemade Arduino on a breadboard; but with the crystal and all the extra capacitors and resistors it ends up seeming like a lot of parts and wires. If an ATtiny can use a built-in clock and not need all those extra parts, couldn't I use an Atmega328 the same way?

[Circuit for Atmega328 on breadboard with ISP] Why, yes, as it turns out. But there are a few tricks.

Wire it

Wiring a bare Atmega chip is easy. You'll want to keep a good pinout diagram handy, like this Arduino ATmega328 Pinout from HobbyTronics.

For the initial wiring, all you need is two power and two ground lines, the pins marked - and +, plus a pullup resistor on RST (something large, like 10kΩ). The excellent tutorial From Arduino to a Microcontroller on a Breadboard is a good guide if you need additional details: the third section shows a circuit without external clock.

Add an LED and resistor on pin 13 (atmega pin 19, called SCK) so you can test it using a blink program.

Now you need to set up the software.

Set up a hardware profile for a bare Arduino

To program it with the Arduino libraries, you'll need a hardware definition for an atmega328 chip with an internal clock. I used the download from the last section of the excellent tutorial, From Arduino to a Microcontroller on a Breadboard. (Keep that page up: it has good wiring diagrams.)

For Arduino 1.8.5, download and unpack it in your ~/sketchbook/hardware/ directory, making a directory there called breadboard. Then you'll need to make one change: the 1.6 directory is missing a file called pins_arduino.h", so if you try to compile with this hardware definition, you'll get an error like:

mkdir -p build-atmega328bb-atmega328
/usr/local/share/arduino/hardware/tools/avr/bin/avr-g++ -x c++ -include Arduino.h -MMD -c -mmcu=atmega328p -DF_CPU=8000000L -DARDUINO=185 -DARDUINO_ARCH_AVR -D__PROG_TYPES_COMPAT__ -I/usr/local/share/arduino/hardware/arduino/avr/cores/arduino -I/home/akkana/sketchbook/hardware/breadboard/avr/variants/standard    -Wall -ffunction-sections -fdata-sections -Os -fpermissive -fno-exceptions -std=gnu++11 -fno-threadsafe-statics -flto blink.ino -o build-atmega328bb-atmega328/blink.ino.o
In file included from :0:0:
/usr/local/share/arduino/hardware/arduino/avr/cores/arduino/Arduino.h:257:26: fatal error: pins_arduino.h: No such file or directory
 #include "pins_arduino.h"
compilation terminated.
/usr/share/arduino/ recipe for target 'build-atmega328bb-atmega328/blink.ino.o' failed
make: *** [build-atmega328bb-atmega328/blink.ino.o] Error 1

The problem is that it's including these directories:
-I/usr/local/share/arduino/hardware/arduino/avr/cores/arduino -I/home/akkana/sketchbook/hardware/breadboard/avr/variants/standard
but the actual file is in:

You can fix that by making a link from the "standard" directory in your Arduino install to breadboard/avr/variants/standard. On Linux, that would be something like this (Mac and Windows people can substitute their local equivalents):

ln -s /usr/local/share/arduino/hardware/arduino/avr/variants/standard ~/sketchbook/hardware/breadboard/avr/variants/

Now your hardware definition should be ready to go. To check, fire up the IDE and look in Tools->Board for ATmega328 on a breadboard (8 MHz internal clock). Or if you use Arduino-mk, run ALTERNATE_CORE=breadboard make show_boards and make sure it lists atmega328bb ATmega328 on a breadboard (8 MHz internal clock).

Reprogram the Fuses and Bootloader for an Internal Clock

The next trick is that an Atmega chip programmed with the Arduino bootloader is also fused to use an external, 16MHz clock. If you wire it to use its internal 8MHz clock, you won't be able to talk to it with either an ISP or FTDI.

You'll definitely run into this if you pull the CPU out of an Arduino. But even if you buy new chips you may see it: many Atmega328s come pre-programmed with the Arduino bootloader. After all, that's what most people want.

The easiest way to reprogram the fuses is to use the hardware definition you just installed to burn a new bootloader, which resets the fuse settings at the same time. So you need an In-System Programmer, or ISP. You can use an Arduino as an ISP, but I'm told that this tends to be flaky and isn't recommended. After I had problems using an Arduino I ordered a cheap USBtinyUSP, which works fine.

Regardless of which ISP you use, if you wire up your atmega without an external clock when it's fused for one, you won't be able to burn a bootloader. A typical error:

[ ... ]
Reading | ################################################## | 100% 0.02s

avrdude: Device signature = 0x000000 (retrying)

Error while burning bootloader.
Reading | ################################################## | 100% 0.02s

avrdude: Device signature = 0x000000
avrdude: Yikes!  Invalid device signature.
     Double check connections and try again, or use -F to override
     this check.

The solution is to burn the bootloader using an external clock. You can add a crystal and two capacitors to your breadboard circuit if you have them. If not, an easy solution is to pull the chip out of the breadboard, plug it into the socket in an Arduino and burn it there. (Note: if you're using an Arduino as your ISP, you'll need a second Arduino.)

Plug your ISP into the Arduino's ISP header: on an Uno, that's the header labeled ICSP at the end of the chip farthest away from the USB plug. It's a six-pin connector (2x3), it's easy to plug in backward and you can't depend on either the Arduino's header or the ISP's cable being labeled as to direction; if in doubt, use a multimeter in continuity mode to see which pin is ground on each side, then make sure those pins match. Once you're sure, mark your connector somehow so you'll know next time.

In the Arduino IDE, set Tools->Board to ATmega328 on a breadboard (8 MHz internal clock), set Programmer to whatever ISP you're using. then run Tools->Burn Bootloader.

If you're using Arduino-mk instead of the IDE, set up a Makefile that looks like this:

ALTERNATE_CORE = breadboard
BOARD_TAG      = atmega328bb
ISP_PROG     = usbtiny
include /usr/local/share/Arduino-Makefile/
Substitute your ISP, if different, and your location for Then type make burn_bootloader

Program it

Once you're wired, you should be able to program it either with an FTDI board or an ISP, as I discussed in homemade Arduino, Part 1. You should be able to use your minimal Atmega328 to run anything you can run on a normal Arduino (albeit at half the clock speed).

I plan to make a little board with a ZIF socket and connectors for both the USBtinyISP and the FTDI Friend so I don't have to plug in all those wires again each time.

Tags: ,
[ 13:14 Dec 16, 2017    More hardware | permalink to this entry | comments ]

Sat, 09 Dec 2017

Homemade Arduino Part 1: Programming an Atmega328 on a Breadboard

There are lots of tutorials around for building an Arduino on a breadboard, using an Atmega328 (or the older 168) chip, a crystal, a few capacitors and resistors and a power supply. It's a fun project that every Arduino hacker should try at least once.

But while there are lots of instructions on how to wire up a breadboard Arduino, most instructions on how to program one are confusing and incomplete.

Of course, you can program your Atmega chip while it's in an Arduino, then unplug it from the Arduino's socket and move it to the breadboard. But what a hassle! It's so more convenient to leave the chip in the breadboard while you test new versions of the code. And you can, in two different ways: with FTDI, which uses the Arduino bootloader, or with an ISP, which doesn't.

Either way, start by downloading a good pinout diagram for the Atmega328 chip. I use this one: the Arduino ATmega328 Pinout from HobbyTronics, which is very compact yet does a good job of including both the mappings to Arduino digital and analog pins and the functions like RX, TX, MOSI and MISO you'll need for programming the chip.

Load Programs with FTDI

[Circuit for Atmega328 on breadboard with FTDI friend] An FTDI board is a little trickier to wire than an ISP, but it's less risky because it loads the code the same way an Arduino would, so you don't overwrite the bootloader and you can still put your chip back into an Arduino if things go wrong. So let's start with FTDI.

I use an Adafruit "FTDI Friend", but there are lots of similar FTDI boards from Sparkfun and other vendors. They have six outputs, but you'll need only five of those. Referring to your Atmega pinout, wire up power, ground, TX, and RX. For some FTDI boards you may need pullup resistors on the TX and RX lines; I didn't need them.

Now you have four pins connected. Wiring the reset line is more complicated because it requires a 0.1μF capacitor. A lot of tutorials don't mention the capacitor, but it didn't work for me without one. Connect from RTS on the FTDI board, through the 0.1μF cap, to the RST line.

A 0.1μF capacitor is an electrolytic cap with a positive and a negative lead, but the few online tutorials that even mention the capacitor don't bother to say which side is whick. I connected the FTDI friend to the cap's negative lead, and the positive lead to the Atmega chip, and it worked.

You may also need a pullup on that RST/RTS line: a resistor around 10kΩ from the RST pin 1 of the atmega chip to the 5v power line. Note: the Fritzing diagram here shows pullup resistors on RST, TX and RX. You may not need any of them.

Incidentally, RST stands for "reset", while RTS stands for "Ready To Send"; they're not meant as anagrams of each other. The remaining pin on the FTDI friend, CTS, is "Clear To Send" and isn't needed for an Arduino.

Once the wiring is ready, plug in the FTDI board, check to make sure Port is set to whatever port the FTDI board registered, and try uploading a program as if you were uploading to a normal Arduino Uno. And cross your fingers. If it doesn't work, try fiddling with pullups and capacitor values.

Load Programs with an ISP

[Circuit for Atmega328 on breadboard with ISP] An In-System Programmer, or ISP, writes programs straight to the chip, bypassing (and overwriting) the Arduino bootloader. You can also use an ISP to burn a new bootloader and reprogram the fuses on your Arduino, to change parameters like the clock rate. (More on that in Part 2.)

You can use an Arduino as an ISP, but it's somewhat unreliable and prone to unexplained errors. A dedicated ISP isn't expensive, is easier to wire and is more likely to work. A common type of ISP is called a "USBtinyISP", and you can buy one from vendors like Sparkfun or Adafruit, or search for usbtinyisp on sites like ebay or aliexpress.

Update: I've been curious about this flakiness: why does "Arduino as ISP" work fine for some people and utterly fail for others? One person I asked thought it had to do with the way Arduinos reset the RESET line whenever the serial port is opened: so RESET gets toggled at the wrong time as the bootloader code is being transferred. An alternate method that may get around this is Gammon Forum's Atmega bootloader programmer, which includes the bootloader bits as part of the code so it doesn't need to re-open the serial port. Someone else says a 10 uF capacitor between reset and ground should prevent that from happening; and another person says it should be a 100nF capacitor between RST on the programmer and RST on the AVR-chip plus a 10k pullup resistor, Most Arduino-as-ISP tutorials, including the official ones on, don't mention either capacitors or pullups, so that may explain why the method works for some people and not others.

Arduino ISP pinout ISPs typically use a six-pin connector (2x3). It's not always easy to figure out which end is which, so use a multimeter in continuity mode to figure out which pin is ground. Once you're sure, mark your connector so you'll know which pin is pin 1 (MISO, the pin opposite ground).

Once you have your ISP pins straight, refer to your handy-dandy Atmega328 pinout and connect power, ground, MOSI, MISO, SCK, and RST to the appropriate Atmega pins.

All wired up? In the Arduino IDE, set Programmer to your ISP, for instance, USBtinyISP or Arduino as ISP Then use the Upload button to upload sketches. If you prefer Arduino-mk instead of the IDE, add this to your Makefile:

ISP_PROG     = usbtiny
(or whatever ISP you're using). Then type make ispload instead of make upload

Once you have your FTDI or ISP working, then you can think about making an even simpler circuit -- without the external clock and its associated capacitors. But there are a couple of additional tricks to that. Stay tuned for Part 2.

Tags: ,
[ 15:44 Dec 09, 2017    More hardware | permalink to this entry | comments ]