Shallow Thoughts : : linux

Finding versions of installed packages in Debian/Ubuntu

Checking versions in Debian-based systems is a bit of a pain.

This happens to me a couple of times a month: for some reason I need to know what version of something I'm currently running -- often a library, like libgtk. aptitude show will tell you all about a package -- but only if you know its exact name. You can't do aptitude show libgtk or even aptitude show '*libgtk*' -- you have to know that the package name is libgtk2.0-0. Why is it libgtk2.0-0? I have no idea, and it makes no sense to me.

So I always have to do something like aptitude search libgtk | egrep '^i' to find out what packages I have installed that matches the name libgtk, find the package I want, then copy and paste that name after typing aptitude show.

But it turns out it's super easy in Python to query Debian packages using the Python apt package. In fact, this is all the code you need:

import sys
import apt

cache = apt.cache.Cache()

pat = sys.argv[1]

for pkgname in cache.keys():
    if pat in pkgname:
        pkg = cache[pkgname]
        instver = pkg.installed
        if instver:
            print pkg.name, instver.version

Then run aptver libgtk and you're all set.

In practice, I wanted nicer formatting, with columns that lined up, so the actual script is a little longer. I also added a -u flag to show uninstalled packages as well as installed ones. Amusingly, the code to format the columns took about twice as many lines as the code that does the actual work. There doesn't seem to be a standard way of formatting columns in Python, though there are lots of different implementations on the web. Now there's one more -- in my aptver on github.

Tags: linux, debian, ubuntu, python, programming
[ 15:07 May 15, 2013    More linux | permalink to this entry | comments ]

Pasting the Primary X Selection into Firefox

Sometimes I need to take a URL from some text app -- like a shell window, or an IRC comment -- and open it in Firefox.

If it's a standard http://, that's trivial: I highlight the URL with my house (often a doubleclick will do it), go to my Firefox window and middleclick somewhere in the content area, anywhere that's not a link, and Firefox goes to the URL.

That works because selecting anything, in X, copies the selection to the Primary selection buffer. The Primary selection is different from the Clipboard selection that's used with Windows and Mac style Ctrl-X/Ctrl-C/Ctrl-V copy and paste; it's faster and doesn't require switching between keyboard and mouse. Since your hand is already on the mouse (from selecting the text), you don't have to move to the keyboard to type Ctrl-C, then back to the mouse to go to the Firefox window, then back to the keyboard to type Ctrl-V.

But it fails in some cases. Like when someone says in IRC, "There's a great example of that at coolhacks.org/greatexample". You can highlight coolhacks.org/greatexample and middleclick in Firefox all you want, but Firefox doesn't recognize it as a URL and won't go there. Or if I want to highlight a couple of search terms and pass them into a Google search.

(Rant: middlemouse used to work for these cases, but it was disabled -- without even an option for getting it back -- due to a lot of whining in bugzilla by people coming from Windows backgrounds who didn't like middleclick paste because they found it unexpected, yet who weren't willing to turn it off with the middlemouse.contentLoadURL pref).

So in those cases, what I've been doing is:

• Highlight the URL or search terms
• Switch to the Firefox window
• Ctrl-L -- this focuses and highlights the URL bar without changing the X selection
• Backspace (or Ctrl-U, Ctrl-K, any key that clears the URLbar)
• Move the mouse to the URL bar (a small target) and middleclick
• Hit Enter

It works, but it's a lot more steps, and entails several switches between keyboard and mouse. Frustrating!

It would be a little less frustrating if I had a key binding in Firefox that said "Paste the current X primary selection." A web search shows that quite a few other people have been bothered by this problem -- for instance, here and here -- but without any solutions. Apparently in a lot of apps, Ctrl-Insert inserts the Primary X selection -- but in Firefox and a few others, it inserts the Clipboard instead, just like Ctrl-C.

I could write my own fix, by unzipping Firefox's omni.ja file and editing various .xul and .js files inside it. But if I were doing that, I could just as easily revert Firefox's original behavior of going to the link. Neither of these is difficult; the problem is that every time I Firefox updates (which is about twice a week these days), things break until I manually go in and unzip the jar and make my changes again. I used to do that, but I got tired of needing to do it so often. And I tried to do it via a Firefox extension, until Mozilla changed the Firefox extension API so that extensions couldn't modify key bindings any more.

Since Firefox changes so often, it's nicer to have a solution that's entirely outside of Firefox. And a comment in one of those discussion threads gave me an idea: make a key binding in my window manager that uses xset to copy the primary selection to the clipboard, then use my crikey program to insert a fake Ctrl-V that Firefox will see.

Here's a command to do that:

xsel -o -p | xsel -i -b; crikey -s 1 "^V"

xsel -o prints a current X selection, and -p specifies the Primary. xsel -i sets an X selection to whatever it gets on standard input (which in this case will be whatever was in the Primary selection), and -b tells it to set the Clipboard selection.

Then crikey -s 1 "^V" waits one second (I'll probably reduce this after more testing) and then generates an X event for a Ctrl-V.

I bound that command to Ctrl-Insert in my window manager, Openbox, like this:

<keybind key="C-Insert">
  <action name="Execute">
    <execute>/bin/sh -c 'xsel -o -p | xsel -i -b; crikey -s 1 "^V"'</execute>
  </action>
</keybind>

Openbox didn't seem happy with the pipe, so I wrapped the whole thing in a sh -c.

Now, whenever I type Ctrl-Insert, whatever program I'm in will do a Ctrl-V but insert the Primary selection rather than the Clipboard. It should work in other recalcitrant programs, like LibreOffice, as well. In Firefox, now, I just have to type Ctrl-L Ctrl-Insert Return.

Of course, it's easy enough to make a binding specific to Firefox that does the Ctrl-L and the Return automatically. I've bound that to Alt-Insert, and its execute line looks like this:

    <execute>/bin/sh -c 'xsel -o -p | xsel -i -b; crikey -s 1 "^L^V\\n"'</execute>

Fun with Linux! Now the only hard part will be remembering to use the bindings instead of doing things the hard way.

Tags: linux, firefox, X11, cmdline, crikey
[ 19:35 Mar 26, 2013    More linux/cmdline | permalink to this entry | comments ]

SimpleCV on Raspberry Pi

I'm at PyCon, and I spent a lot of the afternoon in the Raspberry Pi lab.

Raspberry Pis are big at PyCon this year -- because everybody at the conference got a free RPi! To encourage everyone to play, they have a lab set up, well equipped with monitors, keyboards, power and ethernet cables, plus a collection of breadboards, wires, LEDs, switches and sensors.

I'm primarily interested in the RPi as a robotics controller, one powerful enough to run a camera and do some minimal image processing (which an Arduino can't do). And on Thursday, I attended a PyCon tutorial on the Python image processing library SimpleCV. It's a wrapper for OpenCV that makes it easy to access parts of images, do basic transforms like greyscale, monochrome, blur, flip and rotate, do edge and line detection, and even detect faces and other objects. Sounded like just the ticket, if I could get it to work on a Raspberry Pi.

SimpleCV can be a bit tricky to install on Mac and Windows, apparently. But the README on the SimpleCV git repository gives an easy 2-line install for Ubuntu. It doesn't run on Debian Squeeze (though it installs), because apparently it depends on a recent version of pygame and Squeeze's is too old; but Ubuntu Pangolin handled it just fine.

The question was, would it work on Raspbian Wheezy? Seemed like a perfect project to try out in the PyCon RPi lab. Once my RPi was set up and I'd run an apt-get update, I used used netsurf (the most modern of the lightweight browsers available on the RPi) to browse to the SimpleCV installation instructions. The first line,

sudo apt-get install ipython python-opencv python-scipy python-numpy python-pygame python-setuptools python-pip

was no problem. All those packages are available in the Raspbian repositories.

But the second line,

sudo pip install https://github.com/ingenuitas/SimpleCV/zipball/master

failed miserably. Seems that pip likes to put its large downloaded files in /tmp; and on Raspbian, running off an SD card, /tmp quite reasonably is a tmpfs, running in RAM. But that means it's quite small, and programs that expect to be able to use it to store large files are doomed to failure.

I tried a couple of simple Linux patches, with no success. You can't rename /tmp to replace it with a symlink to a directory on the SD card, because /tmp is always in use. And pip makes a new temp directory name each time it's run, so you can't just symlink the pip location to a place on the SD card.

I thought about rebooting after editing the tmpfs out of /etc/fstab, but it turns out it's not set up there, and it wasn't obvious how to disable the tmpfs. Searching later from home, the size is set in /etc/default/tmpfs. As for disabling the tmpfs and using the SD card instead, it's not clear. There's a block of code in /etc/init.d/mountkernfs.sh that makes that decision; it looks like symlinking /tmp to somewhere else might do it, or else commenting out the code that sets RAMTMP="yes". But I haven't tested that.

Instead of rebooting, I downloaded the file to the SD card:

wget https://github.com/ingenuitas/SimpleCV/master

But it turned out it's not so easy to pip install from a local file. After much fussing around I came up with this, which worked:

pip install http:///home/pi/master --download-cache /home/pi/tmp

That worked, and the resulting SimpleCV install worked nicely! I typed some simple tests into the simplecv shell, playing around with their built-in test image "lenna":

img = Image('lenna')
img.show()
img.binarize().show()
img.toGray().show()
img.edges().show()
img.invert().show()

And, for something a little harder, some face feature detection: let's find her eyes and outline them in yellow.

img.listHaarFeatures()
img.findHaarFeatures('eye.xml').draw(color=Color.YELLOW)

SimpleCV is lots of fun! And the edge detection was quite fast on the RPi -- this may well be usable by a robot, once I get the motors going.

Tags: raspberry pi, python, programming, hardware, linux
[ 20:43 Mar 16, 2013    More linux/install | permalink to this entry | comments ]

How to enable/disable laptop wireless hardware

My Lenovo laptop has a nifty button, Fn-F5, to toggle wi-fi and bluetooth on and off. Works fine, and the indicator lights (of which the Lenovo has many -- it's quite nice that way) obligingly go off or on.

But when I suspend and resume, the settings aren't remembered. The machine always comes up with wireless active, even if it wasn't before suspending.

Since wireless can be a drain on battery life, as well as a potential security issue, I don't want it on when I'm not actually using it. So I wanted a way to turn it off programmatically.

The answer, it turns out, is rfkill.

$ rfkill list
0: tpacpi_bluetooth_sw: Bluetooth
        Soft blocked: yes
        Hard blocked: no
0: phy0: Wireless LAN
        Soft blocked: yes
        Hard blocked: no

tells you what hardware is currently enabled or disabled.

To toggle something off,

$ rfkill block bluetooth
$ rfkill block wifi

Type rfkill -h for more details on arguments you can use.

Fn-F5 still works to enable or disable them together. I think this is being controlled by /etc/acpi/ibm-wireless.sh, though I can't find where it's tied to Fn-F5.

You can make it automatic by creating /etc/pm/sleep.d/. (That's on Ubuntu; of course, the exact file location may vary with distro and version.) To disable wireless on resume, do this:

#! /bin/sh

case "$1" in
  resume)
    rfkill block bluetooth
    rfkill block wifi
    ;;
esac
exit $?

Of course, you can also tie that into other things, like your current network scheme, or what wireless networks are visible (which you can get with iwlist wlan0 scan).

Tags: linux, ubuntu, laptop, tip
[ 18:46 Mar 04, 2013    More linux/laptop | permalink to this entry | comments ]

Converting C to Python with a vi regexp

I'm fiddling with a serial motor controller board, trying to get it working with a Raspberry Pi. (It works nicely with an Arduino, but one thing I'm learning is that everything hardware-related is far easier with Arduino than with RPi.)

The excellent Arduino library helpfully provided by Pololu has a list of all the commands the board understands. Since it's Arduino, they're in C++, and look something like this:

#define QIK_GET_FIRMWARE_VERSION         0x81
#define QIK_GET_ERROR_BYTE               0x82
#define QIK_GET_CONFIGURATION_PARAMETER  0x83
[ ... ]
#define QIK_CONFIG_DEVICE_ID                        0
#define QIK_CONFIG_PWM_PARAMETER                    1

and so on.

On the Arduino side, I'd prefer to use Python, so I need to get them to look more like:

    QIK_GET_FIRMWARE_VERSION = 0x81
    QIK_GET_ERROR_BYTE = 0x82
    QIK_GET_CONFIGURATION_PARAMETER = 0x83
[ ... ]
    QIK_CONFIG_DEVICE_ID = 0
    QIK_CONFIG_PWM_PARAMETER = 1

... and so on ... with an indent at the beginning of each line since I want this to be part of a class.

There are 32 #defines, so of course, I didn't want to make all those changes by hand. So I used vim. It took a little fiddling -- mostly because I'd forgotten that vim doesn't offer + to mean "one or more repetitions", so I had to use * instead. Here's the expression I ended up with:

.,$s/\#define *\([A-Z0-9_]*\) *\(.*\)/ \1 = \2/

In English, you can read this as:

From the current line to the end of the file (,.$/), look for a pattern consisting of only capital letters, digits and underscores ([A-Z0-9_]). Save that as expression #1 (\( \)). Skip over any spaces, then take the rest of the line (.*), and call it expression #2 (\( \)).

Then replace all that with a new line consisting of 4 spaces, expression 1, a spaced-out equals sign, and expression 2 ( \1 = \2).

Who knew that all you needed was a one-line regular expression to translate C into Python?

(Okay, so maybe it's not quite that simple. Too bad a regexp won't handle the logic inside the library as well, and the pin assignments.)

Tags: regexp, linux, programming, python, editors, vim
[ 20:38 Jan 19, 2013    More linux/editors | permalink to this entry | comments ]

Integrating graphics with text in Emacs

I discussed Emacs's artist-mode a few days ago as a simple, but incomplete, solution to the problem of sketching graphs while taking notes during a math class. But I've found a much better way, one that allows for including any images -- drawings, photos, or screenshots. It took a little work and some custom .emacs code, but I love the result.

Iimage mode

The key is iimage-mode, which displays inline images. In this mode, you put a line in your buffer with a reference to your image file, something like this:

file://myimage.jpg

and Emacs will replace it with the contents of that image. Marvellous!

You can use other patterns for filenames as well, but I'm fine with using URLs. Note there are only two slashes in file:// -- it's a local file in the same directory as the text file being edited.

It's a little tricky to enable it. The docs are not entirely clear on the differences between iimage-mode, turn-on-iimage-mode and iimage-mode-buffer. I found I could get a file that already had existing images to display them with:

  (turn-on-iimage-mode)
  (iimage-mode-buffer t)

Very cool! But too much to type every time. And to use it for note-taking, I needed a way to say, "Create a new image here, let me edit it, then display the image I just edited inline."

Enabling iimage-mode automatically

First, I wanted iimage mode displayed automatically on files in my note-taking directories. I normally use text-mode for these files, with spell checking and line wrapping turned on (auto-fill mode). So I defined a new minor mode based on text-mode:

(define-derived-mode text-img-mode text-mode "Image display mode"
  (auto-fill-mode)
  (turn-on-iimage-mode)
  (iimage-mode-buffer t)
  )

Then I wanted this mode to be called whenever I'm editing a file in my classes directory. So I added it to my auto-mode-alist:

(setq auto-mode-alist
   ...
      (cons '("Docs/classes/" . text-img-mode)
   ...
      auto-mode-alist) )

Inserting a new image

Next, I needed a way to insert an image URL into the buffer and call up an image editor on it. I shouldn't have to type the filename twice and keep track of it; that's what computers are for.

And I needed a drawing program. As a longtime GIMP geek, most of my computer drawing has been in GIMP. But GIMP is overkill for calling up a quick sketch window. I was tempted to use TuxPaint; it's a good sketching app even if you're not five years old, and it's fun and easy to use. But by default, TuxPaint has some features that get in the way of note-taking, like distracting sound effects. I'm sure it's possible to turn those off, and I do plan to investigate that.

I saw a reference to pinta as a lightweight drawing app, but it required a boatload of Mono libraries that I don't otherwise need, and Krita has the same problem with KDE services. So I opted for MyPaint. It works okay, though it's rather slow to start up and has some other issues, so I'm still hoping to find a more lightweight sketching app.

In any case, I fiddled around with start-process until I figured out how to use it to start a program. Then I wrote a little function that lets the user pick a filename, inserts a URL to that filename into the buffer, then calls up mypaint on the file.

(defun img ()
  "Prompt for a filename, then call up mypaint to create an image"
  (interactive)
  (let ((imgfile (read-string "Filename? " "xxx.jpg" 'my-history)))
    (insert "\nfile://" imgfile "\n" )
    (start-process "mypaint" nil "/usr/bin/mypaint" imgfile)
  ))

Worked fine! I can run M-x img, be prompted for a filename, and get a mypaint window where I can make my sketch.

Noticing that a new image has been added

But wait. I finish sketching, write the file and quit mypaint ... and the buffer still shows something like file://xxx.jpg, even if it's showing other images inline. I needed a way to tell it to refresh and load any new images. (I considered having emacs wait for mypaint to exit, but decided I might sometimes want to keep editing while mypaint was still up.)

M-x eval-expression (iimage-mode-buffer t) will do that, but that's a lot of typing to do. Obviously, I needed a key binding.

Strangely enough, C-c i wasn't taken for text buffers, so that seemed like a natural. So I added a key binding to the end of the text-img-mode. iimage-mode-buffer requires that t argument -- it gives an error without it -- so the key binding looks a little more complicated than one that just calls a simple function. I added it to the end of my text-img-mode function.

(define-derived-mode text-img-mode text-mode "Image display mode"
 ...
  (local-set-key "\C-ci" 
    (lambda () (interactive) (iimage-mode-buffer t)))
  )

But after using it a bit, I discovered that this didn't reload images if I edited them a second time. Fortunately, vwood had the answer:

(defun refresh-iimages ()
  "Only way I've found to refresh iimages (without also recentering)"
  (interactive)
  (clear-image-cache nil)
  (iimage-mode nil)
  (iimage-mode t)
  (message "Refreshed images")
 )

I added the message at the end, since otherwise the function left a distracting "Toggling iimage-mode off; better pass an explicit argument" error.

Then the key binding in my text-img-mode became

(local-set-key "\C-ci" 'refresh-iimages)

Inserting a screenshot

Wait -- one more thing. As I actually used text-img-mode to take notes, I discovered that taking screenshots would actually be much more useful than making my own drawings. Then I could copy small sections of the slides and graphs into my notes at the appropriate place, without needing to copy equations at all.

Why not write a function to allow that? The unpleasantly named scrot program fills the bill nicely, and gives me a choice of clicking in a window or dragging out an area of the screen.

(defun screenshot ()
 "Prompt for a filename, then call up scrot to create an interactive screenshot"
  (interactive)
  (let ((imgfile (read-string "Filename? " "scr.jpg" 'my-history)))
    (insert "\nfile://" imgfile "\n" )
    (start-process "scrot" nil "/usr/bin/scrot" "-s" imgfile)
  ))

This turned out to be so useful that I added a key for it in text-img-mode:

(local-set-key "\C-cs" 'screenshot)

I'm so happy with the result! Iimage mode is working great, and having text and images together is turning out to be perfect for note-taking.

My only problem now -- okay, I admit it -- is a tendency to get so excited over inserting screenshots that I get distracted and forget to actually listen to the lecture. I'm sure I'll get over that, but for now, Thank goodness vlc is good at skipping back!

Tags: editors, emacs, graphics, education
[ 12:42 Jan 12, 2013    More linux/editors | permalink to this entry | comments ]

ASCII graphics in Emacs with Artist Mode

I found a cool package in Emacs the other day: Artist Mode. It lets you draw ASCII graphics in your text file.

I was actually looking for the solution to a different problem: taking notes in a math-intensive Coursera class. I've been taking notes in Emacs, but a text editor is awkward for equations and even more awkward for graphs.

What I really wanted was something like the old Claris Works (or so I'm told; I never used it myself) -- something that's primarily a text editor but lets you drawings, equations, and tables when you need to. In theory, word processors like LibreOffice could do that, but in practice they're not very good at switching modes, nor at integrating several types of media into one document. Texmacs is great for the equations and apparently it can do tables too, but it can't do freehand drawing.

And none of these programs is very configurable -- I can't use my fast, comfortable Emacs bindings while typing, and that's a deal-breaker for me, because being able to make corrections quickly makes a huge difference in my typing speed. LibreOffice's key bindings are only partially configurable, and after you've spent half a day chasing down all the action names you need (the ones that are actually available), you upgrade to a newer version and discover you have to do it all over again because there's no way to migrate configuration files. Even Texmacs, ironically, is no better: the documentation claims it's possible to configure key bindings, but it doesn't appear anyone has ever succeeded in figuring out how.

Anyway, ASCII graphics aren't the ultimate solution to note-taking. And I've found a better solution for that, while I'll write about separately. But for now, Artist Mode is just so cool I had to share it.

Enable it by running M-x artist-mode. You can immediately start drawing in your buffer with the mouse. Whatever you draw gets turned into ASCII graphics.

For note-taking, it's fine for scribbling the rough shape of a curve. It takes no time to mouse in a little sketch like

       |                               .. 20
       |                              ..
       |                            ...
    10 |.                          ..
       |..                       ...
       | ...                   ...
       |   ..              ....
       |    .....      .....
       |    ............
       |.....        .............
       +-------------------------------------

It even has primitives (middleclick to get a.menu) for things like lines, rectangles and circles, and for filling regions. When you're done drawing, M-x artist-mode goes back to whatever mode you were using before.

I probably won't use it very much for note taking. But there are times when I've wanted to draw ASCII graphics -- a laborious process in ordinary text modes -- and other times when it would just be fun to play around with my buffer. I'm happy to know about Artist Mode. I may not need it often, but it sure is fun to use now and then.

Tags: editors, emacs, graphics, education
[ 19:02 Jan 10, 2013    More linux/editors | permalink to this entry | comments ]

Customize the Emacs modeline color

I wrote last week about how to customize syntax highlighting colors in Emacs. And as part of that, I ditched the color theme I'd been using and let Emacs go back to its default colors.

Which mostly was fine, except that when I split the window into two windows, to look at two files at once or two different parts of the same file, the separator between the two windows -- the mode line -- was the same grey as Emacs's normal background, so it wasn't very obvious where the window split was.

A web search turned out lots of different ways to set the mode line color. Many of them involve color themes and are fairly complicated. Here's the simplest method I found:

(set-face-foreground 'modeline "white")
(set-face-background 'modeline "purple")
(set-face-background 'modeline-inactive "light blue")

You can set your active mode line to a pretty color, so it stands out a bit and makes it easy to tell which of the visible windows is the one you're actually typing in, and set the inactive mode lines -- windows that are visible but you arne't actually typing in -- to a less striking color.

Tags: editors, emacs
[ 12:50 Jan 02, 2013    More linux/editors | permalink to this entry | comments ]