Shallow Thoughts : : Mar
Akkana's Musings on Open Source Computing and Technology, Science, and Nature.
Sun, 31 Mar 2013
Happy Easter! In keeping with the season, here's a dinosaur egg I
spotted on a recent hike.
Okay, or maybe it's just a vaguely egg-shaped rock. But there's been a lot
going on this spring now that the weather is turning.
First, we seem to have Eurasian collared doves nesting somewhere near
our house. There's a dove up on the power pole, cooing, most of the day.
I know I've heard lots of reports of collared doves around the south bay
in past years, particularly down around Morgan Hill, but this is the first
time I'd seen more than a glimpse of them here at home in San Jose.
It's fun to see new species, though I hope these European interlopers
don't push out the native mourning doves entirely.
In addition, the wildflowers have been great out on the trails, especially
around the south end of Windy Hill OSP and Coal Mine Ridge. A hike up
there last week revealed nearly every wildflower on my
that could be in flower now -- California poppy, wild cucumber
(intriguingly also called manroot), giant trillium, hound's tongue,
milkmaids, the most impressive profusion of Indian warrior I've seen,
blue larkspur, miner's lettuce, Sierra suncup,
vetch (it's pretty despite the unfortunate name), red maid,
wild radish, wood sorrel, broom, and my favorite, shooting star.
Dave had to keep waiting for me while I argued with the camera
over macro focus distances. So if you like wildflowers, get out there
and take a look!
[ 17:22 Mar 31, 2013
More nature |
permalink to this entry |
Tue, 26 Mar 2013
Sometimes I need to take a URL from some text app -- like a shell window,
or an IRC comment -- and open it in Firefox.
If it's a standard http://, that's trivial: I highlight the URL
with my house (often a doubleclick will do it), go to my Firefox window
and middleclick somewhere in the content area, anywhere that's not
a link, and Firefox goes to the URL.
That works because selecting anything, in X, copies the selection to
the Primary selection buffer. The Primary selection is different from
the Clipboard selection that's used with Windows and Mac style
Ctrl-X/Ctrl-C/Ctrl-V copy and paste; it's faster and doesn't require
switching between keyboard and mouse. Since your hand is already on the
mouse (from selecting the text), you don't have to move to the keyboard
to type Ctrl-C, then back to the mouse to go to the Firefox window,
then back to the keyboard to type Ctrl-V.
But it fails in some cases. Like when someone says in IRC,
"There's a great example of that at coolhacks.org/greatexample".
You can highlight
coolhacks.org/greatexample and middleclick
in Firefox all you want, but Firefox doesn't recognize it as a URL and
won't go there. Or if I want to highlight a couple of search terms and
pass them into a Google search.
(Rant: middlemouse used to work for these cases, but it was
disabled -- without even an option for getting it back -- due to a lot of
whining in bugzilla by people coming from Windows backgrounds who
didn't like middleclick paste because they found it unexpected, yet
who weren't willing to turn it off with the
So in those cases, what I've been doing is:
- Highlight the URL or search terms
- Switch to the Firefox window
- Ctrl-L -- this focuses and highlights the URL bar without
changing the X selection
- Backspace (or Ctrl-U, Ctrl-K, any key that clears the URLbar)
- Move the mouse to the URL bar (a small target) and middleclick
- Hit Enter
It works, but it's a lot more steps, and entails several switches
between keyboard and mouse. Frustrating!
It would be a little less frustrating if I had a key binding in Firefox
that said "Paste the current X primary selection." A web search shows
that quite a few other people have been bothered by this problem --
for instance, here
-- but without any solutions. Apparently in a lot of apps, Ctrl-Insert
inserts the Primary X selection -- but in Firefox and a few others,
it inserts the Clipboard instead, just like Ctrl-C.
I could write my own fix, by unzipping Firefox's omni.ja file
and editing various .xul and .js files inside it. But if I were doing
that, I could just as easily revert Firefox's original behavior of
going to the link. Neither of these is difficult; the problem is that
every time I Firefox updates (which is about twice a week these days),
things break until I manually go in and unzip the jar and make my
changes again. I used to do that, but I got tired of needing to do it
so often. And I tried to do it via a Firefox extension, until Mozilla
changed the Firefox extension API so that extensions couldn't modify
key bindings any more.
Since Firefox changes so often, it's nicer to have a solution that's
entirely outside of Firefox. And a comment in one of those discussion
threads gave me an idea: make a key binding in my window manager that
uses xset to copy the primary selection to the clipboard, then
use my crikey
program to insert a fake Ctrl-V that Firefox will see.
Here's a command to do that:
xsel -o -p | xsel -i -b; crikey -s 1 "^V"
xsel -o prints a current X selection, and -p specifies
xsel -i sets an X selection to whatever it
gets on standard input (which in this case will be whatever was in the
Primary selection), and -b tells it to set the Clipboard selection.
Then crikey -s 1 "^V" waits one second (I'll probably reduce this after
more testing) and then generates an X event for a Ctrl-V.
I bound that command to Ctrl-Insert in my window manager, Openbox,
<execute>/bin/sh -c 'xsel -o -p | xsel -i -b; crikey -s 1 "^V"'</execute>
Openbox didn't seem happy with the pipe, so I wrapped the whole thing in
Now, whenever I type Ctrl-Insert, whatever program I'm in will do a
Ctrl-V but insert the Primary selection rather than the Clipboard.
It should work in other recalcitrant programs, like LibreOffice, as well.
In Firefox, now, I just have to type Ctrl-L Ctrl-Insert Return.
Of course, it's easy enough to make a binding specific to Firefox
that does the Ctrl-L and the Return automatically. I've bound that
to Alt-Insert, and its execute line looks like this:
<execute>/bin/sh -c 'xsel -o -p | xsel -i -b; crikey -s 1 "^L^V\\n"'</execute>
Fun with Linux! Now the only hard part will be remembering to use the
bindings instead of doing things the hard way.
[ 20:35 Mar 26, 2013
More linux/cmdline |
permalink to this entry |
Tue, 19 Mar 2013
One of the closing lightning talks at PyCon this year concerned the answers
to a list of Python programming puzzles given at some other point during
the conference. I hadn't seen the questions (I'm still not sure
where they are), but some of the problems looked fun.
One of them was: "What are the letters not used in Python keywords?"
I hadn't known about Python's
keyword module, which could
come in handy some day:
>>> import keyword
['and', 'as', 'assert', 'break', 'class', 'continue', 'def', 'del', 'elif', 'else', 'except', 'exec', 'finally', 'for', 'from', 'global', 'if', 'import', 'in', 'is', 'lambda', 'not', 'or', 'pass', 'print', 'raise', 'return', 'try', 'while', 'with', 'yield']
So, given the list of keywords, what's the best way to find the list
of unique letters?
Any time you want a list of unique anything, you want a
>>> set([1, 2, 3, 2, 2, 4, 5, 1, 5])
set([1, 2, 3, 4, 5])
But first you need a list of letters so can make a set out of it.
Split the list of words into a list of letters
My first idea was to use list comprehensions. You can split a single
word into letters like this:
>>> [ x for x in 'hello' ]
['h', 'e', 'l', 'l', 'o']
It took a bit of fiddling to get the right syntax to apply that to
every word in the list:
>>> [[c for c in w] for w in keyword.kwlist]
[['a', 'n', 'd'], ['a', 's'], ['a', 's', 's', 'e', 'r', 't'], ... ]
Update: Dave Foster points out that
[list(w) for w in keyword.kwlist] is another way,
simpler and cleaner way than the double list comprehension.
That's a list of lists, so it needs to be flattened into a single
list of letters before we can turn it into a set.
Flatten the list of lists
There are lots of ways to flatten a list of lists.
Here are four of them:
[item for sublist in [[c for c in w] for w in keyword.kwlist] for item in sublist]
reduce(lambda x,y: x+y, [[c for c in w] for w in keyword.kwlist])
list(itertools.chain.from_iterable([[c for c in w] for w in keyword.kwlist]))
sum([[c for c in w] for w in keyword.kwlist], )
That last one, using sum(), makes use of the fact that
Python uses + for list concatenation -- in other words, that
[1, 2, 3] + [4, 5, 6] is
[1, 2, 3, 4, 5, 6].
But the first method (
item for sublist in) is faster: see
Making a flat list out of list of lists in Python
And another StackOverflow thread has a
for plotting speed vs. list size of various flatteners.
A simpler way of making the set
But it turns out none of this list comprehension stuff is needed anyway.
set('word') splits words into letters already:
set(['e', 'b', 'u', 'l'])
Ignore the order -- elements of a set often end up displaying in some
strange order. The important thing is that it has all the letters
and no repeats.
Now we have an easy way of making a set containing the letters in
one word. But how do we apply that to a list of words?
Again I initially tried using list comprehensions, then realized
there's an easier way. Given a list of strings, it's trivial to
join them into a single string using ''.join(). And that gives us
our set of letters within keywords:
set(['a', 'c', 'b', 'e', 'd', 'g', 'f', 'i', 'h', 'k', 'm', 'l', 'o', 'n', 'p', 's', 'r', 'u', 't', 'w', 'y', 'x'])
What letters are not in the set?
Almost done! But the original problem was to find the letters not in
keywords. We can do that by subtracting this set from the set of all
letters from a to z. How do we get that? The
module will give us a list:
You could also use a list comprehension and
range won't give you a range of
>>> [chr(i) for i in range(ord('a'), ord('z')+1)]
['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z']
It's a bit longer, but doesn't require an import.
Now that you have your a-z set, just subtract the two sets:
>>> set(string.lowercase[:]) - set(''.join(keyword.kwlist))
set(['q', 'j', 'z', 'v'])
So the only letters not used in Python keywords are q, j, z and v.
Just a useless little ditty, really ... but I thought it was a fun exercise,
so maybe you will too.
[ 13:36 Mar 19, 2013
More programming |
permalink to this entry |
Sat, 16 Mar 2013
I'm at PyCon, and I spent a lot of the afternoon in the Raspberry Pi lab.
Raspberry Pis are big at PyCon this year -- because everybody at
the conference got a free RPi! To encourage everyone to play, they
have a lab set up, well equipped with monitors, keyboards, power
and ethernet cables, plus a collection of breadboards, wires, LEDs,
switches and sensors.
I'm primarily interested in the RPi as a robotics controller,
one powerful enough to run a camera and do some minimal image processing
(which an Arduino can't do).
And on Thursday, I attended a PyCon tutorial on the Python image processing
It's a wrapper for OpenCV that makes it easy to access parts of images,
do basic transforms like greyscale, monochrome, blur, flip and rotate,
do edge and line detection, and even detect faces and other objects.
Sounded like just the ticket, if I could get it to work on a Raspberry Pi.
SimpleCV can be a bit tricky to install on Mac and Windows, apparently.
But the README on the SimpleCV
git repository gives an easy 2-line install for Ubuntu. It doesn't
run on Debian Squeeze (though it installs), because apparently it
depends on a recent version of pygame and Squeeze's is too old;
but Ubuntu Pangolin handled it just fine.
The question was, would it work on Raspbian Wheezy? Seemed like a
perfect project to try out in the PyCon RPi lab. Once my RPi was
set up and I'd run an
apt-get update, I used
used netsurf (the most modern of the lightweight browsers available
on the RPi) to browse to the
The first line,
sudo apt-get install ipython python-opencv python-scipy python-numpy python-pygame python-setuptools python-pip
was no problem. All those packages are available in the Raspbian repositories.
But the second line,
sudo pip install https://github.com/ingenuitas/SimpleCV/zipball/master
failed miserably. Seems that pip likes to put its large downloaded
files in /tmp; and on Raspbian, running off an SD card, /tmp quite
reasonably is a tmpfs, running in RAM. But that means it's quite small,
and programs that expect to be able to use it to store large files
are doomed to failure.
I tried a couple of simple Linux patches, with no success.
You can't rename /tmp to replace it with a symlink to a directory on the
SD card, because /tmp is always in use. And pip makes a new temp directory
name each time it's run, so you can't just symlink the pip location to
a place on the SD card.
I thought about rebooting after editing the tmpfs out of /etc/fstab,
but it turns out it's not set up there, and it wasn't obvious how to
disable the tmpfs. Searching later from home, the size is
set in /etc/default/tmpfs. As for disabling the tmpfs and using the
SD card instead, it's not clear. There's a block of code in
/etc/init.d/mountkernfs.sh that makes that decision; it looks like
symlinking /tmp to somewhere else might do it, or else commenting out
the code that sets RAMTMP="yes". But I haven't tested that.
Instead of rebooting, I downloaded the file to the SD card:
But it turned out it's not so easy to pip install from a local file.
After much fussing around I came up with this, which worked:
pip install http:///home/pi/master --download-cache /home/pi/tmp
That worked, and the resulting SimpleCV install worked nicely!
I typed some simple tests into the simplecv shell, playing around
with their built-in test image "lenna":
img = Image('lenna')
And, for something a little harder, some face feature detection:
let's find her eyes and outline them in yellow.
SimpleCV is lots of fun! And the edge detection was quite fast on the RPi --
this may well be usable by a robot, once I get the motors going.
[ 21:43 Mar 16, 2013
More linux/install |
permalink to this entry |
Tue, 12 Mar 2013
I wrote last week about an upcoming
of Europa by Jupiter's shadow. One of the interesting things I'd found
was how much the predicted times of Europa's appearance from behind
Jupiter, and subsequent disappearance into Jupiter's shadow, varied
depending on which program you were using. I had just recently managed
page showing eclipse events, and its times didn't agree with any of the
other programs either. So I was burning with curiosity to know who was right.
The predicted times were:
| ||Europa appears ||Europa disappears
|XEphem ||7:43 ||7:59
|S&T Jupiter's Moons ||7:40 ||7:44
|Stellarium ||6:49 ||7:05
I was out of town on March 10. I brought along a travel scope,
an Orion 80mm f/6 Orion Express. Not the perfect planetary scope, but
certainly enough to see Europa. (The Galilean moons are even visible in
binoculars, as long as you mount the binoculars on a tripod or
otherwise hold them steady.)
I synchronized my watch and had the telescope set up by 7:35. Sure enough,
there was no Europa there. But at 7:38 on the dot, I saw the first
hint of Europa peeking out. No question about it. I watched, and
timed, and by 7:41 the whole disk of Europa was visible and I could
start to think I could see blackness between it and Jupiter.
I'd been to a school star party a few days earlier and hadn't cleaned
my eyepieces afterward -- oops! -- so the view was a little foggy
and it was hard to tell for sure exactly when Europa's disk cleared Jupiter.
In fact, no matter which eyepiece I used, the fogginess seemed to get
worse and worse. I had a hard time seeing Europa at all. Finally I
realized that I was looking through a tree branch, and moved the
scope. But by the time I got it moved again, Europa had gotten
even harder to see. That was when I realized that it had been going
into eclipse practically the whole time I was watching it.
It was already significantly dimmed by 7:43, very dim indeed by 7:48
and gone -- in the 80mm -- by 7:49:20, though I suspect it still would
have been visible in a larger scope with clean eyepieces.
So that's why the times in different programs varied so much! Galilean
moons aren't point sources: you can't predict a single time for a moon
disappearance, appearance or eclipse. Do you want to predict the
beginning of the event, the end of the event or the time at the moon's
And that goes double for eclipses, where the moon is gradually sliding
into the shadow of Jupiter's atmosphere. I found that it took over
seven minutes the moon to go from full brightness to fully eclipsed.
So what part of that do you predict?
All in all, a very interesting observing session. I'm looking forward to
observing more of these eclipses, doing more timings, and tuning my
program to give better predictions. (I notice my program was
significantly late on both the appearance and the eclipse. I'll work
on that. Better to err on the early side, and not miss anything!)
While I was adding eclipses to my Jupiter program, I also added
longer-range predictions, so it would be easier to find out when these events
will happen. Once that was implemented, I looked for upcoming Whac-a-Moon
events. I found one on Mar 26, when Ganymede appears at 7:29pm PDT
(add 7 hours for GMT).
Europa and its shadow are transiting Jupiter's disk, too, so there's
plenty to look at. Ganymede then enters eclipse at 9:40pm PDT.
A long time between the events, I know, but it's easy enough to leave
a scope set up in the backyard and go out to check it now and then.
a few minutes late. Always be ready at least five minutes early in
case the predictions are off, no matter which program you use.
Don't say I didn't warn you.
I found no events in April visible at night in California
(for other time zones, you can generate predictions on my
But May 8 has a decent one:
Ganymede appears at 9:44pm PDT, then disappears into
eclipse at 9:46. Based on what I saw tonight with Europa, that means
the moon should start to fade almost immediately after it has emerged from
behind Jupiter, maybe even before it has fully emerged. Ganymede's
larger size may also mean the fade-out will take longer. Stay tuned.
Jupiter will be very low by then, only 7 degrees above the horizon.
Not many events to observe -- this is a bit rarer than I'd thought.
Of course, there are lots of moons disappearing into eclipse and
appearing from out of it every night, so watching that long gradual
appearance or disappearance isn't difficult; the only rare part is
when they appear briefly between Jupiter and Jupiter's shadow.
That is relatively rare, and I'm glad I had a chance to catch it.
[ 19:55 Mar 12, 2013
More science/astro |
permalink to this entry |
Sat, 09 Mar 2013
This is an edited and updated version of my "Shallow Sky" column
this month in the
SJAA Ephemeris newsletter.
A few months ago, I got email from a Jupiter observer
calling my attention to an interesting phenomenon of Jupiter's moons
that I hadn't seen before. The person who mailed me described himself
as a novice, and wasn't quite sure what he had seen, but he knew it
was odd. After some further discussion we pinned it down.
He was observing Jupiter at 11/11/12 at 00.25 UT (which would have
been mid-afternoon here in San Jose). Three of the moons were
visible, with only Ganymede missing. Then Ganymede appeared: near
Jupiter's limb, but not right on it. As he watched over the next
few minutes, Ganymede seemed to be moving backward -- in toward Jupiter
rather than away from it. Eventually it disappeared behind the planet.
It turned out that what he was seeing was the end of an eclipse.
Jupiter was still a few months away from opposition, so the shadow
thrown by the planet streamed off to one side as seen from our
inner-planet vantage point on Earth. At 0:26 UT on that evening, long
before he started observing, Ganymede, still far away from Jupiter's
limb, had entered Jupiter's shadow and disappeared into eclipse. It
took over two hours for Ganymede to cross Jupiter's shadow; but at
2:36, when it left the shadow, it hadn't yet disappeared behind the
planet. So it became visible again. It wasn't until 2:50
that Ganymede finally disappeared behind Jupiter.
So it was an interesting effect -- bright Ganymede appearing out of
nowhere, moving in toward Jupiter then disappearing again fourteen
minutes later. It was something I'd never seen, or thought to look for.
It's sort of like playing Whac-a-mole -- the moon appears only
briefly, so you've got to hit it with your telescope at just the right
time if you want to catch it before it disappears again.
A lot of programs don't show this eclipse effect -- including, I'm sad
Jupiter's moons web page. (I have since remedied that.)
The open source program Stellarium shows the effect; on the web,
Sky and Telescope's Jupiter's Moons page shows it, and even prints out
a table of times of various moon events, including eclipses.
These eclipse events aren't all that uncommon -- but only when the sun
angle is just right.
Searching in late February and early March this year, I found
several events for Ganymede and Europa (though, sadly, many of them were
during our daytime). By mid-March, the angles have changed so that
Europa doesn't leave Jupiter's shadow until after it's disappeared
behind the planet's limb; but Ganymede is farther out, so we can see
Ganymede appearances all the way through March and for months after.
The most interesting view, it seems to me, is right on the boundary
when the moon only appears for a short time before disappearing again.
Like the Europa eclipse that's happening this Sunday night, March 10.
Reporting on that one got a little tricky -- because that's the day we
switch to Daylight Savings time. I have to confess that I got a little
twisted up trying to compare results between programs that use UTC and
programs that use local time -- especially when the time zone converter
I was using to check my math told me "That time doesn't exist!"
Darnit, if we'd all just use UTC all the time, astronomy calculations
would be a lot easier! (Not to mention dropping the silly Daylight
Savings Time fiasco, but that's another rant.)
Before I go into the details, I want to point out that Jupiter's moons
are visible even in binoculars. So even if you don't have a telescope,
grab binoculars and set them up in as steady a way as you can -- if
you don't have a tripod adaptor, try bracing them on the top of a
gate or box.
On Sunday night, March 10, at some time around 7:40 pm PDT,
Europa peeks out from behind Jupiter's northeast limb.
(All times are given in PDT; add 7 hours for GMT.)
The sky will still be bright here in California -- the
sun sets at 7:12 that night -- but Jupiter will be 66 degrees up and
well away from the sun, so it shouldn't give you too much trouble.
Once Europa pops out, keep a close eye on it -- because if Sky & Tel's
calculations are right, it will disappear again just four minutes
later, at 7:44, into eclipse in Jupiter's shadow. It will remain
invisible for almost three hours, finally reappearing out of nowhere,
well off Jupiter's limb, at around 10:24 pm.
Here's a link to my
Jupiter just before Europa reappears.
I want to stress that those times are approximate. In fact,
I tried simulating the event in several different programs, and got
wildly varying times:
| ||Io disappears ||Europa appears ||Europa disappears ||Europa reappears ||Io appears
|XEphem ||7:15 ||7:43 ||7:59 ||10:06 ||10:43
|S&T Jupiter's Moons ||7:16 ||7:40 ||7:44 ||10:24 ||10:48
|Stellarium ||6:21 ||6:49 ||7:05 ||9:32 ||10:01
You'll note Stellarium seems to have a time zone problem ...
maybe because I ran the prediction while we were still in standard time,
not daylight savings time.
I'm looking forward to timing the events to see which program is
most accurate. I'm betting on XEphem. Once I know the real times,
If anyone else times the event, please send me your times, in case
something goes wrong here!
Anyway, the spread of times makes it clear that when observing this
sort of phenomenon, you should always set up the telescope ten or
fifteen minutes early, just in case. And ten extra minutes spent
observing Jupiter -- even without moons -- is certainly never
time wasted! Just keep an eye out for Europa to appear -- and be
ready to whack that moon before it disappears again.
[ 11:30 Mar 09, 2013
More science/astro |
permalink to this entry |
Mon, 04 Mar 2013
My Lenovo laptop has a nifty button, Fn-F5, to toggle wi-fi and bluetooth
on and off. Works fine, and the indicator lights (of which the Lenovo
has many -- it's quite nice that way) obligingly go off or on.
But when I suspend and resume, the settings aren't remembered.
The machine always comes up with wireless active, even if it wasn't
Since wireless can be a drain on battery life, as well as a potential
security issue, I don't want it on when I'm not actually using it.
So I wanted a way to turn it off programmatically.
The answer, it turns out, is rfkill.
$ rfkill list
0: tpacpi_bluetooth_sw: Bluetooth
Soft blocked: yes
Hard blocked: no
0: phy0: Wireless LAN
Soft blocked: yes
Hard blocked: no
tells you what hardware is currently enabled or disabled.
To toggle something off,
$ rfkill block bluetooth
$ rfkill block wifi
rfkill -h for more details on arguments you can use.
Fn-F5 still works to enable or disable them together.
I think this is being controlled by /etc/acpi/ibm-wireless.sh,
though I can't find where it's tied to Fn-F5.
You can make it automatic by creating /etc/pm/sleep.d/.
(That's on Ubuntu; of course, the exact file location may vary with distro
and version.) To disable wireless on resume, do this:
case "$1" in
rfkill block bluetooth
rfkill block wifi
Of course, you can also tie that into other things, like your current
network scheme, or what wireless networks are visible (which you can
iwlist wlan0 scan).
[ 19:46 Mar 04, 2013
More linux/laptop |
permalink to this entry |