Shallow Thoughts : : Sep
Akkana's Musings on Open Source Computing and Technology, Science, and Nature.
Sat, 27 Sep 2014
In the canyons below White Rock there are many wonderful petroglyphs,
some dating back many centuries, like this jaguar:
as well as collections like these:
Of course, to see them you have to negotiate a trail down the basalt cliff
Up the hill in Los Alamos there are petroglyphs too, on trails that are
a bit more accessible ... but I suspect they're not nearly so old.
[ 21:47 Sep 27, 2014
More humor |
permalink to this entry |
Mon, 22 Sep 2014
I had the opportunity to borrow a commercial crittercam
for a week from the local wildlife center.
Having grown frustrated with the high number of false positives on my
Raspberry Pi based
crittercam, I was looking forward to see how a commercial camera compared.
The Bushnell Trophycam I borrowed is a nicely compact,
waterproof unit, meant to strap to a tree or similar object.
It has an 8-megapixel camera that records photos to the SD card -- no
wi-fi. (I believe there are more expensive models that offer wi-fi.)
The camera captures IR as well as visible light, like the PiCam NoIR,
and there's an IR LED illuminator (quite a bit stronger than the cheap
one I bought for my crittercam) as well as what looks like a passive IR sensor.
I know the TrophyCam isn't immune to false positives; I've heard
complaints along those lines from a student who's using them to do
wildlife monitoring for LANL.
But how would it compare with my homebuilt crittercam?
I put out the TrophyCam first night, with bait (sunflower seeds) in
front of the camera. In the morning I had ... nothing. No false
positives, but no critters either. I did have some shots of myself,
walking away from it after setting it up, walking up to it to adjust
it after it got dark, and some sideways shots while I fiddled with the
latches trying to turn it off in the morning, so I know it was
working. But no woodrats -- and I always catch a woodrat or two
in PiCritterCam runs. Besides, the seeds I'd put out were gone,
so somebody had definitely been by during the night. Obviously
I needed a more sensitive setting.
I fiddled with the options, changed the sensitivity from automatic
to the most sensitive setting, and set it out for a second night, side
by side with my Pi Crittercam. This time it did a little better,
though not by much: one nighttime shot with a something in it,
plus one shot of someone's furry back and two shots of a mourning dove
What few nighttime shots there were were mostly so blown out you
couldn't see any detail to be sure. Doesn't this camera know how to
adjust its exposure? The shot here has a creature in it. See it?
I didn't either, at first. It's just to the right of the bush.
You can just see the curve of its back and the beginning of a tail.
Meanwhile, the Pi cam sitting next to it caught eight reasonably exposed
nocturnal woodrat shots and two dove shots after dawn.
And 369 false positives where a leaf had moved in the wind or a dawn
shadow was marching across the ground. The TrophyCam only shot 47
photos total: 24 were of me, fiddling with the camera setup to get
them both pointing in the right direction, leaving 20 false positives.
So the Bushnell, clearly, gives you fewer false positives to hunt
through -- but you're also a lot less likely to catch an actual critter.
It also doesn't deal well with exposures in small areas and close distances:
its IR light source seems to be too bright for the camera to cope with.
I'm guessing, based on the name, that it's designed for shooting
deer walking by fifty feet away, not woodrats at a two-foot distance.
Okay, so let's see what the camera can do in a larger space. The next
two nights I set it up in large open areas to see what walked by. The
first night it caught four rabbit shots that night, with only five
false positives. The quality wasn't great, though: all long exposures
of blurred bunnies. The second night it caught nothing at all
overnight, but three rabbit shots the next morning. No false positives.
The final night, I strapped it to a piñon tree facing a little
clearing in the woods. Only two morning rabbits, but during the night
it caught a coyote. And only 5 false positives. I've never caught a
coyote (or anything else larger than a rabbit) with the PiCam.
So I'm not sure what to think. It's certainly a lot more relaxing to
go through the minimal output of the TrophyCam to see what I caught.
And it's certainly a lot easier to set up, and more waterproof, than
my jury-rigged milk carton setup with its two AC cords, one for the Pi
and one for the IR sensor. Being self-contained and battery operated
makes it easy to set up anywhere, not just near a power plug.
But it's made me rethink my pessimistic notion that I should give up
on this homemade PiCam setup and buy a commercial camera.
Even on its most sensitive setting, I can't make the TrophyCam
sensitive enough to catch small animals.
And the PiCam gets better picture quality than the Bushnell, not to
mention the option of hooking up a separate camera with flash.
So I guess I can't give up on the Pi setup yet. I just have to come up
with a sensible way of taming the false positives. I've been doing a lot
of experimenting with SimpleCV image processing, but alas, it's no better
at detecting actual critters than my simple pixel-counting script was.
But maybe I'll find the answer, one of these days. Meanwhile, I may
look into battery power.
[ 14:29 Sep 22, 2014
More hardware |
permalink to this entry |
Thu, 18 Sep 2014
A female hummingbird -- probably a black-chinned -- hanging out at
our window feeder on a cool cloudy morning.
[ 19:04 Sep 18, 2014
More nature/birds |
permalink to this entry |
Sun, 14 Sep 2014
Global key bindings in emacs. What's hard about that, right?
Just something simple like
(global-set-key "\C-m" 'newline-and-indent)
and you're all set.
global-set-key gives you a nice key binding
that works ... until the next time you load a mode that wants
to redefine that key binding out from under you.
For many years I've had a huge collection of mode hooks that run when
specific modes load. For instance, python-mode defines \C-c\C-r, my
binding that normally runs revert-buffer, to do something called run-python.
I never need to run python inside emacs -- I do that in a shell window.
But I fairly frequently want to revert a python file back to the last
version I saved. So I had a hook that ran whenever python-mode loaded
to override that key binding and set it back to what I'd already set
(defun reset-revert-buffer ()
(define-key python-mode-map "\C-c\C-r" 'revert-buffer) )
(setq python-mode-hook 'reset-revert-buffer)
That worked fine -- but you have to do it for every mode that
overrides key bindings and every binding that gets overridden.
It's a constant chase, where you keep needing to stop editing
whatever you wanted to edit and go add yet another mode-hook to
.emacs after chasing down which mode is causing the problem.
There must be a better solution.
A web search quickly led me to the StackOverflow discussion
override key bindings. I tried the techniques there; but they
It took a lot of help from the kind folks on #emacs, but after an hour
or so they finally found the key:
documented -- the key there is "The “active” keymaps in each alist
are used before minor-mode-map-alist and minor-mode-overriding-map-alist" --
and there seem to be no examples anywhere on the web for how to use it.
It's a list of alists mapping names to keymaps. Oh, clears it right up! Right?
Okay, here's what it means. First you define a new keymap and add your
bindings to it:
(defvar global-keys-minor-mode-map (make-sparse-keymap)
(define-key global-keys-minor-mode-map "\C-c\C-r" 'revert-buffer)
(define-key global-keys-minor-mode-map (kbd "C-;") 'insert-date)
Now define a minor mode that will use that keymap. You'll use that
minor mode for basically everything.
"A minor mode so that global key settings override annoying major modes."
t "global-keys" 'global-keys-minor-mode-map)
Now build an alist consisting of a list containing a single dotted
pair: the name of the minor mode and the keymap.
;; A keymap that's supposed to be consulted before the first
(defconst global-minor-mode-alist (list (cons 'global-keys-minor-mode
Finally, set emulation-mode-map-alists to a list containing only
(setf emulation-mode-map-alists '(global-minor-mode-alist))
There's one final step. Even though you want these bindings to be
global and work everywhere, there is one place where you might not
want them: the minibuffer. To be honest, I'm not sure if this part
is necessary, but it sounds like a good idea so I've kept it.
(defun my-minibuffer-setup-hook ()
(add-hook 'minibuffer-setup-hook 'my-minibuffer-setup-hook)
Whew! It's a lot of work, but it'll let me clean up my .emacs file and
save me from endlessly adding new mode-hooks.
[ 16:46 Sep 14, 2014
More linux/editors |
permalink to this entry |
Thu, 11 Sep 2014
I don't use web forums, the kind you have to read online, because they
don't scale. If you're only interested in one subject, then they work fine:
you can keep a browser tab for your one or two web forums perenially open
and hit reload every few hours to see what's new.
If you're interested in twelve subjects, each of which has several
different web forums devoted to it -- how could
you possibly keep up with that? So I don't bother with forums unless
they offer an email gateway, so they'll notify me by email when new
discussions get started, without my needing to check all those web
pages several times per day.
LinkedIn discussions mostly work like a web forum.
But for a while, they had a reasonably usable email gateway. You
could set a preference to be notified of each new conversation.
You still had to click on the web link to read the conversation so far,
but if you posted something, you'd get the rest of the discussion
emailed to you as each message was posted.
Not quite as good as a regular mailing list, but it worked pretty well.
I used it for several years to keep up with the very active Toastmasters
About a year ago, something broke in their software, and they lost
the ability to send email for new conversations. I filed a trouble
ticket, and got a note saying they were aware of the problem and
working on it. I followed up three months later (by filing another
ticket -- there's no way to add to an existing one) and got a response
saying be patient, they were still working on it. 11 months later,
I'm still being patient, but it's pretty clear they have no intention
of ever fixing the problem.
Just recently I fiddled with something in my LinkedIn prefs, and
started getting "Popular Discussions" emails every day or so.
The featured "popular discussion" is always something stupid that
I have no interest in, but it's followed by a section headed
"Other Popular Discussions" that at least gives me some idea what's
been posted in the last few days. Seemed like it might be worth
clicking on the links even though it means I'd always be a few days
late responding to any conversations.
Except -- none of the links work. They all go to a generic page with a red
header saying "Sorry it seems there was a problem with the link you followed."
I'm reading the plaintext version of the mail they send out. I tried
viewing the HTML part of the mail in a browser, and sure enough, those
links worked. So I tried comparing the text links with the HTML:
Well, that's clear as mud, isn't it?
HTML entity substitution
I pasted both links one on top of each other, to make it easier to
compare them one at a time. That made it fairly easy to find the first
Time to die laughing: they're doing HTML entity substitution on the
plaintext part of their email notifications, changing & to &
everywhere in the link.
If you take the link from the text email and replace & with &,
the link works, and takes you to the specific discussion.
Except you can't actually read the discussion. I went to a discussion
that had been open for 2 days and had 35 responses, and LinkedIn only
showed four of them. I don't even know which four they are -- are
they the first four, the last four, or some Facebook-style "four
responses we thought you'd like". There's a button to click on to
show the most recent entries, but then I only see a few of the most
recent responses, still not the whole thread.
Hooray for the web -- of course, plenty of other people have had this
problem too, and a little web searching unveiled a solution. Add a
pagination token to the end of the URL that tells LinkedIn to show
1000 messages at once.
It won't actually show 1000 (or all) responses -- but
if you start at the beginning of the page and scroll down reading
responses one by one, it will auto-load new batches.
Yes, infinite scrolling pages
can be annoying
, but at least it's a way to read a LinkedIn
conversation in order.
Making it automatic
Okay, now I know how to edit one of their URLs to make it work. Do I
want to do that by hand any time I want to view a discussion? Noooo!
Time for a script! Since I'll be selecting the URLs from mutt, they'll
be in the X PRIMARY clipboard. And unfortunately, mutt adds newlines so
I might as well strip those as well as fixing the LinkedIn problems.
(Firefox will strip newlines for me when I paste in a multi-line URL,
but why rely on that?)
Here's the important part of the script:
import subprocess, gtk
primary = gtk.clipboard_get(gtk.gdk.SELECTION_PRIMARY)
if not primary.wait_is_text_available() :
link = primary.wait_for_text()
link = link.replace("\n", "").replace("&", "&") + \
subprocess.call(["firefox", "-new-tab", link])
And here's the full script:
on GitHub. I also added it to
the script I call from Openbox to open a URL in Firefox
when I middle-click on the desktop.
Now I can finally go back to participating in those discussions.
[ 13:10 Sep 11, 2014
More tech/web |
permalink to this entry |
Sun, 07 Sep 2014
I read about cool computer tricks all the time. I think "Wow, that
would be a real timesaver!" And then a week later, when it actually
would save me time, I've long since forgotten all about it.
After yet another session where I wanted to open a frequently opened
file in emacs and thought "I think I made a
for that a while back", but then decided it's easier to type the whole
long pathname rather than go re-learn how to use emacs bookmarks,
I finally decided I needed a reminder system -- something that would
poke me and remind me of a few things I want to learn.
I used to keep cheat sheets and quick reference cards on my desk;
but that never worked for me. Quick reference cards tend to be
50 things I already know, 40 things I'll never care about and 4 really
great things I should try to remember. And eventually they get
burned in a pile of other papers on my desk and I never see them again.
My new system is working much better. I created a file in my home
directory called .reminders, in which I put a few -- just a few
-- things I want to learn and start using regularly. It started out
at about 6 lines but now it's grown to 12.
Then I put this in my .zlogin (of course, you can do this for any
shell, not just zsh, though the syntax may vary):
if [[ -f ~/.reminders ]]; then
Now, in every login shell (which for me is each new terminal window
I create on my desktop), I see my reminders. Of course, I don't read
them every time; but I look at them often enough that I can't forget
the existence of great things like
emacs bookmarks, or
diff <(cmd1) <(cmd2).
And if I forget the exact
keystroke or syntax, I can always
cat ~/.reminders to
remind myself. And after a few weeks of regular use, I finally have
internalized some of these tricks, and can remove them from my
It's not just for tech tips, either; I've used a similar technique
for reminding myself of hard-to-remember vocabulary words when I was
studying Spanish. It could work for anything you want to teach yourself.
Although the details of my .reminders are specific to Linux/Unix and zsh,
of course you could use a similar system on any computer. If you don't
open new terminal windows, you can set a reminder to pop up when you
first log in, or once a day, or whatever is right for you. The
important part is to have a small set of tips that you see regularly.
[ 21:10 Sep 07, 2014
More tech |
permalink to this entry |
Tue, 02 Sep 2014
I was using strace to figure out how to set up a program, lftp, and
a friend commented that he didn't know how to use it
and would like to learn. I don't use strace often, but when I do,
it's indispensible -- and it's easy to use. So here's a little tutorial.
My problem, in this case, was that I needed to find out what
configuration file I needed to modify in order to set up an alias
in lftp. The lftp man page tells you how to define an alias, but doesn't
tell you how to save it for future sessions; apparently you have
to edit the configuration file yourself.
But where? The man page suggested
a couple of possible config file locations -- ~/.lftprc and
~/.config/lftp/rc -- but neither of those existed. I wanted
to use the one that already existed. I had already set up bookmarks
in lftp and it remembered them, so it must have a config file already,
somewhere. I wanted to find that file and use it.
So the question was, what files does lftp read when it starts up?
strace lets you snoop on a program and see what it's doing.
strace shows you all system calls being used by a program.
What's a system call? Well, it's anything in section 2 of the Unix manual.
You can get a complete list by typing:
man 2 syscalls
(you may have to install developer man pages first -- on Debian that's
the manpages-dev package). But the important thing is that most
file access calls -- open, read, chmod, rename, unlink (that's how you
remove a file), and so on -- are system calls.
You can run a program under strace directly:
$ strace lftp sitename
Interrupt it with Ctrl-C when you've seen what you need to see.
Pruning the output
And of course, you'll see tons of crap you're not interested in,
like rt_sigaction(SIGTTOU) and fcntl64(0, F_GETFL). So let's get rid
of that first. The easiest way is to use grep. Let's say I want to know
every file that lftp opens. I can do it like this:
$ strace lftp sitename |& grep open
I have to use |& instead of just | because strace prints its
output on stderr instead of stdout.
That's pretty useful, but it's still too much. I really don't care
to know about strace opening a bazillion files in
/usr/share/locale/en_US/LC_MESSAGES, or libraries like
In this case, I'm looking for config files, so I really only want to know
which files it opens in my home directory. Like this:
$ strace lftp sitename |& grep 'open.*/home/akkana'
In other words, show me just the lines that have either the word "open"
or "read" followed later by the string "/home/akkana".
Digression: grep pipelines
Now, you might think that you could use a simpler pipeline with two greps:
$ strace lftp sitename |& grep open | grep /home/akkana
But that doesn't work -- nothing prints out. Why? Because grep, under
certain circumstances that aren't clear to me, buffers its output, so
in some cases when you pipe grep | grep, the second grep will wait
until it has collected quite a lot of output before it prints anything.
(This comes up a lot with
tail -f as well.)
You can avoid that with
$ strace lftp sitename |& grep --line-buffered open | grep /home/akkana
but that's too much to type, if you ask me.
Back to that strace | grep
Okay, whichever way you grep for open and your home directory,
open("/home/akkana/.local/share/lftp/bookmarks", O_RDONLY|O_LARGEFILE) = 5
open("/home/akkana/.netrc", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
open("/home/akkana/.local/share/lftp/rl_history", O_RDONLY|O_LARGEFILE) = 5
open("/home/akkana/.inputrc", O_RDONLY|O_LARGEFILE) = 5
Now we're getting somewhere! The file where it's getting its bookmarks
-- and I probably can't use that
to set my alias.
But wait, why doesn't it show lftp trying to open those other config files?
Using script to save the output
At this point, you might be sick of running those grep pipelines over
and over. Most of the time, when I run strace, instead of piping it
through grep I run it under script to save the whole output.
script is one of those poorly named, ungoogleable commands, but it's
incredibly useful. It runs a subshell and saves everything that appears
in that subshell, both what you type and all the output, in a file.
Start script, then run lftp inside it:
$ script /tmp/lftp.strace
Script started on Tue 26 Aug 2014 12:58:30 PM MDT
$ strace lftp sitename
After the flood of output stops, I type Ctrl-D or Ctrl-C to exit lftp,
then another Ctrl-D to exit the subshell script is using.
Now all the strace output was in /tmp/lftp.strace and I can
grep in it, view it in an editor or anything I want.
So, what files is it looking for in my home directory and why don't
they show up as open attemps?
$ grep /home/akkana /tmp/lftp.strace
Ah, there it is! A bunch of lines like this:
access("/home/akkana/.lftprc", R_OK) = -1 ENOENT (No such file or directory)
stat64("/home/akkana/.lftp", 0xbff821a0) = -1 ENOENT (No such file or directory)
mkdir("/home/akkana/.config", 0755) = -1 EEXIST (File exists)
mkdir("/home/akkana/.config/lftp", 0755) = -1 EEXIST (File exists)
access("/home/akkana/.config/lftp/rc", R_OK) = 0
So I should have looked for access and stat as well as
Now I have the list of files it's looking for. And, curiously,
it creates ~/.config/lftp if it doesn't exist already, even though
it's not going to write anything there.
So I created ~/.config/lftp/rc and put my alias there. Worked fine.
And I was able to edit my bookmark in ~/.local/share/lftp/bookmarks
later when I had a need for that. All thanks to strace.
[ 13:06 Sep 02, 2014
More linux/cmdline |
permalink to this entry |