Shallow Thoughts

Akkana's Musings on Open Source Computing and Technology, Science, and Nature.

Sun, 23 Jul 2017

Nambé Lake Nutcrackers

[Nambe Lake]

This week's hike was to Nambé Lake, high in the Sangre de Cristos above Santa Fe.

It's a gorgeous spot, a clear, shallow mountain lake surrounded by steep rocky slopes up to Lake Peak and Santa Fe Baldy. I assume it's a glacial cirque, though I can't seem to find any confirmation of that online.

[Clark's nutcracker taking bread from my hand.] There's a raucous local population of Clark's nutcrackers, a grey relative of the jays (but different from the grey jay) renowned for its fearlessness and curiosity. One of my hiking companions suggested they'd take food from my hand if I offered. I broke off a bit of my sandwich and offered it, and sure enough, a nutcracker flew right over. Eventually we had three or four of them hanging around our lunch spot.

The rocky slopes are home to pikas, but they're shy and seldom seen. We did see a couple of marmots in the rocks, and I caught a brief glimpse of a small, squirrel-sized head that looked more grey than brown like I'd expect from a rock squirrel. Was it a pika? I'll never know.

We also saw some great flowers. Photos: Nambé Lake Nutcrackers.

Tags: , ,
[ 09:55 Jul 23, 2017    More | permalink to this entry | comments ]

Sun, 16 Jul 2017

Translating Markdown to LibreOffice or Word

For the Raspberry Pi Zero W book I'm writing, the publisher, Maker Media, wants submissions in Word format (but stressed that LibreOffice was fine and lots of people there use it, a nice difference from Apress). That's fine ... but when I'm actually writing, I want to be able to work in emacs; I don't want to be distracted fighting with LibreOffice while trying to write.

For the GIMP book, I wrote in plaintext first, and formatted it later. But that means the formatting step took a long time and needed exceptionally thorough proofreading. This time, I decided to experiment with Markdown, so I could add emphasis, section headings, lists and images all without leaving my text editor.

Of course, it would be nice to be able to preview what the formatted version will look like, and that turned out to be easy with a markdown editor called ReText, which has a lovely preview mode, as long as you enable Edit->Use WebKit renderer (I'm not sure why that isn't the default).

Okay, a chapter is written and proofread. The big question: how to get it into the Word format the publisher wants?

First thought: ReText has a File->Export menu. Woohoo -- it offers ODT. So I should be able to export to ODT then open the resulting file in LibreOffice.

Not so much. The resulting LibreOffice document is a mess, with formatting that doesn't look much like the original, and images that are all sorts of random sizes. I started going through it, resizing all the images and fixing the formatting, then realized what a big job it was going to be and decided to investigate other options first.

ReText's Export menu also offers HTML, and the HTML it produces looks quite nice in Firefox. Surely I could open that in LibreOffice, then save it (maybe with a little minor reformatting) as DOCX?

Well, no, at least not directly. It turns out LibreOffice has no obvious way to import an HTML file into a normal text document. If you Open the HTML file, it displays okay (except the images are all tiny thumbnails and need to be resized one by one); but LibreOffice can't save it in any format besides HTML or plaintext. Those are the only formats available in the menu in the Save dialog. LibreOffice also has a Document Converter, but it only converts Office formats, not HTML; and there's no Import... in LO's File. There's a Wizards->Web Page, but it's geared to creating a new web page and saving as HTML, not importing an existing HTML-formatted document.

But eventually I discovered that if I "Create a new Text Document" in LibreOffice, I can Select All and Copy in Firefox, followed by Paste into Libre Office. It works great. All the images are the correct size, the formatting is correct needing almost no corrections, and LibreOffice can save it as DOCX, ODT or whatever I need.

Image Captions

I mentioned that the document needed almost no corrections. The exception is captions. Images in a book need captions and figure numbers, unlike images in HTML.

Markdown specifies images as

![Image description][path/to/image.jpg)

Unfortunately, the Image description part is only visible as a mouseover, which only works if you're exporting to a format intended for a web browser that runs on desktop and laptop computers. It's no help in making a visible caption for print, or for tablets or phones that don't have mouseover. And the mouseover text disappears completely when you paste the document from Firefox into LibreOffice.

I also tried making a table with the image above and the caption underneath. But I found it looked just as good in ReText, and much better in HTML, just to add a new paragraph of italics below the image:

![][path/to/image.jpg)

*Image description here*

That looks pretty nice in a browser or when pasted into LibreOffice. But before submitting a chapter, I changed them into real LibreOffice captions.

In LibreOffice, right-click on the image; Add Caption is in the context menu. It can even add numbers automatically. It initially wants to call every caption "Illustration" (e.g. "Illustration 1", "Illustration 2" and so on), and strangely, "Figure" isn't one of the available alternatives; but you can edit the category and change it to Figure, and that persists for the rest of the document, helpfully numbering all your figures in order. The caption dialog when you add each caption always says that the caption will be "Illustration 1: (whatever you typed)" even if it's the fourteenth image you've captioned; but when you dismiss the dialog it shows up correctly as Figure 14, not as a fourteenth Figure 1.

The only problem arises if you have to insert a new image in the middle of a chapter. If you do that, you end up with two Figure 6 (or whatever the number is) and it's not clear how to persuade LibreOffice to start over with its renumbering. You can fix it if you remove all the captions and start over, but ugh. I never found a better way, and web searches on LibreOffice caption numbers suggest this is a perennial source of frustration with LibreOffice.

The bright side: struggling with captions in LibreOffice convinced me that I made the right choice to do most of my work in emacs and markdown!

Tags: ,
[ 14:12 Jul 16, 2017    More writing | permalink to this entry | comments ]

Thu, 06 Jul 2017

Writing a Book on the Raspberry Pi Zero W

It's official: I'm working on another book!

This one will be much shorter than Beginning GIMP. It's a mini-book for Make Media on the Raspberry Pi Zero W and some fun projects you can build with it. [Raspberry Pi Zero W]

I don't want to give too much away at this early stage, but I predict it will include light shows, temperature sensors, control of household devices, Twitter access and web scraping. And lots of code samples.

I'll be posting more about the book, and about various Raspberry Pi Zero W projects I'm exploring during the course of writing it. But for now ... if you'll excuse me, I have a chapter that's due today, and a string of addressable LEDs sitting on my desk calling out to be played with as part of the next chapter.

Tags: , ,
[ 09:50 Jul 06, 2017    More writing | permalink to this entry | comments ]

Sat, 24 Jun 2017

Mutt: Fixing Erroneous Charsets, part 632

Someone forwarded me a message from the Albuquerque Journal. It was all about "New Mexico\222s schools".

Sigh. I thought I'd gotten all my Mutt charset problems fixed long ago. My system locale is set to en_US.UTF-8, and accented characters in Spanish and in people's names usually show up correctly. But I do see this every now and then.

When I see it, I usually assume it's a case of incorrect encoding: whoever sent it perhaps pasted characters from a Windows Word document or something, and their mailer didn't properly re-encode them into the charset they were using to send the message.

In this case, the message had User-Agent: SquirrelMail/1.4.13. I suspect it came from a "Share this" link on the newspaper's website.

I used vim to look at the source of the message, and it had

Content-Type: text/plain; charset=iso-8859-1
For the bad characters, in vim I saw things like
New Mexico<92>s schools

I checked an old web page I'd bookmarked years ago that had a table of the iso-8859-1 characters, and sure enough, hex 0x92 was an apostrophe. What was wrong?

I got some help on the #mutt IRC channel, and, to make a long story short, that web table I was using was wrong. ISO-8859-1 doesn't include any characters in the range 8x-9x, as you can see on the Wikipedia ISO/IEC 8859-1.

What was happening was that the page was really cp1252: that's where those extra characters, like hex 92/octal 222 for an apostrophe, or hex 96/octal 226 for a dash (nitpick: that's an en dash, but it was used in a context that called for an em dash; if someone is going to use something other than the plain old ASCII dash - you'd think they'd at least use the right one. Sheesh!)

Anyway, the fix for this is to tell mutt when it sees iso-8859-1, use cp1252 instead:

charset-hook iso-8859-1 cp1252

Voilà! Now I could read the article about New Mexico's schools.

A happy find related to this: it turns out there's a better way of looking up ISO-8859 tables, and I can ditch that bookmark to the old, erroneous page. I've known about man ascii forever, but someone I'd never thought to try other charsets. Turns out man iso_8859-1 and man iso_8859-15 have built-in tables too. Nice!

(Sadly, man utf-8 doesn't give a table. Of course, that would be a long man page, if it did!)

Tags: , ,
[ 11:06 Jun 24, 2017    More linux | permalink to this entry | comments ]

Fri, 16 Jun 2017

Flycatchers Fledging, and The Buck Stops Here

We've had a pair of ash-throated flycatchers in the nest box I set up in the yard. I've been watching them bring bugs to the nest for a couple of weeks now, but this morning they've been acting unusual: fluttering around the corner of the house near my office window, calling to each other, not spending nearly as much time near the nest. I suspect one or more of the chicks may have fledged this morning, though I have yet to see more than two flycatchers at once. They still return to the nest box occasionally (one of them just delivered a big grasshopper), so not all the chicks have fledged yet. Maybe if I'm lucky I'll get to see one fledge.

I hope they're not too affected by the smoky air. We have two fires filling the air with smoke: the Bonita Fire, 50 miles north, and as of yesterday a new fire in Jemez Springs, only about half that distance. Yesterday my eyes were burning, my allergies were flaring up, and the sky was worse than the worst days in Los Angeles in the 70s. But it looks like the firefighters have gotten a handle on both fires; today is still smoky, with a major haze down in the Pojoaque Valley and over toward Albuquerque, but the sky above is blue and the smoke plume from Jemez Springs is a lot smaller and less dark than it was yesterday. Fingers crossed!

[Buck in velvet, drinking at the pond] And just a few minutes ago, a buck with antlers in velvet wandered into our garden to take a drink at the pond. Such a nice change from San Jose!

Tags: ,
[ 10:40 Jun 16, 2017    More nature | permalink to this entry | comments ]

Fri, 09 Jun 2017

Emacs: Typing dashes in html mode (update for Emacs 24)

Back in 2006, I wrote an article on making a modified copy of sgml-mode.el to make it possible to use double-dashed clauses -- like this -- in HTML without messing up auto-fill mode.

That worked, but the problem is that if you use your own copy of sgml-mode.el, you miss out on any other improvements to HTML and SGML mode. There have been some good ones, like smarter rewrap of paragraphs. I had previously tried lots of ways of customizing sgml-mode without actually replacing it, but never found a way.

Now, in emacs 24.5.1, I've found a easier way that seems to work. The annoying mis-indentation comes from the function sgml-comment-indent-new-line, which sets variables comment-start, comment-start-skip and comment-end and then calls comment-indent-new-line.

All I had to do was redefine sgml-comment-indent-new-line to call comment-indent-new-line without first defining the comment characters:

(defun sgml-comment-indent-new-line (&optional soft)
  (comment-indent-new-line soft))

Finding emacs source

I wondered if it might be better to call whatever underlying indent-new-line function comment-indent-new-line calls, or maybe just to call (newline-and-indent). But how to find the code of comment-indent-new-line?

Happily, describe-function (on C-h f, or if like me you use C-h for backspace, try F-1 h) tells you exactly what file defines a function, and it even gives you a link to click on to view the source. Wonderful!

It turned out just calling (newline-and-indent) wasn't enough, because sgml-comment-indent-new-line typically calls comment-indent-new-line when you've typed a space on the end of a line, and that space gets wrapped and then messes up indentation. But you can fix that by copying just a couple of lines from the source of comment-indent-new-line:

(defun sgml-comment-indent-new-line (&optional soft)
  (save-excursion (forward-char -1) (delete-horizontal-space))
  (delete-horizontal-space)
  (newline-and-indent))

That's a little longer than the other definition, but it's cleaner since comment-indent-new-line is doing all sorts of extra work you don't need if you're not handling comments. I'm not sure that both of the delete-horizontal-space lines are needed: the documentation for delete-horizontal-space says it deletes both forward and backward. But I have to assume they had a good reason for having both: maybe the (forward-char -1) is to guard against spurious spaces already having been inserted in the next line. I'm keeping it, to be safe.

Tags: ,
[ 11:16 Jun 09, 2017    More linux/editors | permalink to this entry | comments ]

Mon, 05 Jun 2017

HTML Email from Mutt

I know, I know. We use mailers like mutt because we don't believe in HTML mail and prefer plaintext. Me, too.

But every now and then a situation comes up where it would be useful to send something with emphasis. Or maybe you need to highlight changes in something. For whatever reason, every now and then I wish I had a way to send HTML mail.

I struggled with that way back, never did find a way, and ended up writing a Python script, htmlmail.py to send an HTML page, including images, as email.

Sending HTML Email

But just recently I found a neat mutt hack. It turns out it's quite easy to send HTML mail.

First, edit the HTML source in your usual mutt message editor (or compose the HTML some other way, and insert the file). Note: if there's any quoted text, you'll have to put a <pre> around it, or otherwise turn it into something that will display nicely in HTML.

Write the file and exit the editor. Then, in the Compose menu, type Ctrl-T to edit the attachment type. Change the type from text/plain to text/html.

That's it! Send it, and it will arrive looking like a regular HTML email, just as if you'd used one of them newfangled gooey mail clients. (No inline images, though.)

Viewing HTML Email

Finding out how easy that was made me wonder why the other direction isn't easier. Of course, I have my mailcap set up so that mutt uses lynx automatically to view HTML email:

text/html; lynx -dump %s; nametemplate=%s.html; copiousoutput

Lynx handles things like paragraph breaks and does in okay job of showing links; but it completely drops all emphasis, like bold, italic, headers, and colors. My terminal can display all those styles just fine. I've also tried links, elinks, and w3m, but none of them seem to be able to handle any text styling. Some of them will do bold if you run them interactively, but none of them do italic or colors, and none of them will do bold with -dump, even if you tell them what terminal type you want to use. Why is that so hard?

I never did find a solution, but it's worth noting some useful sites I found along the way. Like tips for testing bold, italics etc. in a terminal:, and for testing whether the terminal supports italics, which gave me these useful shell functions:

echo -e "\e[1mbold\e[0m"
echo -e "\e[3mitalic\e[0m"
echo -e "\e[4munderline\e[0m"
echo -e "\e[9mstrikethrough\e[0m"
echo -e "\e[31mHello World\e[0m"
echo -e "\x1B[31mHello World\e[0m"

ansi()          { echo -e "\e[${1}m${*:2}\e[0m"; }
bold()          { ansi 1 "$@"; }
italic()        { ansi 3 "$@"; }
underline()     { ansi 4 "$@"; }
strikethrough() { ansi 9 "$@"; }
red()           { ansi 31 "$@"; }

And in testing, I found that a lot of fonts didn't offer italics. One that does is Terminus, so if your normal font doesn't, you can run a terminal with Terminus: xterm -fn '-*-terminus-bold-*-*-*-20-*-*-*-*-*-*-*'

Not that it matters since none of the text-mode browsers offer italic anyway. But maybe you'll find some other use for italic in a terminal.

Tags: , ,
[ 18:28 Jun 05, 2017    More linux | permalink to this entry | comments ]

Wed, 31 May 2017

Tiger Salamander Larvae

[Tiger salamander with gills] [Tiger salamander with gills] I got a tip that there were tiger salamanders with gills swimming around below Los Alamos reservoir, so I had to go see for myself.

They're fabulous! Four to five inch salamanders with flattened tails and huge frilly gills behind their heads -- dozens of them, so many the pond is thick with them. Plenty of them are hanging out in the shallows or just below the surface of the water, obligingly posing for photos.

I had stupidly brought only the pocket camera, not the DSLR -- and then the camera's battery turned out to be low -- so I was sparing with camera, but even so I was pleased at how well they came out, with the camera mostly managing to focus on the salamanders rather than (as I had feared) the surface of the murky water. I may go back soon with the DSLR. It's an easy, pleasant hike.

Photos: Tiger Salamander larvae.

Tags: ,
[ 20:31 May 31, 2017    More nature | permalink to this entry | comments ]