Shallow Thoughts

Akkana's Musings on Open Source Computing and Technology, Science, and Nature.

Tue, 14 Jan 2020

Plotting War

A recent article on Pharyngula blog, You ain’t no fortunate one, discussed US wars, specifically the qeustion: depending on when you were born, for how much of your life has the US been at war?

It was an interesting bunch of plots, constantly increasing until for people born after 2001, the percentage hit 100%.

Really? That didn't seem right. Wasn't the US in a lot of wars in the past? When I was growing up, it seemed like we were always getting into wars, poking our nose into other countries' business. Can it really be true that we're so much more warlike now than we used to be?

It made me want to see a plot of when the wars were, beyond Pharyngula's percentage-of-life pie charts. So I went looking for data.

The best source of war dates I could find was American Involvement in Wars from Colonial Times to the Present. I pasted that data into a table and reformatted it to turn it into Python data, and used matplotlib to plot it as a Gantt chart. (Script here: us-wars.py.)

[US Wars Since 1900]

Sure enough. If that Thoughtco page with the war dates is even close to accurate -- it could be biased toward listing recent conflicts, but I didn't find a more authoritative source for war dates -- the prevalence of war took a major jump in 2001. We used to have big gaps between wars, and except for Vietnam, the wars we were involved with were short, mostly less than a year each. But starting in 2001, we've been involved in a never-ending series of overlapping wars unprecedented in US history.

The Thoughtco page had wars going back to 1675, so I also made a plot showing all of them (click for the full-sized version). It's no different: short wars, not overlapping, all the way back to before the revolution. We've seen nothing in the past like the current warmongering. [US Wars Since 1675]

Depressing. Climate change isn't the only phenomenon showing a modern "hockey stick" curve, it seems.

Tags: , , ,
[ 12:25 Jan 14, 2020    More politics | permalink to this entry | comments ]

Thu, 09 Jan 2020

Updating a Persistent Window from Javascript Part 2: A Clever Hack

I wrote about various ways of managing a persistent popup window from Javascript, eventually settling on a postMessage() solution that turned out not to work in QtWebEngine. So I needed another solution.

Data URI

First I tried using a data: URI. In that scheme, you encode a page's full content into the URL. For instance: try this in your browser: data:text/html,Hello%2C%20World!

So for a longer page, you can do something like:

    var htmlhead = '<html>\n'
        + '<head>\n'
        + '<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">\n'
        + '<link rel="stylesheet" type="text/css" href="stylesheet.css">\n'
        + '</head>\n'
        + '\n'
        + '<body>\n'
        + '<div id="mydiv">\n';
     var htmltail = '</div>\n'
        + '</body>\n'
        + '</html>\n';

    var encodedDataURI = encodeURI(htmlhead + noteText + htmltail);

    var notewin = window.open('data:text/html,' + encodedDataURI, "notewindow",
                              "width=800,height=500");

Nice and easy -- and it even works from file: URIs!

Well, sort of works. It turns out it has a problem related to the same-origin problems I saw with postMessage. A data: URI is always opened with an origin of about:blank; and two about:blank origin pages can't talk to each other.

But I don't need them to talk to each other if I'm not using postMessage, do I? Yes, I do. The problem is that stylesheet I included in htmlhead above:

<link rel="stylesheet" type="text/css" href="stylesheet.css">\n'
All browsers I tested refuse to open the stylesheet in the about:blank popup. This seems strange: don't people use stylesheets from other domains fairly often? Maybe it's a behavior special to null (about:blank) origin pages. But in any case, I couldn't find a way to get my data: URI popup to load a stylesheet. So unless I hard-code all the styles I want for the notes page into the Javascript that opens the popup window (and I'd really rather not do that), I can't use data: as a solution.

Clever hack: Use the Same Page, Loaded in a Different Way

That's when I finally came across Remy Sharp's page, Creating popups without HTML files. Remy first explores the data: URI solution, and rejects it because of the cross-origin problem, just as I did. But then he comes up with a clever hack. It's ugly, as he acknowledges ... but it works.

The trick is to create the popup with the URL of the parent page that created it, but with a named anchor appended: parentPage.html#popup. Then, in the Javascript, check whether #popup is in the URL. If not, we're in the parent page and still need to call window.open to create the popup. If it is there, then the JS code is being executed in the popup. In that case, rewrite the page as needed. In my case, since I want the popup to show only whatever is in the div named #notes, and the slide content is all inside a div called #page, I can do this:

function updateNoteWindow() {
    if (window.location.hash.indexOf('#notes') === -1) {
        window.open(window.location + '#notes', 'noteWin',
                    'width=300,height=300');
        return;
    }

    // If here, it's the popup notes window.
    // Remove the #page div
    var pageDiv = document.getElementById("page");
    pageDiv.remove();

    // and rename the #notes div so it will be displayed in a different place
    var notesDiv = document.getElementById("notes");
    notesDiv.id = "fullnotes";
}

It works great, even in file: URIs, and even in QtWebEngine. That's the solution I ended up using.

Tags: , ,
[ 19:44 Jan 09, 2020    More tech/web | permalink to this entry | comments ]

Sun, 05 Jan 2020

Updating a Persistent Window from Javascript Part 1: postMessage

I'm trying to update my htmlpreso HTML presentation slide system to allow for a separate notes window.

Up to now, I've just used display mirroring. I connect to the projector at 1024x768, and whatever is on the first (topmost/leftmost) 1024x768 pixels of my laptop screen shows on the projector. Since my laptop screen is wider than 1024 pixels, I can put notes to myself to the right of the slide, and I'll see them but the audience won't.

That works fine, but I'd like to be able to make the screens completely separate, so I can fiddle around with other things while still displaying a slide on the projector. But since my slides are in HTML, and I still want my presenter notes, that requires putting the notes in a separate window, instead of just to the right of each slide.

The notes for each slide are in a <div id="notes"> on each page. So all I have to do is pop up another browser window and mirror whatever is in that div to the new window, right? Sure ... except this is JavaScript, so nothing is simple. Every little thing is going to be multiple days of hair-tearing frustration, and this was no exception.

I should warn you up front that I eventually found a much simpler way of doing this. I'm documenting this method anyway because it seems useful to be able to communicate between two windows, but if you just want a simple solution for the "pop up notes in another window" problem, stay tuned for Part 2.

Step 0: Give Up On file:

Normally I use file: URLs for presentations. There's no need to run a web server, and in fact, on my lightweight netbook I usually don't start apache2 by default, only if I'm actually working on web development.

But most of the methods of communicating between windows don't work in file URLs, because of the "same-origin policy". That policy is a good security measure: it ensures that a page from innocent-url.com can't start popping up windows with content from evilp0wnU.com without you knowing about it. I'm good with that. The problem is that file: URLs have location.origin of null, and every null-origin window is considered to be a different origin -- even if they're both from the same directory. That makes no sense to me, but there seems to be no way around it. So if I want notes in a separate window, I have to run a web server and use http://localhost.

Step 1: A Separate Window

The first step is to pop up the separate notes window, or get a handle to it if it's already up.

JavaScript offers window.open(), but there's a trick: if you just call notewin = window.open("notewin.html", "notewindow") you'll actually get a new tab, not a new window. If you actually want a window, the secret code for that is to give it a size:

  notewin = window.open("notewin.html", "notewindow",
                        "width=800,height=500");

There's apparently no way to just get a handle to an existing window. The only way is to call window.open(), pop up a new window if it wasn't there before, or reloads it if it's already there.

I saw some articles implying that passing an empty string "" as the first argument would return a handle to an existing window without changing it, but it's not true: in Firefox and Chromium, at least, that makes the existing window load about:blank instead of whatever page it already has. So just give it the same page every time.

Step 2: Figure Out When the Window Has Loaded

There are several ways to change the content in the popup window from the parent, but they all have one problem: if you update the content right away after calling window.open, whatever content you put there will be overwritten immediately when the popup reloads its notewin.html page (or even about:blank). So you need to wait until the popup is finished loading.

That sounds suspiciously easy. Assuming you have a function called updateNoteWinContent(), just do this:

// XXX This Doesn't work:
notewin.addEventListener('load', updateNoteWinContent, false);

Except it turns out the "load" event listener isn't called on reloads, at least not in popups. So this will work the first time, when the note window first pops up, but never after that.

I tried other listeners, like "DOMContentLoaded" and "readystatechange", but none of them are called on reload. Why not? Who knows? It's possible this is because the listener gets set too early, and then is wiped out when the page reloads, but that's just idle speculation.

For a while, I thought I was going to have to resort to an ugly hack: sleep for several seconds in the parent window to give the popup time to load: await new Promise(r => setTimeout(r, 3000)); (requires declaring the calling function as async). This works, but ... ick. Fortunately, there's a better way.

Step 2.5: Simulate onLoad with postMessage

What finally worked was a tricky way to use postMessage() in reverse. I'd already experimented with using postMessage() from the parent window to the popup, but it didn't work because the popup was still loading and wasn't ready for the content.

What works is to go the other way. In the code loaded by the popup (notewin.html in this example), put some code at the end of the page that calls

window.opener.postMessage("Loaded");

Then in the parent, handle that message, and don't try to update the popup's content until you've gotten the message:

function receiveMessageFromPopup(event) {
    console.log("Parent received a message from the notewin:", event.data);
    // Optionally, check whether event.data == "Loaded"
    // if you want to support more than one possible message.

    // Update the "notes" div in the popup notewin:
    var noteDiv = notewin.document.getElementById("notes");
    noteDiv.innerHTML = "

Here is some content.

"; } window.addEventListener("message", receiveMessageFromPopup, false);

Here's a complete working test: Test of Persistent Popup Window.

In the end, though, this didn't solve my presentation problem. I got it all debugged and working, only to discover that postMessage doesn't work in QtWebEngine, so I couldn't use it in my slide presentation app. Fortunately, I found a couple of other ways: stay tuned for Part 2.

(Update: Part 2: A Clever Hack.)

Debugging Multiple Windows: Separate Consoles

A note on debugging: One thing that slowed me down was that JS I put in the popup didn't seem to be running: I never saw its console.log() messages. It took me a while to realize that each window has its own web console, both in Firefox and Chromium. So you have to wait until the popup has opened before you can see any debugging messages for it. Even then, the popup window doesn't have a menu, and its context menu doesn't offer a console window option. But it does offer Inspect element, which brings up a Developer Tools window where you can click on the Console tab to see errors and debugging messages.

Tags: , ,
[ 20:29 Jan 05, 2020    More tech/web | permalink to this entry | comments ]

Thu, 02 Jan 2020

Horseshoe Crabs in New Mexico

[Horseshoe crab tracks in snow] On a recent hike to Escobas Mesa, I happened upon these tracks.

"Look! Trilobite tracks!" I exclaimed. But upon examining them more closely, I saw I was wrong. They look a little like trilobites, but they're clearly the tracks of a horseshoe crab.

Either way, quite a rare find in the snowy mountains of New Mexico.

Tags: ,
[ 19:23 Jan 02, 2020    More nature | permalink to this entry | comments ]

Tue, 10 Dec 2019

Planetarium Show Friday: Hitchhiker's Guide to the Moon

[Schroter's Valley on the Moon] This Friday, Dave and I will be presenting a planetarium show called The Hitchhiker's Guide to the Moon: Visit the Moon Without Leaving Your Home Planet.

I'm jazzed about this show. I think it'll be the most fun planetarium show we've given so far. We'll be showing a variety of lunarfeatures: maria, craters, mountains, rilles, domes, catenae and more. For each one, we'll discuss what the feature actually is and how it was created, where to see good examples on the moon, and -- the important part -- where you can go on Earth, and specifically in the Western US, to see a similar feature up close.

Plus: a short flyover of some of the major features using the full-dome planetarium. Some features, like Tycho, the Straight Wall, Reiner Gamma, plus lots of rilles, look really great in the planetarium.

If you can't get to the moon yourself, this is the next best thing!

The Hitchhiker's Guide to the Moon: 7pm at the PEEC nature center. Admission is free. Come find out how to explore the moon without leaving your home planet!

Tags: , ,
[ 18:06 Dec 10, 2019    More science/astro | permalink to this entry | comments ]

Fri, 06 Dec 2019

Bluebird Houses in Winter

[bluebird peering out from birdhouse] Last week, a flock of western bluebirds suddenly became fascinated with my two bluebird houses.

First I noticed a bluebird clinging to the outside of the downhill bluebird house. He would poke his head in the hole briefly, a couple of times, flutter to the top of the house, flutter back down to cling outside the hole and stick his head in. He never actually went in, and eventually lost interest and flew away.

Then a few minutes later, there were several bluebirds fluttering around the birdhouse that's outside the upstairs bedroom. I counted at least five individuals; I think they were all males. (The photos here are of a different, mixed-gender flock.) They were taking turns perching on top of the birdhouse, clinging to the outside and poking their heads in the hole. They attracted a junco, a robin and a flicker who apparently came to see what was so interesting; eventually the big flicker was apparently too intimidating, though she wasn't doing anything threatening, and all the bluebirds departed.

Neither of my birdhouses has ever had a bluebird breeding in it; they've had ash-throated flycatchers and a juniper titmouse during breeding season. Neither of them has been cleaned out since the last breeding season; I've been meaning to do that but haven't gotten around to it yet.

[bluebirds hanging out at the water dish] Are they looking for a place to shelter in cold weather? Or scouting out sites to have an advantage in next year's breeding season? Should I hurry to clean them out so they'll look more appealing during the winter? I posted to the local birders' list, but nobody seemed to know.

I'd love to have more bluebirds around; they usually only visit briefly to bathe and drink. Alas, they haven't been back, but I put the heated birdback out a few days ago and it should be popular once the days get colder.

Tags: ,
[ 20:39 Dec 06, 2019    More nature/birds | permalink to this entry | comments ]

Sat, 30 Nov 2019

Installing Lenovo Firmware Packaged as a .exe on a Linux Machine

My new Lenovo Carbon X1 Gen 7 has one irritating problem: the trackpad sometimes disappears, flooding dmesg with messages like "i2c_designware i2c_designware.1: controller timed out". Once this happens, the only fix is to reboot.

Lenovo has a fix -- new trackpad firmware -- but unlike their BIOS updates, which are installable from Linux, device firmware updates are distributed as Windows EXE files that require running Windows on the bare metal, leaving Linux users out in the cold. Ironic, since Lenovo is so popular among Linux users and is a member of the Linux Firmware Service, and the CX1 is supposedly Ubuntu certified.

Those Linux users on the forums who managed to install the firmware update raved about it, saying that indeed it solved their problem. But finding a way to to install it led me on a not-so-merry four-day quest.

Here's how I installed the firmware, in the end:

Make a Windows to Go using Rufus on a Real Windows Box

  1. Back up anything you don't want to lose, because you never know.

  2. Borrow a real Windows box. I tried many times using Windows inside VirtualBox and QEmu on top of Linux, but it never worked.
  3. On Windows, install Rufus.

  4. Download the Windows 10 Installer ISO (5 gigabytes, give or take)

  5. Find a USB stick or SD card, 16G or larger. Actually, find a bunch of them: this process is incredibly finicky about the stick you use and the only way you find out is that it doesn't work and you have to try again (see below).

  6. Use Rufus to create a Windows to Go image. The alternative is to make a Windows installer; that won't work, because you can't run anything useful from the installer, and you don't want to actually install Windows, or you wouldn't be in this fix in the first place.

    Be patient: creating a W2G image takes several hours. Click on Rufus' log file button (it's the rightmost of four obscure icons down near the lower left of the Rufus window; it has a mouseover tooltip) at any time to see what's happening; if things don't go right you might at least get some idea why.

  7. When the W2G stick is finished (whew!), move it to your Linux machine and mount its second partition (/dev/sdb2 or whatever). This will tell you it wasn't properly unmounted and it's fixing it, giving you a heart attack about whether Linux is going to change the filesystem in some way that makes it fail after you waited all that time creating it.

  8. Copy the firmware .exe to it. Wherever you want; I just put it at the root of the filesystem. Sync and unmount it.

  9. Boot your computer from the USB stick. This will take forever and may fail if the phase of the moon is wrong.

    If you're lucky and the planets are in alignment, eventually a Windows installer will come up and ask you a bunch of annoying questions about language, keyboard, whether you consent to having Microsoft spy on you in a skillion different ways, name, password, three security questions, etc. Meanwhile, you're having another heart attack because does this mean it's going to install Windows to your real disk on top of Linux? Hopefully not -- at least it didn't in my case -- but here's where you really want to have that recent backup.

  10. If you make it all the way through the questions and get a Windows screen, rejoice! Navigate to wherever you put your exe and run it. Cross your fingers -- maybe you're done!

  11. If it hangs or bluescreens during boot, or Rufus fails to create the W2G stick in the first place, try running Rufus again with a different USB stick. I think I tried five before finally finding one that worked, and the successful one (a Transcend SD card in an old Patriot USB adapter) wasn't the newest, or the fastest, or the largest. It's a mystery.

Some Approaches that Didn't Work

Before I finally got this working, I wasted four days trying many other approaches. Many of them sound very clever and reasonable and ought to work, but they didn't work for me. These include:

So, lots of different ways. Some of them have worked at some time for someone. Also, I never did try Wine. I don't think Wine would be able to run the actual exe and update the trackpad firmware (I was afraid to try it), but it's possible that Rufus in Wine might have been able to make a Windows To Go stick.

If anyone manages that -- or any other way of getting this to work -- I'd love to hear about it.

Tags: ,
[ 20:16 Nov 30, 2019    More linux | permalink to this entry | comments ]

Sun, 24 Nov 2019

Adjusting PulseAudio Volume from the Command-Line (or a Window Manager)

I have a new laptop, a birthday present to myself last month. For once, rather than buying a cut-rate netbook, I decided to treat myself to a fancy Lenovo Carbon X1 with an up-to-date processor and lots of RAM.

Since I have way more resources than I'm used to, I decided I'd try installing a full Ubuntu and not trying to pare it down to a super lightweight system. I'm still running the lightweight, fast, highly configurable Openbox window manager instead of a full Gnome desktop: Openbox does just what I tell it and no more, and doesn't surprise me with random redesigns. But I did let Ubuntu install some system utilities I've always avoided in the past, like NetworkManager and PulseAudio. I decided I'd give them a chance, see if they've gotten better since I last checked.

They have, though they're still a bit of a hassle to deal with. NetworkManager can be controlled through nmcli, which is poorly documented but works okay if you google long enough to find the proper incancations. PulseAudio gave me a bit more trouble.

The standard GUI for controlling PulseAudio is pavucontrol. It showed two audio devices: "USB PnP Audio Device Analog Stereo" and "Built-in Audio Analog Stereo". Turns out the USB PnP option is a sound card built into the USB hub, a Totu tt-hb003a 11-in-1 USB-C hub that lets me connect to a charger, external monitor, SD and micro-SD slots, and extra USB ports without juggling a lot of extra cables.

Pulse assumes -- probably reasonably, though it's wrong in this case -- that if I have a USB audio device connected, I probably want to use it in preference to the laptop's built-in audio. That would make sense if I had external speakers plugged in, but I left all my computer speakers behind when I moved. I should probably order some speakers. But meanwhile, I needed to persuade PulseAudio to ignore the hub and use the laptop's built-in sound system.

Mute/Unmute via the Keyboard

The Lenovo, like most laptops, has a dedicated key for muting, Fn-F1. It even has a little light on it to show whether it's muted. In Openbox, pressing Fn-F1 actually muted the sound, and even turned on the light. This is probably because I'd previously set key="XF86AudioMute" to run amixer set Master toggle in .config/openbox/rc.xml, which worked on my Pulse-free pared-down Debian netbook. The problem is that pressing iFn-F1 again didn't bring the sound back. Instead, it was unmuting the USB hub's audio. Clicking "Set as fallback" on the built-in audio in pavucontrol made no difference.

It turns out that it is virtually impossible to persuade PulseAudio to use "Built-in Audio" when a "USB PnP Audio Device" is available. I finally found the secret: in pavucontrol's Configuration tab, set Profile for the PnP USB device to Off. Now only the built-in device shows up in the other tabs.

But that amixer command still wasn't unmuting properly, so the next step was to find a command that would actually unmute. Someone on #linux suggested pactl set-sink-mute @DEFAULT_SINK@ toggle and that worked great from the command line. But when I tried to bind it in Openbox to the XF86AudioMute key, it did nothing. I still don't understand why not; I wasted a lot of time comparing my shell environment to openbox's environment and never found the difference.

Back to web searching, I found an askbuntu thread suggesting some Openbox stanzas. In particular, it apparently works better to use alsamixer rather than pactl. This finally worked for toggling mute:

    <keybind key="XF86AudioMute">
        <action name="Execute">
            <command>amixer -q -D pulse sset Master toggle</command>
        </action>
    </keybind>

Volume Controls via Function Keys

Partial success! Unfortunately, the volume control commands in that same askbuntu post, amixer -q -D pulse sset Master 3%+ unmute, did nothing. I had already noticed that in pavucontrol, the volume controls didn't work either. In fact, if I started some music playing and then called up alsamixer, channels like Master and Speaker didn't do anything; the only channel that affected volume was ALSA PCM. After some fiddling, I discovered that I had to change Master to PCM and remove the -D pulse:

    <keybind key="XF86AudioRaiseVolume">
      <action name="Execute">
            <command>amixer sset PCM 4%+ unmute</command>
        </action>
    </keybind>
    <keybind key="XF86AudioLowerVolume">
        <action name="Execute">
            <command>amixer sset PCM 4%- unmute</command>
        </action>
    </keybind>

I'm sure I'll eventually need to fiddle some more. For one thing, if I ever want to use audio during a talk (as I did briefly at my Stonehenge talk earlier this year) I'll need to figure out how to enable a temporary HDMI sound sink quickly without needing to fiddle with pavucontrol. But for now, I'm happy to have the basic laptop volume and mute keys working.

Tags: , , ,
[ 15:43 Nov 24, 2019    More linux | permalink to this entry | comments ]