Shallow Thoughts : tags : javascript

A Tricky Website Bug After Upgrading

Someone contacted me because my Galilean Moons of Jupiter page stopped working.

We've been upgrading the web server to the latest Debian, Bookworm (we were actually two revs back, on Buster, rather than on Bullseye, due to excessive laziness) and there have been several glitches that I had to fix, particularly with the apache2 configuration. But Galilean? That's just a bunch of JavaScript, no server-side involvement like Flask or PHP or CGI.

Read more ...

Tags: web, javascript, apache
[ 13:53 Jun 22, 2023    More linux | permalink to this entry | ]

Fiddling with JavaScript Astronomy: ThreeWorlds

I have another PEEC Planetarium talk coming up in a few weeks, a talk on the summer solstice co-presenting with Chick Keller on Fri, Jun 18 at 7pm MDT.

I'm letting Chick do most of the talking about archaeoastronomy since he knows a lot more about it than I do, while I'll be talking about the celestial dynamics -- what is a solstice, what is the sun doing in our sky and why would you care, and some weirdnesses relating to sunrise and sunset times and the length of the day. And of course I'll be talking about the analemma, because just try to stop me talking about analemmas whenever the topic of the sun's motion comes up.

But besides the analemma, I need a lot of graphics of the earth showing the terminator, the dividing line between day and night.

Read more ...

Tags: science, astronomy, programming, javascript, web
[ 18:33 Jun 06, 2021    More science/astro | permalink to this entry | ]

MarsMap: What Features of Mars are Visible?

I've been working on my upcoming PEEC talk, Observing Mars at Opposition on October 16.

Mars' closest approach was yesterday, October 6, and the actual opposition will be next Tuesday, October 13.

So, wait, we've already missed closest approach, and the opposition will be over before the actual talk happens? Then why bother?

Fortunately, opposition is actually an "opposition season", not a single date. And for most people, the best part is a little past opposition.

Read more ...

Tags: astronomy, programming, javascript
[ 18:10 Oct 07, 2020    More science/astro | permalink to this entry | ]

Displaying Quotes on a Kiosk -- and Javascript Memory Leaks

The LWV had a 100th anniversary celebration earlier this week. In New Mexico, that included a big celebration at the Roundhouse. One of our members has collected a series of fun facts that she calls "100-Year Minutes". You can see them at lwvnm.org. She asked me if it would be possible to have them displayed somehow during our display at the Roundhouse.

Of course! I said. "Easy, no problem!" I said.

Famous last words.

There are two parts: first, display randomly (or sequentially) chosen quotes with large text in a fullscreen window. Second, set up a computer (the obvious choice is a Raspberry Pi) run the kiosk automatically. This article only covers the first part; I'll write about the Raspberry Pi setup separately.

A Simple Plaintext Kiosk Python Script

When I said "easy" and "no problem", I was imagining writing a little Python program: get text, scale it to the screen, loop. I figured the only hard part would be the scaling. the quotes aren't all the same length, but I want them to be easy to read, so I wanted each quote displayed in the largest font that would let the quote fill the screen.

Indeed, for plaintext it was easy. Using GTK3 in Python, first you set up a PangoCairo layout (Cairo is the way you draw in GTK3, Pango is the font/text rendering library, and a layout is Pango's term for a bunch of text to be rendered). Start with a really big font size, ask PangoCairo how large the layout would render, and if it's so big that it doesn't fit in the available space, reduce the font size and try again. It's not super elegant, but it's easy and it's fast enough. It only took an hour or two for a working script, which you can see at quotekiosk.py.

But some of the quotes had minor HTML formatting. GtkWebkit was orphaned several years ago and was never available for Python 3; the only Python 3 option I know of for displaying HTML is Qt5's QtWebEngine, which is essentially a fully functioning browser window.

Which meant that it seeming made more sense to write the whole kiosk as a web page, with the resizing code in JavaScript. I say "seemingly"; it didn't turn out that way.

JavaScript: Resizing Text to Fit Available Space

The hard part about using JavaScript was the text resizing, since I couldn't use my PangoCairo resizing code.

Much web searching found lots of solutions that resize a single line to fit the width of the screen, plus a lot of hand-waving suggestions that didn't work. I finally found a working solution in a StackOverflow thread: Fit text perfectly inside a div (height and width) without affecting the size of the div. The only one of the three solutions there that actually worked was the jQuery one. It basically does the same thing my original Python script did: check element.scrollHeight and if it overflows, reduce the font size and try again.

I used the jquery version for a little while, but eventually rewrote it to pure javascript so I wouldn't have to keep copying jquery-min.js around.

JS Timers on Slow Machines

There are two types of timers in Javascript: setTimeout, which schedules something to run once N seconds from now, and setInterval, which schedules something to run repeatedly every N seconds. At first I thought I wanted setInterval, since I want the kiosk to keep running, changing its quote every so often.

I coded that, and it worked okay on my laptop, but failed miserably on the Raspberry Pi Zero W. The Pi, even with a lightweight browser like gpreso (let alone chromium), takes so long to load a page and go through the resize-and-check-height loop that by the time it has finally displayed, it's about ready for the timer to fire again. And because it takes longer to scale a big quote than a small one, the longest quotes give you the shortest time to read them.

So I switched to setTimeout instead. Choose a quote (since JavaScript makes it hard to read local files, I used Python to read all the quotes in, turn them into a JSON list and write them out to a file that I included in my JavaScript code), set the text color to the background color so you can't see all the hacky resizing, run the resize loop, set the color back to the foreground color, and only then call setTimeout again:

function newquote() {
    // ... resizing and other slow stuff here

    setTimeout(newquote, 30000);
}

// Display the first page:
newquote();

That worked much better on the Raspberry Pi Zero W, so I added code to resize images in a similar fashion, and added some fancy CSS fade effects that it turned out the Pi was too slow to run, but it looks nice on a modern x86 machine. The full working kiosk code is quotekioska>).

Memory Leaks in JavaScript's innerHTML

I ran it for several hours on my development machine and it looked great. But when I copied it to the Pi, even after I turned off the fades (which looked jerky and terrible on the slow processor), it only ran for ten or fifteen minutes, then crashed. Every time. I tried it in several browsers, but they all crashed after running a while.

The obvious culprit, since it ran fine for a while then crashed, was a memory leak. The next step was to make a minimal test case.

I'm using innerHTML to change the kiosk content, because it's the only way I know of to parse and insert a snippet of HTML that may or may not contain paragraphs and other nodes. This little test page was enough to show the effect:

<h1>innerHTML Leak</h1>

<p id="thecontent">
</p>

<script type="text/javascript">
var i = 0;
function changeContent() {
    var s = "Now we're at number " + i;
    document.getElementById("thecontent").innerHTML = s;
    i += 1;

    setTimeout(changeContent, 2000);
}

changeContent();
</script>

Chromium has a nice performance recording tool that can show you memory leaks. (Firefox doesn't seem to have an equivalent, alas.)

To test a leak, go to More Tools > Developer Tools and choose the Performance tab. Load your test page, then click the Record button. Run it for a while, like a couple of minutes, then stop it and you'll see a graph like this (click on the image for a full-size version).

Both the green line, Nodes, and the blue line, JS Heap, are going up. But if you run it for longer, say, ten minutes, the garbage collector eventually runs and the JS Heap line drops back down. The Nodes line never does: the node count just continues going up and up and up no matter how long you run it.

So it looks like that's the culprit: setting innerHTML adds a new node (or several) each time you call it, and those nodes are never garbage collected. No wonder it couldn't run for long on the poor Raspberry Pi Zero with 512Gb RAM (the Pi 3 with 1Gb didn't fare much better).

It's weird that all browsers would have the same memory leak; maybe something about the definition of innerHTML causes it. I'm not enough of a Javascript expert to know, and the experts I was able to find didn't seem to know anything about either why it happened, or how to work around it.

Python html2text

So I gave up on JavaScript and went back to my original Python text kiosk program. After reading in an HTML snippet, I used the Python html2text module to convert the snippet to text, then displayed it. I added image resizing using GdkPixbuf and I was good to go.

quotekiosk.py ran just fine throughout the centennial party, and no one complained about the formatting not being fancy enough. A happy ending, complete with cake and lemonade. But I'm still curious about that JavaScript leak, and whether there's a way to work around it. Anybody know?

Tags: programming, raspberry pi, python, javascript
[ 18:48 Feb 08, 2020    More tech/web | permalink to this entry | ]

Updating a Persistent Window from Javascript Part 2: A Clever Hack

I wrote about various ways of managing a persistent popup window from Javascript, eventually settling on a postMessage() solution that turned out not to work in QtWebEngine. So I needed another solution.

Data URI

First I tried using a data: URI. In that scheme, you encode a page's full content into the URL. For instance: try this in your browser: data:text/html,Hello%2C%20World!

So for a longer page, you can do something like:

    var htmlhead = '<html>\n'
        + '<head>\n'
        + '<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">\n'
        + '<link rel="stylesheet" type="text/css" href="stylesheet.css">\n'
        + '</head>\n'
        + '\n'
        + '<body>\n'
        + '<div id="mydiv">\n';
     var htmltail = '</div>\n'
        + '</body>\n'
        + '</html>\n';

    var encodedDataURI = encodeURI(htmlhead + noteText + htmltail);

    var notewin = window.open('data:text/html,' + encodedDataURI, "notewindow",
                              "width=800,height=500");

Nice and easy -- and it even works from file: URIs!

Well, sort of works. It turns out it has a problem related to the same-origin problems I saw with postMessage. A data: URI is always opened with an origin of about:blank; and two about:blank origin pages can't talk to each other.

But I don't need them to talk to each other if I'm not using postMessage, do I? Yes, I do. The problem is that stylesheet I included in htmlhead above:

<link rel="stylesheet" type="text/css" href="stylesheet.css">\n'

All browsers I tested refuse to open the stylesheet in the about:blank popup. This seems strange: don't people use stylesheets from other domains fairly often? Maybe it's a behavior special to null (about:blank) origin pages. But in any case, I couldn't find a way to get my data: URI popup to load a stylesheet. So unless I hard-code all the styles I want for the notes page into the Javascript that opens the popup window (and I'd really rather not do that), I can't use data: as a solution.

Clever hack: Use the Same Page, Loaded in a Different Way

That's when I finally came across Remy Sharp's page, Creating popups without HTML files. Remy first explores the data: URI solution, and rejects it because of the cross-origin problem, just as I did. But then he comes up with a clever hack. It's ugly, as he acknowledges ... but it works.

The trick is to create the popup with the URL of the parent page that created it, but with a named anchor appended: parentPage.html#popup. Then, in the Javascript, check whether #popup is in the URL. If not, we're in the parent page and still need to call window.open to create the popup. If it is there, then the JS code is being executed in the popup. In that case, rewrite the page as needed. In my case, since I want the popup to show only whatever is in the div named #notes, and the slide content is all inside a div called #page, I can do this:

function updateNoteWindow() {
    if (window.location.hash.indexOf('#notes') === -1) {
        window.open(window.location + '#notes', 'noteWin',
                    'width=300,height=300');
        return;
    }

    // If here, it's the popup notes window.
    // Remove the #page div
    var pageDiv = document.getElementById("page");
    pageDiv.remove();

    // and rename the #notes div so it will be displayed in a different place
    var notesDiv = document.getElementById("notes");
    notesDiv.id = "fullnotes";
}

It works great, even in file: URIs, and even in QtWebEngine. That's the solution I ended up using.

Tags: programming, web, javascript
[ 19:44 Jan 09, 2020    More tech/web | permalink to this entry | ]

Updating a Persistent Window from Javascript Part 1: postMessage

I'm trying to update my htmlpreso HTML presentation slide system to allow for a separate notes window.

Up to now, I've just used display mirroring. I connect to the projector at 1024x768, and whatever is on the first (topmost/leftmost) 1024x768 pixels of my laptop screen shows on the projector. Since my laptop screen is wider than 1024 pixels, I can put notes to myself to the right of the slide, and I'll see them but the audience won't.

That works fine, but I'd like to be able to make the screens completely separate, so I can fiddle around with other things while still displaying a slide on the projector. But since my slides are in HTML, and I still want my presenter notes, that requires putting the notes in a separate window, instead of just to the right of each slide.

The notes for each slide are in a <div id="notes"> on each page. So all I have to do is pop up another browser window and mirror whatever is in that div to the new window, right? Sure ... except this is JavaScript, so nothing is simple. Every little thing is going to be multiple days of hair-tearing frustration, and this was no exception.

I should warn you up front that I eventually found a much simpler way of doing this. I'm documenting this method anyway because it seems useful to be able to communicate between two windows, but if you just want a simple solution for the "pop up notes in another window" problem, stay tuned for Part 2.

Step 0: Give Up On file:

Normally I use file: URLs for presentations. There's no need to run a web server, and in fact, on my lightweight netbook I usually don't start apache2 by default, only if I'm actually working on web development.

But most of the methods of communicating between windows don't work in file URLs, because of the "same-origin policy". That policy is a good security measure: it ensures that a page from innocent-url.com can't start popping up windows with content from evilp0wnU.com without you knowing about it. I'm good with that. The problem is that file: URLs have location.origin of null, and every null-origin window is considered to be a different origin -- even if they're both from the same directory. That makes no sense to me, but there seems to be no way around it. So if I want notes in a separate window, I have to run a web server and use http://localhost.

Step 1: A Separate Window

The first step is to pop up the separate notes window, or get a handle to it if it's already up.

JavaScript offers window.open(), but there's a trick: if you just call notewin = window.open("notewin.html", "notewindow") you'll actually get a new tab, not a new window. If you actually want a window, the secret code for that is to give it a size:

  notewin = window.open("notewin.html", "notewindow",
                        "width=800,height=500");

There's apparently no way to just get a handle to an existing window. The only way is to call window.open(), pop up a new window if it wasn't there before, or reloads it if it's already there.

I saw some articles implying that passing an empty string "" as the first argument would return a handle to an existing window without changing it, but it's not true: in Firefox and Chromium, at least, that makes the existing window load about:blank instead of whatever page it already has. So just give it the same page every time.

Step 2: Figure Out When the Window Has Loaded

There are several ways to change the content in the popup window from the parent, but they all have one problem: if you update the content right away after calling window.open, whatever content you put there will be overwritten immediately when the popup reloads its notewin.html page (or even about:blank). So you need to wait until the popup is finished loading.

That sounds suspiciously easy. Assuming you have a function called updateNoteWinContent(), just do this:

// XXX This Doesn't work:
notewin.addEventListener('load', updateNoteWinContent, false);

Except it turns out the "load" event listener isn't called on reloads, at least not in popups. So this will work the first time, when the note window first pops up, but never after that.

I tried other listeners, like "DOMContentLoaded" and "readystatechange", but none of them are called on reload. Why not? Who knows? It's possible this is because the listener gets set too early, and then is wiped out when the page reloads, but that's just idle speculation.

For a while, I thought I was going to have to resort to an ugly hack: sleep for several seconds in the parent window to give the popup time to load: await new Promise(r => setTimeout(r, 3000)); (requires declaring the calling function as async). This works, but ... ick. Fortunately, there's a better way.

Step 2.5: Simulate onLoad with postMessage

What finally worked was a tricky way to use postMessage() in reverse. I'd already experimented with using postMessage() from the parent window to the popup, but it didn't work because the popup was still loading and wasn't ready for the content.

What works is to go the other way. In the code loaded by the popup (notewin.html in this example), put some code at the end of the page that calls

window.opener.postMessage("Loaded");

Then in the parent, handle that message, and don't try to update the popup's content until you've gotten the message:

function receiveMessageFromPopup(event) {
    console.log("Parent received a message from the notewin:", event.data);
    // Optionally, check whether event.data == "Loaded"
    // if you want to support more than one possible message.

    // Update the "notes" div in the popup notewin:
    var noteDiv = notewin.document.getElementById("notes");
    noteDiv.innerHTML = "
Here is some content.
";
}

window.addEventListener("message", receiveMessageFromPopup, false);

Here's a complete working test: Test of Persistent Popup Window.

In the end, though, this didn't solve my presentation problem. I got it all debugged and working, only to discover that postMessage doesn't work in QtWebEngine, so I couldn't use it in my slide presentation app. Fortunately, I found a couple of other ways: stay tuned for Part 2.

(Update: Part 2: A Clever Hack.)

Debugging Multiple Windows: Separate Consoles

A note on debugging: One thing that slowed me down was that JS I put in the popup didn't seem to be running: I never saw its console.log() messages. It took me a while to realize that each window has its own web console, both in Firefox and Chromium. So you have to wait until the popup has opened before you can see any debugging messages for it. Even then, the popup window doesn't have a menu, and its context menu doesn't offer a console window option. But it does offer Inspect element, which brings up a Developer Tools window where you can click on the Console tab to see errors and debugging messages.

Tags: programming, web, javascript
[ 20:29 Jan 05, 2020    More tech/web | permalink to this entry | ]

Faking Javascript <body onload=""> in Wordpress

After I'd switched from the Google Maps API to Leaflet get my trail map working on my own website, the next step was to move it to the Nature Center's website to replace the broken Google Maps version.

PEEC, unfortunately for me, uses Wordpress (on the theory that this makes it easier for volunteers and non-technical staff to add content). I am not a Wordpress person at all; to me, systems like Wordpress and Drupal mostly add obstacles that mean standard HTML doesn't work right and has to be modified in nonstandard ways. This was a case in point.

The Leaflet library for displaying maps relies on calling an initialization function when the body of the page is loaded:

<body onLoad="javascript:init_trailmap();">

But in a Wordpress website, the <body> tag comes from Wordpress, so you can't edit it to add an onload.

A web search found lots of people wanting body onloads, and they had found all sorts of elaborate ruses to get around the problem. Most of the solutions seemed like they involved editing site-wide Wordpress files to add special case behavior depending on the page name. That sounded brittle, especially on a site where I'm not the Wordpress administrator: would I have to figure this out all over again every time Wordpress got upgraded?

But I found a trick in a Stack Overflow discussion, Adding onload to body, that included a tricky bit of code. There's a javascript function to add an onload to the tag; then that javascript is wrapped inside a PHP function. Then, if I'm reading it correctly, The PHP function registers itself with Wordpress so it will be called when the Wordpress footer is added; at that point, the PHP will run, which will add the javascript to the body tag in time for for the onload even to call the Javascript. Yikes!

But it worked. Here's what I ended up with, in the PHP page that Wordpress was already calling for the page:

<?php
/* Wordpress doesn't give you access to the <body> tag to add a call
 * to init_trailmap(). This is a workaround to dynamically add that tag.
 */
function add_onload() {
?>

<script type="text/javascript">
  document.getElementsByTagName('body')[0].onload = init_trailmap;
</script>

<?php
}

add_action( 'wp_footer', 'add_onload' );
?>

Complicated, but it's a nice trick; and it let us switch to Leaflet and get the PEEC interactive Los Alamos area trail map working again.

Tags: web, programming, javascript, php, wordpress, mapping
[ 15:49 May 27, 2018    More tech/web | permalink to this entry | ]

Google Maps API No Longer Free?

A while ago I wrote an interactive trail map page for the PEEC nature center website. At the time, I wanted to use an open library, like OpenLayers or Leaflet; but there were no good sources of satellite/aerial map tiles at the time. The only one I found didn't work because they had a big blank area anywhere near LANL -- maybe because of the restricted airspace around the Lab. Anyway, I figured people would want a satellite option, so I used Google Maps instead despite its much more frustrating API.

This week we've been working on converting the website to https. Most things went surprisingly smoothly (though we had a lot more absolute URLs in our pages and databases than we'd realized). But when we got through, I discovered the trail map was broken. I'm still not clear why, but somehow the change from http to https made Google's API stop working. In trying to fix the problem, I discovered that Google's map API may soon cease to be free:

New pricing and product changes will go into effect starting June 11, 2018. For more information, check out the Guide for Existing Users.

That has a button for "Transition Tool" which, when you click it, won't tell you anything about the new pricing structure until you've already set up a billing account. Um ... no thanks, Google.

Googling for google maps api billing led to a page headed "Pricing that scales to fit your needs", which has an elaborate pricing structure listing a whole bnch of variants (I have no idea which of these I was using), of which the first $200/month is free. But since they insist on setting up a billing account, I'd probably have to give them a credit card number -- which one? My personal credit card, for a page that isn't even on my site? Does the nonprofit nature center even have a credit card? How many of these API calls is their site likely to get in a month, and what are the chances of going over the limit?

It all rubbed me the wrong way, especially when the context of "Your trail maps page that real people actually use has broken without warning, and will be held hostage until you give usa credit card number". This is what one gets for using a supposedly free (as in beer) library that's not Free open source software.

So I replaced Google with the excellent open source Leaflet library, which, as a bonus, has much better documentation than Google Maps. (It's not that Google's documentation is poorly written; it's that they keep changing their APIs, but there's no way to tell the dozen or so different APIs apart because they're all just called "Maps", so when you search for documentation you're almost guaranteed to get something that stopped working six years ago -- but the documentation is still there making it look like it's still valid.) And I was happy to discover that, in the time since I originally set up the trailmap page, some open providers of aerial/satellite map tiles have appeared. So we can use open source and have a satellite view.

Our trail map is back online with Leaflet, and with any luck, this time it will keep working. PEEC Los Alamos Area Trail Map.

Tags: mapping, web, programming, javascript
[ 16:13 May 24, 2018    More programming | permalink to this entry | ]

Quickly seeing bird sightings maps on eBird

The local bird community has gotten me using eBird. It's sort of social networking for birders -- you can report sightings, keep track of what birds you've seen where, and see what other people are seeing in your area.

The only problem is the user interface for that last part. The data is all there, but asking a question like "Where in this county have people seen broad-tailed hummingbirds so far this spring?" is a lengthy process, involving clicking through many screens and typing the county name (not even a zip code -- you have to type the name). If you want some region smaller than the county, good luck.

I found myself wanting that so often that I wrote an entry page for it.

My Bird Maps page is meant to be used as a smart bookmark (also known as bookmarklets or keyword bookmarks), so you can type birdmap hummingbird or birdmap golden eagle in your location bar as a quick way of searching for a species. It reads the bird you've typed in, and looks through a list of species, and if there's only one bird that matches, it takes you straight to the eBird map to show you where people have reported the bird so far this year.

If there's more than one match -- for instance, for birdmap hummingbird or birdmap sparrow -- it will show you a list of possible matches, and you can click on one to go to the map.

Like every Javascript project, it was both fun and annoying to write. Though the hardest part wasn't programming; it was getting a list of the nonstandard 4-letter bird codes eBird uses. I had to scrape one of their HTML pages for that. But it was worth it: I'm finding the page quite useful.

How to make a smart bookmark

I think all the major browsers offer smart bookmarks now, but I can only give details for Firefox. But here's a page about using them in Chrome.

Firefox has made it increasingly difficult with every release to make smart bookmarks. There are a few extensions, such as "Add Bookmark Here", which make it a little easier. But without any extensions installed, here's how you do it in Firefox 36:

First, go to the birdmap page (or whatever page you want to smart-bookmark) and click on the * button that makes a bookmark. Then click on the = next to the *, and in the menu, choose Show all bookmarks. In the dialog that comes up, find the bookmark you just made (maybe in Unsorted bookmarks?) and click on it.

Click the More button at the bottom of the dialog.
(Click on the image at right for a full-sized screenshot.)

Now you should see a Keyword entry under the Tags entry in the lower right of that dialog.

Change the Location to http://shallowsky.com/birdmap.html?bird=%s.

Then give it a Keyword of birdmap (or anything else you want to call it).

Close the dialog.

Now, you should be able to go to your location bar and type:
birdmap common raven or birdmap sparrow and it will take you to my birdmap page. If the bird name specifies just one bird, like common raven, you'll go straight from there to the eBird map. If there are lots of possible matches, as with sparrow, you'll stay on the birdmap page so you can choose which sparrow you want.

How to change the default location

If you're not in Los Alamos, you probably want a way to set your own coordinates. Fortunately, you can; but first you have to get those coordinates.

Here's the fastest way I've found to get coordinates for a region on eBird:

• Click "Explore a Region"
• Type in your region and hit Enter
• Click on the map in the upper right

Then look at the URL: a part of it should look something like this: env.minX=-122.202087&env.minY=36.89291&env.maxX=-121.208778&env.maxY=37.484802 If the map isn't right where you want it, try editing the URL, hitting Enter for each change, and watch the map reload until it points where you want it to. Then copy the four parameters and add them to your smart bookmark, like this: http://shallowsky.com/birdmap.html?bird=%s&minX=-122.202087&minY=36.89291&maxX=-121.208778&maxY=37.484802

Note that all of the the "env." have been removed.

The only catch is that I got my list of 4-letter eBird codes from an eBird page for New Mexico. I haven't found any way of getting the list for the entire US. So if you want a bird that doesn't occur in New Mexico, my page might not find it. If you like birdmap but want to use it in a different state, contact me and tell me which state you need, and I'll add those birds.

Tags: nature, birds, eBird, web, programming, javascript, firefox, bookmarklets
[ 14:30 Apr 06, 2015    More nature/birds | permalink to this entry | ]

Whac-a-Moon: Watch Europa appear and disappear this Sunday

This is an edited and updated version of my "Shallow Sky" column this month in the SJAA Ephemeris newsletter.

A few months ago, I got email from a Jupiter observer calling my attention to an interesting phenomenon of Jupiter's moons that I hadn't seen before. The person who mailed me described himself as a novice, and wasn't quite sure what he had seen, but he knew it was odd. After some further discussion we pinned it down.

He was observing Jupiter at 11/11/12 at 00.25 UT (which would have been mid-afternoon here in San Jose). Three of the moons were visible, with only Ganymede missing. Then Ganymede appeared: near Jupiter's limb, but not right on it. As he watched over the next few minutes, Ganymede seemed to be moving backward -- in toward Jupiter rather than away from it. Eventually it disappeared behind the planet.

It turned out that what he was seeing was the end of an eclipse. Jupiter was still a few months away from opposition, so the shadow thrown by the planet streamed off to one side as seen from our inner-planet vantage point on Earth. At 0:26 UT on that evening, long before he started observing, Ganymede, still far away from Jupiter's limb, had entered Jupiter's shadow and disappeared into eclipse. It took over two hours for Ganymede to cross Jupiter's shadow; but at 2:36, when it left the shadow, it hadn't yet disappeared behind the planet. So it became visible again. It wasn't until 2:50 that Ganymede finally disappeared behind Jupiter.

So it was an interesting effect -- bright Ganymede appearing out of nowhere, moving in toward Jupiter then disappearing again fourteen minutes later. It was something I'd never seen, or thought to look for. It's sort of like playing Whac-a-mole -- the moon appears only briefly, so you've got to hit it with your telescope at just the right time if you want to catch it before it disappears again.

A lot of programs don't show this eclipse effect -- including, I'm sad to say, my own Javascript Jupiter's moons web page. (I have since remedied that.) The open source program Stellarium shows the effect; on the web, Sky and Telescope's Jupiter's Moons page shows it, and even prints out a table of times of various moon events, including eclipses.

These eclipse events aren't all that uncommon -- but only when the sun angle is just right. Searching in late February and early March this year, I found several events for Ganymede and Europa (though, sadly, many of them were during our daytime). By mid-March, the angles have changed so that Europa doesn't leave Jupiter's shadow until after it's disappeared behind the planet's limb; but Ganymede is farther out, so we can see Ganymede appearances all the way through March and for months after.

The most interesting view, it seems to me, is right on the boundary when the moon only appears for a short time before disappearing again. Like the Europa eclipse that's happening this Sunday night, March 10.

Reporting on that one got a little tricky -- because that's the day we switch to Daylight Savings time. I have to confess that I got a little twisted up trying to compare results between programs that use UTC and programs that use local time -- especially when the time zone converter I was using to check my math told me "That time doesn't exist!" Darnit, if we'd all just use UTC all the time, astronomy calculations would be a lot easier! (Not to mention dropping the silly Daylight Savings Time fiasco, but that's another rant.)

Before I go into the details, I want to point out that Jupiter's moons are visible even in binoculars. So even if you don't have a telescope, grab binoculars and set them up in as steady a way as you can -- if you don't have a tripod adaptor, try bracing them on the top of a gate or box.

On Sunday night, March 10, at some time around 7:40 pm PDT, Europa peeks out from behind Jupiter's northeast limb. (All times are given in PDT; add 7 hours for GMT.) The sky will still be bright here in California -- the sun sets at 7:12 that night -- but Jupiter will be 66 degrees up and well away from the sun, so it shouldn't give you too much trouble. Once Europa pops out, keep a close eye on it -- because if Sky & Tel's calculations are right, it will disappear again just four minutes later, at 7:44, into eclipse in Jupiter's shadow. It will remain invisible for almost three hours, finally reappearing out of nowhere, well off Jupiter's limb, at around 10:24 pm.

Here's a link to my Javascript Jupiter just before Europa reappears.

I want to stress that those times are approximate. In fact, I tried simulating the event in several different programs, and got wildly varying times:

	Io disappears	Europa appears	Europa disappears	Europa reappears	Io appears
XEphem	7:15	7:43	7:59	10:06	10:43
S&T Jupiter's Moons	7:16	7:40	7:44	10:24	10:48
Javascript Jupiter	7:17	7:45	7:52	10:15	10:41
Stellarium	6:21	6:49	7:05	9:32	10:01

You'll note Stellarium seems to have a time zone problem ... maybe because I ran the prediction while we were still in standard time, not daylight savings time.

I'm looking forward to timing the events to see which program is most accurate. I'm betting on XEphem. Once I know the real times, maybe I can adjust my Javascript Jupiter's code to be more accurate. If anyone else times the event, please send me your times, in case something goes wrong here!

Anyway, the spread of times makes it clear that when observing this sort of phenomenon, you should always set up the telescope ten or fifteen minutes early, just in case. And ten extra minutes spent observing Jupiter -- even without moons -- is certainly never time wasted! Just keep an eye out for Europa to appear -- and be ready to whack that moon before it disappears again.

Tags: astronomy, programming, javascript
[ 11:30 Mar 09, 2013    More science/astro | permalink to this entry | ]

HTML and Javascript Presentations

When I give talks that need slides, I've been using my Slide Presentations in HTML and JavaScript for many years. I uploaded it in 2007 -- then left it there, without many updates.

But meanwhile, I've been giving lots of presentations, tweaking the code, tweaking the CSS to make it display better. And every now and then I get reminded that a few other people besides me are using this stuff.

For instance, around a year ago, I gave a talk where nearly all the slides were just images. Silly to have to make a separate HTML file to go with each image. Why not just have one file, img.html, that can show different images? So I wrote some code that lets you go to a URL like img.html?pix/whizzyphoto.jpg, and it will display it properly, and the Next and Previous slide links will still work.

Of course, I tweak this software mainly when I have a talk coming up. I've been working lately on my SCALE talk, coming up on January 22: Fun with Linux and Devices (be ready for some fun Arduino demos!) Sometimes when I overload on talk preparation, I procrastinate by hacking the software instead of the content of the actual talk. So I've added some nice changes just in the past few weeks.

For instance, the speaker notes that remind me of where I am in the talk and what's coming next. I didn't have any way to add notes on image slides. But I need them on those slides, too -- so I added that.

Then I decided it was silly not to have some sort of automatic reminder of what the next slide was. Why should I have to put it in the speaker notes by hand? So that went in too.

And now I've done the less fun part -- collecting it all together and documenting the new additions. So if you're using my HTML/JS slide kit -- or if you think you might be interested in something like that as an alternative to Powerpoint or Libre Office Presenter -- check out the presentation I have explaining the package, including the new features.

You can find it here: Slide Presentations in HTML and JavaScript

Tags: speaking, javascript, html, web, programming, tech
[ 21:08 Jan 12, 2012    More speaking | permalink to this entry | ]

Displaying images from Javascript file inputs

(despite Firefox's attempts to prevent that)

My Linux Planet article last week was on printing pretty calendars. But I hit one bug in Photo Calendar. It had a HTML file chooser for picking an image ... and when I chose an image and clicked Select to use it. it got the pathname wrong every time.

I poked into the code (Photo Calendar's code turned out to be exceptionally clean and well documented) and found that it was expecting to get the pathname from the file input element's value attribute. But input.File.value was just returning the filename, foo.jpg, instead of the full pathname, /home/user/Images/yosemite/foo.jpg. So when the app tried to make it into a file:/// URL, it ended up pointing to the wrong place.

It turned out the cause was a security change in Firefox 3. The issue: it's considered a security hole to expose full pathnames on your computer to Javascript code coming from someone else's server. The Javascript could give bad guys access to information about the directory structures on your disk. That's a perfectly reasonable concern, and it makes sense to consider it as a security hole.

The problem is that this happens even when you're running a local app on your local disk. Programs written in any other language and toolkit -- a Python program using pygtk, say, or a C++ Qt program -- have access to the directories on your disk, but you can't use Javascript inside Firefox to do the same thing. The only ways to make an exception seems to be an elaborate procedure requiring the user to change settings in about:config. Not too helpful.

Perhaps this is even reasonable, given how common cross-site scripting bugs have been in browsers lately -- maybe running a local script really is a security risk if you have other tabs active. But it leaves us with the problem of what to do about apps that need to do things like choose a local image file, then display it.

And it turns out there is: a data URL. Take the entire contents of the file (ouch) and create a URL out of those contents, then set the src attribute of the image to that.

Of course, that makes for a long, horrifying, unreadable URL -- but the user never has to see that part. I suspect it's also horribly memory intensive -- the image has to be loaded into memory anyway, to display it, but is Firefox also translating all of that to a URL-legal syntax? Obviously, any real app using this technique had better keep an eye on memory consumption. But meanwhile, it fixes Photo Calendar's file button.

Here's what the code looks like:

  img = document.getElementById("pic");
  fileinput = document.input.File;
  if (img && fileinput)
    img.src = fileinput.files[0].getAsDataURL();

Here's a working minimal demo of using getAsDataURL() with a file input.

Tags: javascript, web, programming
[ 14:57 Jan 17, 2010    More programming | permalink to this entry | ]

Debug logging in Javascript under Firefox

While debugging Javascript, I've occasionally come across references to a useful function called console.log. Supposedly you can log errors with a line like:

  console.log(""key press, char code " + e.charCode);

Since the only native way of logging debug statements in Javascript is with a pop-up alert() box, having a less obtrusive way to print is something any JS programmer could use.

The catch? It didn't do anything -- except print console is not defined.

Today a friend was crowing about how wonderful Javascript debugging was in Google's Chrome browser -- because it has functions like console.log.

After some searching and poking around, we determined that Firefox also has console.log -- it's just well hidden and a bit hard to get going.

First, you need the Firebug extension. If you're developing Javascript, you probably already have it. If not, you need it.

Run Firebug and click to the Console tab. Now click on the tiny arrow that shows up at the right edge of that tab, as shown. Turns out there's a whole menu of options under there -- one of which is Enabled.

But wait, that's not all. In my case, the console was already Enabled according to the menu. To get the console working, I had to

1. Disable the console
2. Re-enable it
3. Shift-reload the page being debugged

My friend also said that if she didn't enable the console in Firebug, then her script died when she called console.log. That didn't happen for me -- all that happened was that I got error messages in the error console (the one accessed from Firefox's Tools menu, different from the Firebug console). But it's a good idea to check for its existence if you're going to use debugging statements in your code. Like this:

  if (typeof console != "undefined") {
    console.log( "key press, char code " + e.charCode
                + ", key code " + e.keyCode
                + ", " + e.ctrlKey + ", " + e.altKey
		+ ", " + e.metaKey );
  }

Here are some more things you can do with Firebug's console.

Tags: firefox, javascript, programming, tips
[ 16:41 Nov 28, 2009    More tech/web | permalink to this entry | ]

A Pair of URL-Shortening Bookmarklets

I'm not a big fan of URL-shortening services -- I like to see what page I'm about to load so I know if I want to go there. But with Twitter's 160-character limit, URL shorteners become necessary. It's tiresome to type in bit.ly every time, so I wanted a bookmark to say "give me a shortened version of the current URL".

Surprisingly, I had a hard time finding one. bit.ly itself has one on their front page, but it didn't work for me. Upon examination, it looks like their bookmark wants to read the clipboard, so you'd have to select a URL first before shortening it (though they don't actually tell you that). I don't want that extra step, so I made my own. Actually two of them.

First, a Javascript version that takes the current URL, encodes it and sends it to bit.ly. I gave it the keyword "bitly", so when I'm on a page, I just type "bitly" in the URLbar and it goes to bit.ly and makes the shortened URL.

The only problem with that is that I'd rather have the option of opening it in a new tab, so I can continue to read the original page. Normally I open new tabs by typing in a URL and typing Ctrl-Return (normally it's Alt-Return in Firefox, but it drives me nuts that Firefox uses Ctrl-click for new tab but Alt-Return and I can never keep them straight, and Firefox's normal behavior for Ctrl-Return is brain-dead useless so that's the first thing I fix when I get a Firefox update).

With this bitly bookmarklet, Ctrl-Return and Alt-Return don't work -- because then you lose the original tab's URL, and bitly gives you a shortened URL to nowhere ... "nowhere" being defined, in the bitly universe, as http://about.com (go figure). What to do?

So I made a second bookmarklet using a different technique: instead of using Javascript to get the current page's URL, call the bookmarklet with the URL as argument. I called this one bitly2. So if I'm pointing at http://shallowsky.com/blog/ and I want a shortened version in a new tab, I type:

Ctrl-L to go to the URLbar
Ctrl-A to go to the beginning of the URL
bitly2 and a space (inserted at the beginning before the URL)
so now I'll see bitly2 http://shallowsky.com/blog/
Ctrl-Return (or Alt-Return) to open in a new tab.

I'm not sure which one I'll end up using more, but I'll obviously change the bitly2 name to something better if I end up using it a lot.

If you want to use either of these bookmarklets: right-click on the link and choose Bookmark this link. Then, alas, since Firefox still doesn't let you enter a keyword in its Bookmarks dialog, you have to go to Bookmarks->Organize Bookmarks, find the bookmarklet you just added and click on it, click on More, and finally you can give it a keyword.

There used to be a Firefox extension called Openbook that let you see the Keyword field when you first add a bookmark, but it doesn't work any more in 3.5, alas. There's another extension called "Add Bookmark Here 2" that's supposed to do it, but the file on addons.mozilla.org is apparently corrupted and won't install. I don't understand why the Firefox crew is so obsessed with bookmark tags (for which I've never found any use) but won't let you add something as truly useful as a keyword. (It's bug 242834, marked WONTFIX.)

Of course, after I had my bookmarklets I finally found a page with a decent bit.ly bookmarklet very similar to my first one: A Quick Tutorial on JavaScript Bookmarklets.

Tags: bookmarklets, twitter, javascript
[ 15:34 Aug 18, 2009    More tech/web | permalink to this entry | ]

JS Jup: now, with variable animation speed

I wrote last week about the sorts of programmer compulsions that lead to silly apps like my animated Javascript Jupiter. I got it working well enough and stopped, knowing there were more features that would be easy to add but trying to ignore them.

My mom, immediately upon seeing it, unerringly zeroed in on the biggest missing feature I'd been trying to ignore. "Can you make it go faster or slower?"

I put it off for a while, but of course I had to do it -- so now there are Faster and Slower buttons. It still goes by hour jumps, so the fastest you can go is an hour per millisecond. Fun to watch. Or you can slow it down to 1 hour per 3600000 milliseconds if you want to see it animate in real time. :-)

Tags: programming, astronomy, javascript
[ 11:42 May 31, 2009    More programming | permalink to this entry | ]

Javascript Jupiter

It's a sickness, I tell you.

It's not like I needed another Jupiter's moons application. I've already written more or less the same app for four platforms.

I don't use the Java web version, Juplet, very much any more, because I often have Java disabled or missing. And I don't use my Zaurus any more so Juplet for Zaurus isn't very relevant. But I can always call up my Xlib or PalmOS Jupiter's moons app if I need to check on those Galilean moons. They work fine. Another version would be really pointless. A waste of time.

So it should have been no big deal when, during the course of explaining to someone the difference between Java and Javascript, it suddenly occurred to me that it would be awfully easy to re-implement that Java Juplet web page using Javascript, HTML and CSS. I mean, a rational person would just say "oh, yeah, I suppose that's true" and go on with life.

But what I'm trying to say is that programming isn't a career path, or a hobby, or a field of academic study. It's a disease. It's a compulsion, where, sometimes, just realizing that something could be done renders you unable to think about anything else until you just ... try ... just a few minutes ... see how well it works ... oh, wow, that really looks a lot better than the Java version, wouldn't it look even nicer if you just added in this one other little tweak ... but wait, now it's so close to working, I bet it wouldn't be all that hard to take the Java class and turn it into ...

... and before you know it, it's tomorrow and you have something that's almost a working app, and it's just really a shame to get that far and not finish it at least to the point where you can share it.

But then, Javascript and web pages are so easy to work on that it really isn't that much extra work to add in some features that the old version didn't have, like an animate button ...

... and your Saturday morning is gone forever, and there's not much you can do about that, but at least you have a nice animated Jupiter's moons (and shadows) page when the sickness passes and you can finally think about other things.

Tags: programming, astronomy, javascript
[ 21:10 May 23, 2009    More programming | permalink to this entry | ]

GetSET: Teaching Javascript to high school girls

Every summer I volunteer as an instructor for a one-day Javascript programming class at the GetSET summer technology camp for high school girls. GetSET is a great program run by the Society of Women Engineers. it's intended for minority girls from relatively poor neighborhoods, and the camp is free to the girls (thanks to some great corporate sponsors). They're selected through a competitive interview process so they're all amazingly smart and motivated, and it's always rewarding being involved.

Teaching programming in one day to people with no programming background at all is challenging, of course. You can't get into any of the details you'd like to cover, like style, or debugging techniques. By the time you get through if-else, for and while loops, some basic display methods, the usual debugging issues like reading error messages, and typographical issues like "Yes, uppercase and lowercase really are different" and "No, sorry, that's a colon, you need a semicolon", it's a pretty full day and the students are saturated.

I got drafted as lead presenter several years ago, by default by virtue of being the only one of the workshop leaders who actually programs in Javascript. For several years I'd been asking for a chance to rewrite the course to try to make it more fun and visual (originally it used a lot of form validation exercises), and starting with last year's class I finally got the chance. I built up a series of graphics and game exercises (using some of Sara Falamaki's Hangman code, which seemed perfect since she wrote it when she was about the same age as the girls in the class) and it went pretty well. Of course, we had no idea how fast the girls would go or how much material we could get through, so I tried to keep it flexible and we adjusted as needed.

Last year went pretty well, and in the time since then we've exchanged a lot of email about how we could improve it. We re-ordered some of the exercises, shifted our emphasis in a few places, factored some of the re-used code (like windowWidth()) into a library file so the exercise files weren't so long, and moved more of the visual examples earlier.

I also eliminated a lot of the slides. One of the biggest surprises last year was the "board work". I had one exercise where the user clicks in the page, and the student has to write the code to figure out whether the click was over the image or not. I had been nervous about that exercise -- I considered it the hardest of the exercises. You have to take the X and Y coordinates of the mouse click, the X and Y coordinates of the image (the upper left corner of the <div> or <img> tag), and the size of the image (assumed to be 200x200), and turn that all into a couple of lines of working Javascript code. Not hard once you understand the concepts, but hard to explain, right?

I hadn't made a slide for that, so we went to the whiteboard to draw out the image, the location of the mouse click, the location of the image's upper left corner, and figure out the math ... and the students, who had mostly been sitting passively through the heavily slide-intensive earlier stuff, came alive. They understood the diagram, they were able to fill in the blanks and keep track of mouse click X versus image X, and they didn't even have much trouble turning that into code they typed into their exercise. Fantastic!

Remembering that, I tried to use a lot fewer slides this year. I felt like I still needed to have slides to explain the basic concepts that they actually needed to use for the exercises -- but if there was anything I thought they could figure out from context, or anything that was just background, I cut it. I tried for as few slides as possible between exercises, and more places where we could elicit answers from the students. I think we still have too many slides and not enough "board work" -- but we're definitely making progress, and this year went a lot better and kept them much better engaged. We're considering next year doing the first several exercises on the board first, then letting them type it in to their own copies to verify that it works.

We did find we needed to leave code examples visible: after showing slides saying something like "Ex 7: Write a loop that writes a line of text in each color", I had to back up to the previous slide where I'd showed what the code actually looked like. I had planned on their using my "Javascript Quick Reference" handout for reference and not needing that information on the slides; but in fact, I think they were confused about the quickref and most never even opened it. Either that information needs to be in the handout, or it needs to be displayed on the screen as they work, or I have to direct them to the quickref page explicitly ("Now turn to page 3 in ...") or put that information in the exercises.

The graphical flower exercises were a big hit this year (I showed them early and promised we'd get to them, and when we did, just before lunch, several girls cheered) and, like last year, some of the girls who finished them earlier decided on their own that they wanted to change them to use other images, which was also a big hit. Several other girls decided they wanted more than 10 flowers displayed, and others hit on the idea of changing the timeout to be a lot shorter, which made for some very fun displays. Surprisingly, hardly anyone got into infinite loops and had to kill the browser (always a potential problem with javascript, especially when using popups like alert() or prompt()).

I still have some issues I haven't solved, like what to do about semicolons and braces. Javascript is fairly agnostic about them. Should I tell the girls that they're required? (I did that this year, but it's confusing because then when you get to "if" statements you have to explain why that's different.) Not mention them at all? (I'm leaning toward that for next year.)

And it's always a problem figuring out what the fastest girls should do while waiting for the rest to finish. This year, in addition to trying to make each exercise shorter, we tried having the girls work on them in groups of two or three, so they could help each other. It didn't quite work out that way -- they all worked on their own copies of the exercises but they did seem to collaborate more, and I think that's the best balance. We also encourage the ones who finish first to help the girls around them, which mostly they do on their own anyway.

And we really do need to find a better editor we can use on the Windows lab machines instead of Wordpad. Wordpad's font is too small on the projection machine, and on the lab machines it's impossible for most of us to tell the difference between parentheses, brackets and braces, which leads to lots of time-wasting subtle bugs. Surely there's something available for Windows that's easy to use, freely distributable, makes it easy to change the font, and has parenthesis and brace matching (syntax highlighting would be nice too). Well, we have a year to look for one now.

All in all, we had a good day and most of the girls gave the class high marks. Even the ones who concluded "I learned I shouldn't be a programmer because it takes too much attention to detail" said they liked the class. And we're fine with that -- not everybody wants to be a programmer, and the point isn't to force them into any specific track. We're happy if we can give them an idea of what computer programming is really like ... then they'll decide for themselves what they want to be.

Tags: education, javascript, programming, speaking
[ 12:54 Aug 09, 2008    More education | permalink to this entry | ]