Shallow Thoughts : : programming
Akkana's Musings on Open Source Computing and Technology, Science, and Nature.
Fri, 03 Jul 2015
I wrote last week about
apps with PhoneGap/Cordova. But one thing I didn't cover.
When you type
cordova build, you're building only a
debug version of your app. If you want to release it, you have to sign it.
Figuring out how turned out to be a little tricky.
Most pages on the web say you can sign your apps by creating
platforms/android/ant.properties with the same keystore
information in it that you'd put in an
build, then running
cordova build android --release
But Cordova completely ignored my ant.properties file and
went on creating a debug .apk file and no signed one.
I found various other purported solutions on the web, like creating
a build.json file in the app's top-level directory ... but that
just made Cordova die with a syntax error inside one of its own files).
This is the only method
that worked for me:
Create a file called
platforms/android/release-signing.properties, and put this in it:
// if you don't want to enter the password at every build, use this:
cordova build android --release
finally works, and creates a file called
[ 18:02 Jul 03, 2015
More programming |
permalink to this entry |
Tue, 23 Jun 2015
have made Android development much easier, I've long been curious
about the cross-platform phone development apps: you write a simple
app in some common language, like HTML or Python, then run something
that can turn it into apps on multiple mobile platforms, like
Android, iOS, Blackberry, Windows phone, UbuntoOS, FirefoxOS or Tizen.
Last week I tried two of the many cross-platform mobile frameworks:
Kivy and PhoneGap.
Kivy lets you develop in Python, which sounded like a big plus. I went
to a Kivy talk at PyCon a year ago and it looked pretty interesting.
packages them like native applications. PhoneGap seems much more
popular, but I wanted to see how it and Kivy compared.
Both projects are free, open source software.
If you want to skip the gory details, skip to the
summary: how do Kivy and PhoneGap compare?
I tried PhoneGap first.
It's based on Node.js, so the first step was installing that.
Debian has packages for nodejs, so
apt-get install nodejs npm nodejs-legacy did the trick.
You need nodejs-legacy to get the "node" command, which you'll
need for installing PhoneGap.
Now comes a confusing part. You'll be using npm to install ...
something. But depending on which tutorial you're following, it may
tell you to install and use either phonegap or cordova.
Cordova is an Apache project which is intertwined with PhoneGap. After
reading all their FAQs on the subject, I'm as confused as ever about
where PhoneGap ends and Cordova begins, which one is newer, which one
is more open-source, whether I should say I'm developing in PhoneGap
or Cordova, or even whether I should be asking questions on the
#phonegap or #cordova channels on Freenode. (The one question I had,
which came up later in the process, I asked on #phonegap and got a
helpful answer very quickly.) Neither one is packaged in Debian.
After some searching for a good, comprehensive tutorial, I ended up on a
tutorial rather than a PhoneGap one. So I typed:
sudo npm install -g cordova
Once it's installed, you can create a new app, add the android platform
(assuming you already have android development tools installed) and
build your new app:
cordova create hello com.example.hello HelloWorld
cordova platform add android
Error: Please install Android target: "android-22"
Apparently Cordova/Phonegap can only build with its own
preferred version of android, which currently is 22.
Editing files to specify android-19 didn't work for me;
it just gave errors at a different point.
So I fired up the Android SDK manager, selected android-22 for install,
accepted the license ... and waited ... and waited. In the end it took
over two hours to download the android-22 SDK; the system image is 13Gb!
So that's a bit of a strike against PhoneGap.
While I was waiting for android-22 to download, I took a look at Kivy.
As a Python enthusiast, I wanted to like Kivy best.
Plus, it's in the Debian repositories: I installed it with
sudo apt-get install python-kivy python-kivy-examples
They have a nice
tutorial for writing a Hello World app on their site. You write
it, run it locally in python to bring up a window and see what the
app will look like. But then the tutorial immediately jumps into more
advanced programming without telling you how to build and deploy
your Hello World. For Android, that information is in the
Packaging Guide. They recommend an app called Buildozer (cute name),
which you have to pull from git, build and install.
buildozer android debug deploy run
got started on building ... but then I noticed that it was attempting
to download and build its own version of apache
(sort of a Java version of
). I already have ant --
I've been using it for weeks for building my own Java android apps.
Why did it want a different version?
The file buildozer.spec in your project's
directory lets you uncomment and customize variables like:
# (int) Android SDK version to use
android.sdk = 21
# (str) Android NDK directory (if empty, it will be automatically downloaded.)
# android.ndk_path =
# (str) Android SDK directory (if empty, it will be automatically downloaded.)
# android.sdk_path =
Unlike a lot of Android build packages, buildozer will not inherit
variables like ANDROID_SDK, ANDROID_NDK and ANDROID_HOME from your
environment; you must edit buildozer.spec.
But that doesn't help with ant.
Fortunately, when I inspected the Python code for buildozer itself, I
discovered there was another variable that isn't mentioned in the
default spec file. Just add this line:
android.ant_path = /usr/bin
Next, buildozer gave me a slew of compilation errors:
kivy/graphics/opengl.c: No such file or directory
... many many more lines of compilation interspersed with errors
kivy/graphics/vbo.c:1:2: error: #error Do not use this file, it is the result of a failed Cython compilation.
I had to ask on #kivy to solve that one. It turns out that the current
version of cython, 0.22, doesn't work with kivy stable. My choices were
to uninstall kivy and pull the development version from git, or to uninstall
cython and install version 0.21.2 via pip. I opted for the latter option.
Either way, there's no "make clean", so removing the dist and build
directories let me start over with the new cython.
apt-get purge cython
sudo pip install Cython==0.21.2
rm -rf ./.buildozer/android/platform/python-for-android/dist
rm -rf ./.buildozer/android/platform/python-for-android/build
Buildozer was now happy, and proceeded to download and build Python-2.7.2,
pygame and a large collection of other Python libraries for the ARM platform.
Apparently each app packages the Python language and all libraries it needs
into the Android .apk file.
Eventually I ran into trouble because I'd named my python file hello.py
instead of main.py; apparently this is something you're not allowed to change
and they don't mention it in the docs, but that was easily solved.
Then I ran into trouble again:
Exception: Unable to find capture version in ./main.py (looking for `__version__ = ['"](.*)['"]`)
The buildozer.spec file offers two types of versioning: by default "method 1"
is enabled, but I never figured out how to get past that error with
"method 1" so I commented it out and uncommented "method 2".
With that, I was finally able to build an Android package.
The .apk file it created was quite large because of all the embedded
Python libraries: for the little 77-line pong demo,
in the Debian kivy-examples package, the apk came out 7.3Mb.
For comparison, my FeedViewer native java app, roughly 2000 lines of
Java plus a few XML files, produces a 44k apk.
The next step was to make a real mini app.
But when I looked through the Kivy examples, they all seemed highly
specialized, and I couldn't find any documentation that addressed issues
like what widgets were available or how to lay them out. How do I add a
basic text widget? How do I put a button next to it? How do I get
the app to launch in portrait rather than landscape mode?
Is there any way to speed up the very slow initialization?
I'd spent a few hours on Kivy and made a Hello World app, but I
was having trouble figuring out how to do anything more. I needed a
change of scenery.
By this time, android-22 had finally finished downloading.
I was ready to try PhoneGap again.
cordova platforms add android
worked fine. It took a long time, because it downloaded the huge gradle
build system rather than using something simpler like ant. I already have
a copy of gradle somewhere (I downloaded it for the OsmAnd build), but
it's not in my path, and I was too beaten down by this point to figure
out where it was and how to get cordova to point to it.
Cordova eventually produced a 1.8Mb "hello world" apk --
a quarter the size of the Kivy package,
though 20 times as big as a native Java app.
Deployed on Android, it initialized much faster than the Kivy app, and
came up in portrait mode but rotated correctly if I rotated the phone.
to replace pretty much all of the default CSS if you don't want your app
monopolized by the Cordova icon.
The only tricky part was file access: opening a file:// URL
didn't work. I asked on #phonegap and someone helpfully told me I'd
need the file plugin. That was easy to find in the documentation, and
I added it like this:
cordova plugin search file
cordova plugin add org.apache.cordova.file
My final apk, for a small web app I use regularly on Android,
was almost the same size as their hello world example: 1.8Mb.
And it works great: phonegap had no problem playing an audio clip,
something that was tricky when I was trying to do the same thing
from a native Android java WebView class.
Summary: How do Kivy and PhoneGap compare?
This has been a long article, I know. So how do Kivy and PhoneGap compare,
and which one will I be using?
They both need a large amount of disk space for the development environment.
I wish I had good numbers to give you, but I was working with
both systems at the same time, and their packages are scattered all
over the disk so I haven't found a good way of measuring their size. I
suspect PhoneGap is quite a bit bigger, because it uses gradle rather
than ant and because it insists on android-22.
On the other hand, PhoneGap wins big on packaged application size:
its .apk files are a quarter the size of Kivy's.
PhoneGap definitely wins on documentation. Kivy has seemingly lots of
documentation, but its tutorials jumped around rather than following
a logical sequence, and I had trouble finding answers to basic
questions like "How do I display a text field with a button?"
PhoneGap doesn't need that, because the UI is basic HTML and CSS --
limited though they are, at least most people know how to use them.
Finally, PhoneGap wins on startup speed. For my very simple test app,
startup was more or less immediate, while the Kivy Hello World app
required several seconds of startup time on my Galaxy S4.
Kivy is an interesting project. I like the ant-based build, the
straightforward .spec file, and of course the Python language.
But it still has some catching up to do in performance and documentation.
For throwing together a simple app and packaging it for Android, I
have to give the win to PhoneGap.
[ 12:09 Jun 23, 2015
More programming |
permalink to this entry |
Thu, 28 May 2015
I recently needed to update an old Android app that I hadn't touched
in years. My Eclipse setup is way out of date, and I've been hearing
about more and more projects switching to using command-line builds.
I wanted to ditch my fiddly, difficult to install Eclipse setup
and switch to something easier to use.
Some of the big open-source packages, like OsmAnd, have switched to
gradle for their Java builds. So I tried to install gradle -- and
apt-get install gradle wanted to pull
in a total of 153 packages! Maybe gradle wasn't the best option to pursue.
But there's another option for command-line android builds: ant.
When I tried apt-get install ant, since I
already have Java installed (I think the relevant package
openjdk-7-jdk), it installed without needing a single
For a small program, that's clearly a better way to go!
Then I needed to create a build directory and move my project into it.
That turned out to be fairly easy, too -- certainly compared to the
hours it spent setting up an Eclipse environment.
Here's how to set up your ant Android build:
First install the Android "Stand-alone SDK Tools" from
the Android SDK. This requires a fair amount of clicking around,
accepting licenses, and waiting for a long download.
Now install an SDK or two. Use
to install new SDK versions, and
android list targets
to see what versions you have installed.
Create a new directory for your project, cd into it, and then:
android create project --name YourProject --path . --target android-19 --package tld.yourdomain.YourProject --activity YourProject
Adjust the Android target for the version you want to use.
When this is done, type
ant with no arguments to
make sure the directory structure was created properly.
If it doesn't print errors, that's a good sign.
Check that local.properties has sdk.dir set correctly.
It should have picked that up from your environment.
There will be a stub source file in src/tld/yourdomain/YourProject.java.
Edit it as needed, or, if you're transferring a project from another
build system such as eclipse, copy the existing .java files
to that directory.
If you have custom icons for your project, or other resources like
layout or menu files, put them in the appropriate directories under res.
The directory structure is the same as in eclipse, but unlike an eclipse
build, you can edit the files at any time without the build mysteriously
Signing your app
Now you'll need a key to sign your app. Eclipse generates a debugging
key automatically, but ant doesn't. It's better to use a real key
anyway, since debugging keys expire and need to be regenerated periodically.
If you don't already have a key, generate one with:
keytool -genkey -v -keystore my-key.keystore -alias mykey -keyalg RSA -sigalg SHA1withRSA -keysize 2048 -validity 10000
It will ask you for a password; be sure to use one you won't forget
(or record it somewhere).
You can use any filename you want instead of my-key.keystore, and any
alias you want instead of mykey.
Now create a file called ant.properties containing these two lines:
Some tutorials tell you to put this in build.properties
that's outdated and no longer works.
If you forget your key alias, you can find out with this command and
keytool -list -keystore /path/to/my-key.keystore
Optionally, you can also include your key's password:
If you don't, you'll be prompted twice for the password (which echoes
on the terminal, so be aware of that if anyone is bored enough to watch
over your shoulder as you build packages. I guess build-signing keys
aren't considered particularly high security). Of course, you should
make sure not to include both the private keystore file and the password
in any public code repository.
Finally, you're ready to build!
If you get an error like:
AndroidManifest.xml:6: error: Error: No resource found that matches the given name (at 'icon' with value '@drawable/ic_launcher').
it's because older eclipse builds wanted icons named icon.png
while ant wants them named ic_launcher.png
. You can fix this
either by renaming your icons to res/drawable-hdpi/ic_launcher.png
(and the same for res/drawable-lpdi
by removing everything under bin
rm -rf bin/*
and then editing AndroidManifest.xml
. If you don't clear bin
before rebuilding, bin/AndroidManifest.xml
precendence over the AndroidManifest.xml
in the root, so you
might have to edit both files.
ant release, your binary will be in
If you have an adb connection, you can (re)install it with:
adb install -r bin/YourProject-release.apk
Done! So much easier than eclipse, and you can use any editor you want,
and check your files into any version control system.
That just leaves the coding part.
If only Java development were as easy as Python or C ...
[ 20:52 May 28, 2015
More programming |
permalink to this entry |
Thu, 19 Mar 2015
down. They've sent out notices to all project owners suggesting
they migrate projects to other hosting services.
I moved all my personal projects to GitHub years ago, back when
Google Code still didn't support git. But I'm co-owner on another
project that was still hosted there, and I volunteered to migrate it.
I remembered that being very easy back when I moved my personal projects:
GitHub had a one-click option to import from Google Code. I assumed
(I'm sure you know what that stands for) that it would be just as easy now.
Nope. Turns out GitHub no longer has any way to import from Google Code:
it tells you it can't find a repository there when you give it the
address to Google's SVN repository.
Google's announcement said they were providing an exporter to GitHub.
So I tried that next. I had the new repository ready on GitHub --
under the owner's account, not mine -- and I expected Google's
exporter to ask me for the repository.
Not so. As soon as I gave it my OAuth credentials, it immediately
created a new repository on GitHub under my name, using the name
we had used on Google Code (not the right name, since Google Code
project names have to be globally unique while GitHub projects don't).
So I had to wait for the export to finish; then, on GitHub, I went
to our real repository, and did an import there from the new
repository Google had created under my name. I have no idea how
long that took: GitHub's importer said it would email me when the
import was finished, but it didn't, so I waited several hours and
decided it was probably finished. Then I deleted the intermediate repository.
That worked fine, despite being a bit circuitous, and we're up and
running on GitHub now.
If you want to move your Google Code repository to GitHub without the
intermediate step of making a temporary repository, or if you don't
want to give Google OAuth access to your GitHub account,
here are some instructions (which I haven't tested) on how to do
the import via a local copy of the repo on your own machine, rather
than going directly from Google to GitHub:
steps for migrating Google code to GitHub
[ 13:11 Mar 19, 2015
More programming |
permalink to this entry |
Sun, 11 May 2014
I went to a terrific workshop last week on identifying bird songs.
We listened to recordings of songs from some of the trickier local species,
and discussed the differences and how to remember them. I'm not a serious
birder -- I don't do lists or Big Days or anything like that, and I
dislike getting up at 6am just because the birds do -- but I do try to
identify birds (as well as mammals, reptiles, rocks, geographic
features, and pretty much anything else I see while hiking or just
sitting in the yard) and I've always had trouble remembering their songs.
One of the tools birders use to study bird songs is the sonogram.
It's a plot of frequency (on the vertical axis) and intensity (represented
by color, red being louder) versus time. Looking at a sonogram
you can identify not just how fast a bird trills and whether it calls in
groups of three or five, but whether it's buzzy/rattly (a vertical
line, lots of frequencies at once) or a purer whistle, and whether
each note is ascending or descending.
The class last week included sonograms for the species we studied.
But what about other species? The class didn't cover even all the local
species I'd like to be able to recognize.
I have several collections of bird calls on CD
(which I bought to use in combination with my "tweet" script
-- yes, the name messes up google searches, but my tweet predates Twitter --
Python script and
in HTML for Android).
It would be great to be able to make sonograms from some of those
But a search for
Linux sonogram turned up nothing useful.
Audacity has a histogram visualization mode with lots of options, but
none of them seem to result in a usable sonogram, and most discussions
I found on the net agreed that it couldn't do it. There's another
sound editor program called snd which can do sonograms, but it's
fiddly to use and none of the many color schemes produce a sonogram
that I found very readable.
Okay, what about python scripts? Surely that's been done?
I had better luck there. Matplotlib's pylab package has a
specgram() call that does more or less what I wanted,
example of how to use pylab.specgram().
(That post also has another example using a library called timeside,
but timeside's PyPI package doesn't have any dependency information,
and after playing the old RPM-chase game installing another dependency,
trying it, then installing the next dependency, I gave up.)
The only problem with
pylab.specgram() was that it shows
the full range of the sound, both in time and frequency.
The recordings I was examining can
last a minute or more and go up to 20,000 Hz -- and when pylab tries
to fit that all on the screen, you end up with a plot where the details
are too small to show you anything useful.
You'd think there would be a way for pylab.specgram() to show
only part of the spectrum, but that doesn't seem to be.
I finally found a Stack Overflow discussion where "edited"
gives an excellent
version of pylab.specgram which allows setting minimum and maximum
frequency cutoffs. Worked great!
Then I did some fiddling to allow for analyzing only part of the
recording -- Python's wave package has no way to read in just the first
six seconds of a .wav file, so I had to read in the
whole file, read the data into a numpy array, then take a slice
representing the seconds of the recording I actually wanted.
But now I can plot nice sonograms of any bird song I want to see,
print them out or stick them on my Android device so I can carry them
Update: Oops! I forgot to include a link to the script. Here it is:
[ 09:17 May 11, 2014
More programming |
permalink to this entry |
Wed, 11 Dec 2013
When I wrote recently about my
dinosaur doggerel, I glossed over a minor problem with my final poem:
the rules of
doggerel say that the sixth line (or sometimes the seventh) should
be a single double-dactyl word -- something like "paleontologist"
or "hexasyllabic'ly". I used "dinosaur orchestra" -- two words,
which is cheating.
I don't feel too guilty about that.
If you read the post, you may recall that the verse was the result of
drifting grumpily through an insomniac morning where I would have
preferred to be getting back to sleep. Coming up with anything that
scans at all is probably good enough.
Still, it bugged me, not being able to think of a double-dactylic word
that related somehow to Parasaurolophus. So I vowed that, later that
day when I was up and at the computer, I would attempt to find one and
rewrite the poem accordingly.
I thought that would be fairly straightforward. Not so much. I thought
there would be some utility I could run that would count syllables for
me, then I could run /usr/share/dict/words through it, print
out all the 6-syllable words, and find one that fit. Turns out there
is no such utility.
But Python has a library for everything, doesn't it?
Some searching turned up
which includes some syllable-counting functions.
It apparently uses the hyphenation dictionaries that come with
There's a Debian package for it, python-pyhyphen -- but it doesn't work.
First, it depends on another package, hyphen-en-us, but doesn't
have that dependency encoded in the package, even as a suggested or
recommended package. But even when you install the hyphenated dictionary,
it still doesn't work because it doesn't point to the dictionary in
the place it was installed.
Looks like that problem was reported almost two years ago,
python-pyhyphen: doesn't work out-of-the-box with hyphen-* packages.
There's a fix there that involves editing two files,
Or you can just give up on Debian and
pip install pyhyphen,
which is a lot easier.
But once you get it working, you find that it's terrible.
It was wrong about almost every word I tried.
I hope not too many people are relying on this hyphen-en-us dictionary
for important documents. Its results seemed nearly random, and I
quickly gave up on it for getting a useful list of words around
Just for fun, since my
count syllables web search turned
up quite a few websites claiming that functionality, I tried entering
some of my long test words manually. All of the websites I tried were
wrong more than half the time, and often they were off by more than
two syllables. I don't mind off-by-ones -- I can look at words
claiming 5 and 7 syllables while searching for double dactyls --
but if I have to include 4-syllable words as well, I'll never find
what I'm looking for.
That discouraged me from using another Python suggestion I'd seen, the
nltk (natural language toolkit) package. I've been looking for an
excuse to play with nltk, and some day I will, but for this project
I was looking for a quick approximate solution, and the nltk examples
I found mostly looked like using it would require a bigger time
commitment than I was willing to devote to silly poetry. And if
none of the dedicated syllable-counting websites or dictionaries
got it right, would a big time investment in nltk pay off?
Anyway, by this time I'd wasted more than an hour poking around
various libraries and websites for this silly unimportant problem,
and I decided that with that kind of time investment, I could probably
do better on my own than the official solutions were giving me.
Why not basically just count vowels?
So I whipped up a little script,
that did just that. I gave it a list of vowels, with a few simple rules.
Obviously, you can't just say every vowel is a new syllable -- there
are too many double vowels and silent letters and such. But you can't
say that any run of multiple vowels together counts as one syllable,
because sometimes the vowels do count; and you can't make absolute
rules like "'e' at the end of a word is always silent", because
sometimes it isn't. So I kept both minimum and maximum syllable counts
for each word, and printed both.
And much to my surprise, without much tuning at all my silly little
script immediately much better results than the hyphenation dictionary
or the dedicated websites.
Alas, although it did give me quite a few hexasyllabic words in
/usr/share/dict/words, none of them were useful at all for a program
on Parasaurolophus. What I really needed was a musical term (since
that's what the poem is about). What about a musical dictionary?
I found a list of musical terms on
Glossary of musical terminology, saved it as a local file,
ran a few vim substitutes and turned it into a plain list of words.
That did a little better, and gave me some possible ideas:
But none of them worked out, and by then I'd run out of steam.
I gave up and blogged the poem as originally written, with the
cheating two-word phrase "dinosaur orchestra", and vowed to write
up how to count words in Python -- which I have now done.
Quite noncontrapuntally, and definitely not extemporaneously.
But at least I have a useful little script next time I want to
get an approximate syllable count.
[ 17:51 Dec 11, 2013
More programming |
permalink to this entry |
Wed, 28 Aug 2013
Python on Android. Wouldn't that make so many things so much easier?
I've known for a long time about
when I read, a year or two ago, that Google officially disclaimed
support for languages other than Java and C and didn't want their
employees working on projects like SL4A, I decided it wasn't a good bet.
But recently I heard from someone who had just discovered SL4A and
its Python support and talked about it like a going thing. I had an
Android scripting problem I really wanted to solve, and decided it
was time to take another look.
It turns out SL4A and its Python interpreter are still being
maintained, and indeed, I was able to solve my problem that way.
But the documentation was scanty at best. So here are some shortcuts.
Getting Python running on Android
How do you install it in the first place? Took me three or four tries:
it turns out it's extremely picky about the order in which you do
things, and the documentation doesn't warn you about that.
Follow these steps:
- Enable "Unknown Sources" under Application settings if you haven't already.
- Download both sl4a_r6.apk and PythonForAndroid_r4.apk
- Install sl4a from the apk. Do not install Python yet.
- Find SL4A in Applications and run it. It will say "no matches found"
(i.e. no scripts)
but that's okay: the important thing is that it creates the directory
where the scripts will live,
/sdcard/sl4a/scripts, without which PythonForAndroid would fail to install.
- Install PythonForAndroid from the apk.
- Find Python for Android in Applications and run it. Tap Install.
This will install the sample scripts, and you'll be ready to go.
Make a shortcut on the home screen:
You've written a script and it does what you want. But to run it, you
have to run SL4A, choose the Python interpreter, scroll around to find
the script, tap on it, and indicate whether or not you want to see
the console. Way too many steps!
Turns out you can make a shortcut on the home screen to an SL4A
script, like this:
(thanks to this
- Hit the add icon button ("+") on the main screen.
- tap on Shortcuts
- scroll down to Scripts
- choose your script
- choose the icon indicating whether you want to show the console
while the script is running
This will give you the familiar twin-snake Python icon on your home screen.
There doesn't seem to be any way to change this to a different icon.
Wait, what about UI?
Well, that still seems to be a big hole in the whole SL4A model.
You can write great scripts that print to the console. You can even
do a few specialized things, like popup menus, messages (what the
Python Android module calls makeToast()) and notifications.
The test.py sample script is a great illustration of how
to use all those features, plus a lot more.
But what if you want to show a window, put a few buttons in it,
let the user control things? Nobody seems to have thought about
that possibility. I mean, it's not "sorry, we haven't had time to
implement this", it isn't even mentioned as something someone would
want to do on an Android device. Boggle.
The only possibility I've found is that there is apparently a way to use
WebView class from Python.
I have not tried this yet; when I do, I'll write it up separately.
WebView may not be the best way to do UI. I've spent many hours
tearing my hair out over its limitations even when called from Java.
But still, it's something. And one very interesting thing about it
is that it provides an easy way to call up an HTML page, either local
or remote, from an Android home screen icon. So that may be the best
reason yet to check out SL4A.
[ 22:31 Aug 28, 2013
More programming |
permalink to this entry |
Sat, 13 Apr 2013
We've been considering the possibility of moving out of the Bay Area
to somewhere less crowded, somewhere in the desert southwest we so
love to visit. But that also means moving to somewhere
with much harsher weather.
How harsh? It's pretty easy to search for a specific location and get
average temperatures. But what if I want to make a table to compare
several different locations? I couldn't find any site that made
No problem, I say. Surely there's a Python library, I say.
Well, no, as it turns out. There are Python APIs to get the current
weather anywhere; but if you want historical weather data, or weather
data averaged over many years, you're out of luck.
NOAA purports to have historical climate data, but the only dataset I
found was spotty and hard to use. There's an
FTP site containing
directories by year; inside are gzipped files with names like
723710-03162-2012.op.gz. The first two numbers are station numbers,
and there's a file at the top level called ish-history.txt
with a list of the station codes and corresponding numbers.
Once you figure out the station codes, the files themselves are easy to
parse, with lines like
STN--- WBAN YEARMODA TEMP DEWP SLP STP VISIB WDSP MXSPD GUST MAX MIN PRCP SNDP FRSHTT
724945 23293 20120101 49.5 24 38.8 24 1021.1 24 1019.5 24 9.9 24 1.5 24 4.1 999.9 68.0 37.0 0.00G 999.9 000000
Each line represents one day (20120101 is January 1st, 2012),
and the codes are explained in another file called
For instance, MAX is the daily high temperature, and SNDP is snow depth.
So all I needed was to decode the station names, download the right files
and parse them. That took about a day to write (including a lot of
time wasted futzing with mysterious incantations for matplotlib).
Little accessibility refresher: I showed it to Dave -- "Neat, look at
this, San Jose is the blue pair, Flagstaff is green and Page is red."
"This makes no sense. They all look the same to me. I have no idea
which is which."
Oops -- right. Don't use color as your only visual indicator. I knew that,
supposedly! So I added markers in different shapes for each site.
(I wish somebody would teach that lesson to Google Maps, which uses
color as its only indicator on the traffic layer, so it's useless
for red-green colorblind people.)
Back to the data --
it turns out NOAA doesn't actually have that much historical data
available for download. If you search on most of these locations,
you'll find sites that claim to have historical temperatures dating
back 50 years or more, sometimes back to the 1800s. But NOAA typically
only has files starting at about 2005 or 2006. I don't know where
sites are getting this older data, or how reliable it is.
Still, averages since 2006 are still interesting to compare.
Here's a run of
noaatemps.py KSJC KFLG KSAF KLAM KCEZ KPGA KCNY.
It's striking how moderate California weather is compared
to any of these inland sites. No surprise there. Another surprise
was that Los Alamos, despite its high elevation, has more moderate weather
than most of the others -- lower highs, higher lows. I was a bit
disappointed at how sparse the site list was -- no site in Moab?
Really? So I used Canyonlands Field instead.
Anyway, it's fun for a data junkie to play around with, and it prints
data on other weather factors, like precipitation and snowpack, although
it doesn't plot them yet.
The code is on my
scripts page, under Weather.
Anyone found a better source for historical weather information?
I'd love to have something that went back far enough to do some
climate research, see what sites are getting warmer, colder, or
seeing greater or lesser spreads between their extreme temperatures.
The NOAA dataset obviously can't do that, so there must be something
else that weather researchers use. Data on other countries would be
interesting, too. Is there anything that's available to the public?
[ 22:57 Apr 13, 2013
More programming |
permalink to this entry |