Shallow Thoughts : tags : android

Playing an MP3 file from an Android app

I've mostly been enormously happy with my upgrade from my old Archos 5 to the Samsung Galaxy Player 5.0. The Galaxy does everything I always wanted the Archos to do, all those things the Archos should have done but couldn't because of its buggy and unsupported Android 1.6.

That is, I've been happy with everything except one thing: my birdsong app no longer worked.

I have a little HTML app based on my "tweet" python script which lets you choose from a list of birdsong MP3 files. (The actual MP3 files are ripped from the excellent 4-CD Stokes Field Guide to Western Bird Songs set.) The HTML app matches bird names as you type in characters. (If you're curious, an earlier test version is at tweet.html.)

On the Archos, I ran that under my WebClient Android app (I had to modify the HTML to add a keyboard, since in Android 1.6 the soft keyboard doesn't work in WebView text fields). I chose a bird, and WebView passed off the MP3 file to the Archos' built-in audio player. Worked great.

On the Samsung Galaxy, no such luck. Apparently Samsung's built-in media player can only play files it has indexed itself. If you try to use it to play an arbitrary file, say, "Song_Sparrow.mp3", it will say: unknown file type. No matter that the file ends in .mp3 ... and no matter that I've called intent.setDataAndType(Uri.parse(url), "audio/mpeg"); ... and no matter that the file is sitting on the SD cad and has in fact been indexed already by the media player. You didn't navigate to it via the media player's UI, so it refuses to play it.

I haven't been able to come up with an answer to how to make Samsung's media player more flexible, and I was just starting a search for alternate Android MP3 player apps, when I ran across Play mp3 in SD Card, using Android's MediaPlayer and Error creating MediaPlayer with Uri or file in assets which gave me the solution. Instead of using an intent and letting WebView call up a music application, you can use an Android MediaPlayer to play your file directly.

Here's what the code looks like, inside setWebViewClient() which is itself inside onCreate():

            @Override
            public boolean shouldOverrideUrlLoading(WebView view, String url) {
                if (url.endsWith(".mp3")) {
                    MediaPlayer mediaPlayer = new MediaPlayer();
                    try {
                        mediaPlayer.setDataSource(getApplicationContext(), Uri.parse(url));
                        mediaPlayer.prepare();
                        mediaPlayer.start();
                    }
                    catch (IllegalArgumentException e) { showMessage("Illegal argument exception on " + url); }
                    catch (IllegalStateException e) { showMessage("Illegal State exception on " + url); }
                    catch (IOException e) { showMessage("I/O exception on " + url); }
                }
            }

showMessage() is my little wrapper that pops up an error message dialog. Of course, you can handle other types, not just files ending in .mp3.

And now I can take the Galaxy out on a birdwalk and use it to help me identify bird songs.

Android WebView can't goBack from a page with iframes

I've been fighting a bug in Android's WebView class for ages: on some pages, clicking FeedViewer's back arrow (which calls WebView::goBack()) doesn't go back to the previous page. Instead, it jumps to some random position in the current page. If you repeat it, eventually, after five or so tries (depending on the page), eventually goBack() will finally work and you'll be back at the previous page.

It was especially frustrating in that it didn't happen everywhere -- only in pages from certain sites. I saw it all the time in pages from the Los Angeles Times and from Make Magazine, but only rarely on other sites.

But I finally tracked it down: it's because those pages include the HTML <iframe> tag. Apparently, if a WebView is on a page (at least if it's a local page) that contains N iframes, the first N calls to goBack will jump somewhere in the document -- probably the location of the associated iframe, though I haven't verified that -- and only on the N+1st call will the WebView actually go back to the previous page.

The oddest thing is, this doesn't seem to be reported anywhere. Android's bug tracker finds nothing for webview iframe goback, and web searching hasn't found a hint of it, even though I see this in Android versions from 1.6 through 2.3.5. How is it possible that other people haven't noticed this? I wonder if it only happens on local file:// URLs, and not many people use those.

In any case, I'm very happy to have found the answer at last. It was easy enough to modify FeedMe to omit iframes (and who wants iframes in simplified HTML anyway?), and it's great to have a Back button that works on any page.

Mounting a Samsung Galaxy Player on Linux

My new toy: a Samsung Galaxy Player 5.0!

So far I love it. It's everything my old Archos 5 wanted to be when it grew up, except the Archos never grew up. It's the same size, a little lighter weight, reliable hardware (no random reboots), great battery life, fast GPS, modern Android 2.3, and the camera is even not too bad (though it certainly wouldn't tempt me away from my Canon).

For the record, Dave got a Galaxy Player 4.0, and it's very nifty too, and choosing between them was a tough choice -- the 4-inch is light and so easy to hold, and it uses replaceable batteries, while the 5-inch's screen is better for reading and maps.

USB-storage devices don't register

I love the Galaxy ... but there's one thing that bugs me about it. When I plug it in to Linux, dmesg reports two new storage devices, one for main storage and one for the SD card. Just like most Android devices, so far.

The difference is that these Samsung devices aren't fully there. They don't show up in /proc/partitions or in /dev/disk/by-uuid, dmesg doesn't show sizes for them, and, most important, they can't be mounted by UUID from an fstab entry, like

UUID=62B0-C667   /droidsd    vfat   user,noauto,exec,fmask=111,shortname=lower 0 0

I found that if I mounted it explicitly using the device pathname -- mount /dev/sdf /mnt -- then subsequently the device shows up normally, even after I unmount it. So I could check dmesg to find the device name, mount it as root, unmount as root, then mount it again as myself using my fstab entry. What a pain!

A kernel expert I asked thought it looked like the Samsung is pretending to be a removable device, but only "plugging in" when the system actually tries to access it. Annoying. So how do you get Linux to "access" it?

Udev: still an exercise in frustration

The obvious solution is a udev rule. Some scrutiny of /lib/udev/rules.d/60-persistent-storage.rules found some rules that did this intriguing thing: IMPORT{program}="ata_id --export $tempnode".

Naturally, this mysterious ata_id is undocumented. It's hidden in /lib/udev/ata_id, and I found this tiny ata_id man page online since there's none available in Ubuntu. Running ata_id /dev/sdf seemed to do what I needed: it made the device show up in /proc/partitions and /dev/disk/by-uuid, and after that, I could mount it without being root.

I created a file named /etc/udev/rules.d/59-galaxy.rules, with the rule:

KERNEL=="sd[b-g]", SUBSYSTEMS=="usb", ATTRS{idVendor}=="04e8", SYMLINK+="galaxy-%k-%n", IMPORT{program}="ata_id --export $tempnode"

When I tested it with udevadm test /block/sdf, not only did the rule trigger correctly, but it actually ran ata_id and the device became visible -- even though udevadm test states clearly no programs will be run. How do I love udev -- let me count the ways.

But a reboot revealed that udev was not actually running the rule when I actually plugged the Galaxy in -- the devices did not become visible. Did I mention how much I love udev?

Much simpler: a shell alias

But one thing I'd noticed in all this: side by side with /dev/disk/by-uuid is a directory called /dev/disk/by-id. And the Samsung devices did show up there, with names like usb-Android_UMS_Composite_c197022a2b41c1ae-0:0.

Faced with the prospect of spending the rest of the day trying random udev rules, rebooting each time since that's the only way to test udev, I decided to cheat and find another way. I made a shell alias:

alias galaxy='sudo sh -c "for d in /dev/disk/by-id/usb-Android*; do /lib/udev/ata_id --export \$d; done"'

Typing galaxy will now cause the Samsung to register both devices; and from then on I can mount and unmount them without needing root, using my normal fstab entries.

Update: This works for the Nook's main storage, too -- just add x/dev/disk/by-id/usb-B_N_Ebook_Disk* to the list -- but it doesn't work reliably for the Nook's SD card. The SD card does show up in /dev/disk/by-id along with main storage; but running ata_id on it doesn't make its UUID appear. I may just change my fstab entry to refer to the /dev/disk/by-id device directly.

Quick Guide to Android ADB

I've been fiddling with several new Android devices, which means I have to teach myself how to use adb all over again.

adb is the Android Debug Bridge, and it's great for debugging. It lets you type commands on your desktop keyboard rather than tapping them into the soft keyboard in Android's terminal emulator, it gives you a fast way to install apps, and most important, it lets you get Java stack traces from crashing programs.

Alas, the documentation is incomplete and sometimes just plain wrong. Since I don't need adb very often, I always forget how to use it between sessions, and it takes some googling to figure out the tricks. Here are the commands I've found most helpful.

Start the server

First you have to start the adb, and that must be done as root. But adb isn't a system program and probably lives in some path like /home/username/path/to/android-sdk-linux_x86/tools. Even if you put it in your own path, it may not be in root's. You can probably run it with the explicit path:

$ sudo /path/to/android-sdk-linux_x86/tools/sudo adb start-server

# export PATH=$PATH:/path/to/android/android-sdk-linux_x86/tools
# adb start-server

If you're also running eclipse, that probably won't work the first time, because eclipse may also have started an adb server (that gets in the way when you try to run adb manually). if you don't see "* daemon started successfully *", try killing the server and restarting it:

# adb kill-server
# adb start-server
* daemon not running. starting it now on port 5037 *
* daemon started successfully *

Keep trying until you see that "* daemon started successfully *" message.

Connecting

$ adb usb

Occasionally, this will give "error: closed". Don't panic -- sometimes this actually means "I noticed something connected on USB and automatically connected to it, so no need to connect again." It's mysterious, and no one seems to have an explanation for what's really happening. Anyway, try running some adb commands and you may find you're actually connected.

Shell and install

The most fun is running an interactive shell on your Android device.

$ adb shell

You can also install apps. On some devices (like the Nook, where I haven't found a way to allow install from non-market sources), it's the only way to install an apk file.

$ adb install /path/to/appname.apk

If the app is already installed, you'll get an error. Theoretically you can also do adb uninstall first, but when I tried that it just printed "Failure". But you can use -r for "reinstall":

$ adb install -r /path/to/appname.apk

There is no mention of either uninstall or -r in the online adb documentation, though adb -h mentions it.

Update: To uninstall, you need the full name of the package. To get the names of installed packages (another undocumented command), do this: adb shell pm list packages

Debug crashes with logcat

Finally, for debugging crashes, you can start up a logcat and see system messages, including stack traces from crashing apps:

$ adb logcat

Logcat is great for fixing reproducible crashes. Sadly, it's not so good for random/sporadic crashes that happen during the course of using the device.

You're supposed to be able to do adb logcat -s AppName if you're only interested in debugging messages from one app, but that doesn't work for me -- I get no output even when the app runs and crashes.

Reading, converting and editing EPUB ebooks

There are tons of places to get EPUB ebooks -- I won't try to list them all, but Project Gutenberg is a good place to start. The next question was how to read them.

Reading EPUB books: Aldiko or FBReader

I've already mentioned Aldiko in my post on Android as an RSS reader. It's not so good for reading short RSS feeds, but it's excellent for ebooks.

But Aldiko has one fatal flaw: it insists on keeping its books in one place, and you can't change it. When I tried to add a big technical book, Aldiko spun for several minutes with no feedback, then finally declared it was out of space on the device. Frustrating, since I have a nearly empty 8-gigabyte micro-SD card and there's no way to get Aldiko to use it. Fiddling with symlinks didn't help.

A reader gave me a tip a while back that I should check out FBReader. I'd been avoiding it because of a bad experience with the early FBReader on the Nokia 770 -- but it's come a long way since then, and FBReaderJ, the Android port, works very nicely. It's as good a reader as Aldiko (except I wish the line spacing were more configurable). It has better navigation: I can see how far along in the book I am or jump to an arbitrary point, tasks Aldiko makes quite difficult. Most important, it lets me keep my books anywhere I want them. Plus it's open source.

Creating EPUB books: Calibre and ebook-convert

I hadn't had the tablet for long before I encountered an article that was only available as PDF. Wouldn't it be nice to read it on my tablet?

Of course, Android has lots of PDF readers. But most of them aren't smart about things like rewrapping lines or changing fonts and colors, so it's an unpleasant experience to try to read PDF on a five-inch screen. Could I convert the PDF to an EPUB?

Sadly, there aren't very many open-source options for handling EPUB. For converting from other formats, you have one choice: Calibre. It's a big complex GUI program for organizing your ebook library and a whole bunch of other things I would never want to do, and it has a ton of prerequisites, like Qt4. But the important thing is that it comes with a small Python script called ebook-convert.

ebook-convert has no built-in help -- it takes lots of options, but to find out what they are, you have to go to the ebook-convert page on Calibre's site. But here's all you typically need

ebook-convert --authors "Mark Twain" --title "Huckleberry Finn" infile.pdf huckfinn.epub

ebook-convert infile.pdf huckfinn.epub --authors "Mark Twain" --title "Huckleberry Finn"

Pretty easy; the only hard part is remembering that it's --authors and not --author.

Calibre (and ebook-convert) can take lots of different input formats, not just PDF. If you're converting ebooks, you need it. I wish ebook-convert was available by itself, so I could run it on a server; I took a quick stab at separating it, but even once I separated out the Qt parts it still required Python libraries not yet available on Debian stable. I may try again some day, but for now, I'll stick to running it on desktop systems.

Editing EPUB books: Sigil

But we're not quite done yet. Calibre and ebook-convert do a fairly good job, but they're not perfect. When I tried converting my GIMP book from a PDF, the chapter headings were a mess and there was no table of contents. And of course I wanted the cover page to be right, instead of the default Calibre image. I needed a way to edit it.

EPUB is an XML format, so in theory I could have fixed this with a text editor, but I wanted to avoid that if possible.

And I found Sigil. Wikipedia claims it's the only application that can edit EPUB books.

There's no sigil package in Ubuntu (though Arch has one), but it was very easy to install from the sigil website.

And it worked beautifully. I cleaned up the font problems at the beginnings of chapters, added chapter breaks where they were missing, and deleted headings that didn't belong. Then I had Sigil auto-generate a table of contents from headers in the document. I was also able to fix the title and put the real book cover on the title page.

It all worked flawlessly, and the ebook I generated with Sigil looks very nice and has working navigation when I view it in FBReaderJ (it's still too big for Aldiko to handle). Very impressive. If you've ever wanted to generate your own ebook, or edit one you already have, you should definitely check out Sigil.

Getting rid of extra whitespace from Eclipse

I found lots of suggestions on the web for eliminating extra whitespace, and several places to configure this within Eclipse, but most of them don't do anything. Here's the one that actually worked:

Window->Preferences
Jave->Editor->Save Actions
Enable Perform the selected actions on save.
Enable Additional actions.
Click Configure.
In the Code Organizing tab., enable Remove trailing whitespace for All lines.
Review all the other options there, since it will all happen automatically whenever you save -- make sure there isn't anything there you don't want.
Dismiss the Configure window.
Review the other options under Save Actions, since these will also happen automatically now.
Don't forget to click Apply in the Save Actions preference page.

Whew! There are other places to set this, in various Code style and Cleanup options, but all all the others require taking some action periodically, like Source->Clean up...

By the way, while you're changing whitespace preferences, you may also want the Insert spaces for tabs preference under General->Editors->Text Editors.

An easy way to check whether you've succeeded in exorcising the spaces -- eclipse doesn't show them all, even when you tell it to -- is to :set hlsearch in vim, then search for a space. (Here are some other ways to show spaces in vim.) In emacs, you can M-x set-variable show-trailing-whitespace to true, but that doesn't show spaces on blank lines; for that you might want whitespace.el or similar packages.

Developing for Android

I wrote yesterday about my quest for an app for reading news feeds and other timely information from the web. And how existing ebook readers didn't meet that need. That meant I would have to write something.

Android development is done in Java, using Eclipse as an IDE. Let me just state up front that (a) I dislike Java (and have forgotten most of what I once knew about it) and (b) I hate IDEs -- they make you use their crippled editor instead of your own, they control what you can put where on the screen, and they're always popping up windows that get in your way.

Okay, not an auspicious beginning. But let's try to be open-minded, follow the instructions and see what happens.

I installed Eclipse from eclipse.org after being advised that the version on Ubuntu is out of date. Then I installed all the various Android plug-ins and SDKs and set them up (there is no single page that lists all the steps, so I did a lot of googling). It took maybe an hour or so to get it all installed to the point where it could produce its "Hello world".

And then ... wow! Hello world worked right off the bat, and the emulator worked for basic testing. Hmm, okay ... how about if we use HTML as a format ... is there a standard HTML display component? Sure enough -- I added a WebView to my app and right away I had a working HTML reader. Okay, how about a row of buttons and a status bar on top? Sure, no problem.

The standard Android online docs aren't great -- they're a wonderful example of how to write seemingly comprehensive documentation that somehow manages to tell you nothing of what you actually need to know. But that's not as bad as it sounds, because there are lots of forums and tutorials to fill in the gaps. Stack Overflow is particularly good for Android tips.

And yes, I did some swearing at Eclipse and spent too much time googling how to disable features, like the "Content Assist" that periodically freezes the whole UI for a minute or so in the middle of your typing a line of code, while it thinks about some unhelpful and irrelevant hints to offer you in a popup. Turn it off in the prefs under Java/Editor. (Eclipse's actual useful hints, like the ones you get when you hover over something that's red because of an error, will still work. I found them very helpful.)

More specifically: Java/Editor/Content Assist/Hovers, and turn off Combined Hover and maybe anything else that happens without a modifier key. You probably also want to turn off Enable Auto Activation under Java/Editor/Content Assist. And possibly others -- I kept turning things off until the popups and delays went away, and I haven't found anything explaining how all these parameters relate.

Okay, so there were snags, and it's frustrating how there are almost no open source apps for this open source OS. (Yes, my app will be.) But here's the thing: in about 4 days, starting from nothing, I had a little RSS reader that did everything I needed. I've been adding features since then. Android doesn't have a reasonable battery status monitor? Fine, my reader can show battery percentage in the status bar. Android doesn't dim the screen enough? Fine, I can dim it further inside the application (an idea borrowed from Aldiko).

After less than a week of work I have an RSS reader that's better than my Palms running Plucker ever were. And that says a lot about the ease of the Android programming environment. I'm impressed!

Update: The source, an apk, and a brief discussion of how I use my feed reader are now here: FeedViewer.

Android tablet as an ebook/RSS reader

I've been searching for years for something to replace the aging and unsupported Palm platform. I've been using Palms for many years to read daily news feeds; first on the proprietary Avantgo service, but mostly using the open source Plucker.

I don't normally have access to a network when I'm reading -- I might be a passenger in a car or train, in a restaurant, standing in line at the market, or in the middle of the Mojave desert. So I run a script once a day on a network-connected computer to gather up a list of feeds, package it up and transfer it to the mobile device, so I have something to read whenever I find spare time.

For years I used Sitescooper on the host end to translate HTML pages into a mobile format, and eventually became its primary maintainer. But that got cumbersome, and I wrote a simpler RSS feed reader, feedme.

But on the reader side, that still left me buying old PalmOS Clies on ebay. Surely there was a better option?

I've been keeping an eye on ebook readers and tablets for a while now. But the Plucker reader has several key features not available in most ebook reader apps:

1. An easy, open-source way of automatically translating RSS and HTML pages into something the reader can understand;
2. Delete documents after you've read them, without needing to switch to a separate application;
3. Random access to document, e.g. jump to the beginning or end, or 60% in;
4. Follow links: nearly all RSS sites, whether news sites or blogs, are set up as an index page with links to individual story pages;
5. Save external links if you click on them while offline, so you can fetch them later.

Most modern apps seem to assume either (a) that you'll be reading only books packaged commercially, or (b) that you're reading web pages and always have a net connection. Which meant that I'd probably have to roll my own; and that pointed to Android tablets rather than dedicated ebook readers.

Android as a reader

All the reviews I read pointed to Aldiko as the best e-reader on Android, so I installed it first thing. And indeed, it's a wonderful reader. The font is beautiful, and you can adjust size and color easily, including a two-click transition between configurable "day" and "night" schemes. It's easy to turn pages (something surprisingly difficult in most Android apps, since the OS seems to have no concept of "Page down"). It's easy to import new documents and easy to delete them after reading them.

So how about those other requirements? Not so good. Aldiko uses epub format, and it's possible (after much research) to produce those using ebook-convert, a command-line script you can get as part of the huge Calibre package. Alas, Calibre requires all sorts of extraneous packages like Qt even if you're never going to use the GUI; but once you get past that, the ebook-convert script works pretty well.

Except that links don't work, much. Sometimes they do, but mostly they do nothing. I don't know if this is a problem with Calibre's ebook-convert, Aldiko's reader, or the epub format itself, but you can't rely on links from the index page actually jumping anywhere. Aldiko also doesn't have a way to jump to a set point, so once you're inside a story you can't easily go back to the title page (sometimes BACK works, sometimes it doesn't).

And of course there's no way to save external links for later.

So Aldiko is a good book reader, but it wouldn't solve my feed-reading problem.

And that meant I had to write my own reader, and it was time to delve into the world of Android development. And it was surprisingly easy ... which I'll cover in a separate post. For now, I'll skip ahead and ruin the punch line by saying I have a lovely little feed-reading app, and my Archos and Android are working out great.

Archos 5 Android Tablet review

For the past couple weeks I've been using a small Android tablet, an Archos 5. I use it primarily as an ebook and RSS feed reader (more about that separately), though of course I've played with assorted games and other apps too.

I've been trying to wait for the slew of cheap Android tablets the media assure us is coming out any day now. Except "any day now" never turns into "now". And I wanted something suitable for reading: small enough to fit in a jacket pocket and hold in one hand, yet large enough to fit a reasonable amount of text on the screen. A 4-5-inch screen seemed ideal.

There's nothing in the current crop fitting that description, but there's a year-old model, the Archos 5. It has a 4.8-inch screen, plus some other nice hardware like GPS. And it seems to have a fair community behind it, at archosfans.com.

I have the 16G flash version. I've had it for a couple of weeks now and I'm very happy so far. I'm not sure I'd recommend it to a newbie (due to the Android Marketplace's ban on tablets -- see below), but it's a lovely toy for someone fairly tech savvy.

My review turned out quite long, too long for a blog post. So if you're interested in the details of what's good and what's bad, you'll find the details in my Archos 5 Android Tablet review.

Android/Eclipse Spellchecker is a bit confused

What do you think -- should I change all the "Android" references to "Undried"?