• The battery and power management needs some work. Like all smartphones, the Desire has plenty of battery-sucking components, such as GPS and WiFi. I spend more time than I should really have to turning these off and on manually to conserve power. For example, Google Maps should be able to turn the GPS on by itself, rather than rely on me to do it. Most software based solutions, such as Locale, JuiceDefender, and so on, simply don’t work reliably enough (for example, mobile data connections frequently won’t turn off or on): it’s obvious that Android isn’t exposing enough APIs and these applications therefore have to rely on hacks. Better built-in power management would be welcome.
• The volume switch is really annoying. In theory, it’s nice to have a hardware control. But I find myself hitting it accidentally when holding the phone, and reducing the ring volume down to vibrate. Sometimes I don’t even notice. I’d prefer to see a soft volume control, less easy to hit accidentally.
• The openness of the platform is in question. I can install whatever applications I want. However, I still have firmware on the phone that’s been mangled by both HTC and (in my case) Orange. The HTC modifications are fairly nice, but the pre-installed Orange applications are just irritating, and cannot be removed easily. Orange has a bad habit of mangling phones they ship; presumably they think they need to do this to ‘differentiate’ themselves.  Worse still, it turns out one can’t simply ‘reinstall’ the firmware: the closest process is to root the phone – an awkward and unsupported process I’ve yet to be brave enough to attempt.
• Many of the pre-installed widgets are far too large (witness the SMS widget, which consumes an entire screen), with a lot of unnecessary chrome.
• The Music app is a bit flaky and crashes once or twice a day.

After 3-4 hours of research, I found the answer. Running a differencing tool between the C:\Windows\Inf folder on a colleague’s PC and mine showed that mine was missing the usb.inf and usb.PNF files from that folder. It wasn’t clear why they were missing, but having copied them over from his machine, the driver installation then worked perfectly. So that’s worth trying if you encounter a similar problem with this product (or indeed any USB HID device).

#!/usr/bin/perl
#
# Usage:
#  backup-google-reader-opml file-to-write-to.opml google.user.name@domain google-password
 
use strict;
use warnings;
 
use WWW::Mechanize;
 
my $mech = WWW::Mechanize->new();
 
$mech->get("http://reader.google.com")
    or die "Cannot reach Google Reader Homepage";
 
$mech->submit_form(
    form_number => 1,
    fields =>
    {
        Email => $ARGV[1],
        Passwd => $ARGV[2]
    }
)
    or die "Cannot submit form";
 
$mech->get("http://www.google.com/reader/subscriptions/export");
$mech->save_content($ARGV[0]);

However, I recently wrote another script (this time Python) that then takes this OPML, parses out all the URLs that are tagged with ‘podcast’, and outputs a serverlist file for podget (an automated console-based podcast downloader). This enables me to subscribe to a podcast in Google Reader, and have the podcast automatically added to the download list. The script looks like this:

#!/usr/bin/python
#
# Pass in the OPML file as the first command-line parameter. Will output the
# podget serverlist on stdout.
 
import re
import sys
import xml.dom.minidom
 
doc = xml.dom.minidom.parse(sys.argv[1])
 
body = doc.getElementsByTagName("body")[0]
 
p = re.compile('^\W+')
 
for outline in doc.getElementsByTagName("outline"):
    if outline.getAttribute("text") == "podcast":
        for subOutlines in outline.getElementsByTagName("outline"):
            title = subOutlines.getAttribute("title")
            title = p.sub("", title)
            print subOutlines.getAttribute("xmlUrl") + " NoCategory " + title

Feel free to use and adapt to your needs.

• Making a hot copy of your entire drive onto a USB hard disk with VMWare Converter beforehand is a great idea. This basically means you have your entire previous installation still bootable, so you can rescue any data or settings you forgot about, after you’ve done the reformat and reinstall. VMWare Converter is free.
• Windows XP won’t install if the BIOS has the SATA controller set to AHCI mode – it won’t find the HD to install to. Read the install instructions for the SATA driver before installing Windows. Doing this without a floppy drive involves installing Service Pack 2 first (see below).
• Windows, surprisingly, does not contain a driver that can work with the Ethernet controller. Have a copy of it ready on a USB stick.
• The redistributable version of Service Pack 3 isn’t standalone, despite claims to the contrary; it pre-reqs. at least Service Pack 1. Windows Update won’t take you the whole way there, so it’s easiest to get the SP2 and SP3 standalones.
• The USB ports will not operate at high speed until you install the Intel Chipset Device Software.

Then delicious went and released a new version and this trick broke.

After a bit of fiddling, I’ve found a reasonable alternative. feed.informer will take any RSS feed and turn it into fairly plain HTML. So take your RSS feed, which might look something like this:

http://feeds.delicious.com/v2/rss/andrewferrier/mtag?count=100

(the count increases the maximum number of items in the feed to 100)

Now plug it into feed.informer, and edit the options to make the result look how you want (hint: I modify the template, and set the Per-Item Template to <a href=”%URL%”>%TITLE%</a><br />, leaving the rest of the template blank – this prints one bookmark per line). You’ll also need to sign up and create a feed.informer account whilst you’re doing this (assuming you don’t already have one).

Once done, just view the ‘HTML’ version of the digest you created, and set that as your mobile homepage. Bingo.

• To add nice headers and footers to each page use the fancyhdr package.
• Use glosstex for most acronym and glossary needs.
• Use the hyphenat package if you need monospace text (such as that set by the \texttt command) to wrap correctly, with hyphenation marks.
• The varioref package provides easy-to-use cross referencing with automatic phrases such as ‘on the following page…’ etc.
• To include EPS easily in both latex and pdflatex documents, use the following magic incantation in the document preamble:

\newif\ifpdf
\ifx\pdfoutput\undefined
    \pdffalse
\else
    \pdfoutput=1
    \pdftrue
\fi
 
\ifpdf
   \usepackage{hyperref}
   \usepackage[pdftex]{graphicx}
   \DeclareGraphicsExtensions{.pdf,.png,.jpg}
\else
   \usepackage{graphicx}
   \DeclareGraphicsExtensions{.eps}
\fi

Then refer to graphics files, when including them, without an extension. Use epstopdf to convert all EPS files to PDF for inclusion in the pdflatex version of the document.

• Use the custom-bib package to easily create custom BiBTeX style files.
• The following command produces a new environment which gives a nice way of having an introductory paragraph for each new chapter:

\newenvironment{intro}{\sffamily}
{\vspace*{2ex minus 1.5ex}}

Fortunately, underlying Firefly is a sqlite database, which can be queried using standard SQL syntax. This enables a technique of creating static playlists that are automatically re-generated periodically instead.

The prerequisites for the following technique are:

• Perl, with the File::Spec module (to convert from absolute paths to relative ones, which is what Firefly expects).
• The sqlite3 command-line interface.

The three commands that follow will create a standard .m3u playlist with the top 100 most-played songs from Firefly’s database, and another playlist with all the non-Podcasts added in the last month, ordered by the time they were added. Neither of these are possible using Firefly’s query language.

sqlite3 /var/cache/mt-daapd/songs3.db 'select path from songs order by play_count desc limit 100' | perl -nle 'require File::Spec; $_ = File::Spec->abs2rel($_, "$PLAYLIST_DIR"); print;' > "$PLAYLIST_DIR/Most-played songs.m3u"
 
MONTHAGO=$(perl -e 'use Date::Calc::Object qw(:all); $date = Date::Calc>now(); $date += [0,-1,0,0,0,0]; print $date->mktime();')
 
sqlite3 var/cache/mt-daapd/songs3.db "select path from songs where genre!='Podcast' and time_added > $MONTHAGO order by time_added desc" | perl -nle 'require File::Spec; $_ = File::Spec->abs2rel($_, "$PLAYLIST_DIR"); print;' > "$PLAYLIST_DIR/Music added in last month by most recent.m3u"

(obviously, if you use these, you’ll need to alter paths to suit, make sure the correct Perl modules are installed, remove line breaks to make it easier to read, etc.)

Firefly will read these .m3us if configured correctly during its next rescan, and use them as it would any other playlists. You can force a rescan with the following wget command:

wget --delete-after -q --http-user noone --http-password yourpasswd "http://localhost:3689/config-update.html?action=rescan"

Although not fully dynamic (they are not generated on request from the Soundbridge), if these commands are called from cron or similar, the playlist can be kept up-to-date ‘enough’.

wpa_passphrase YOURSSID YourWPAPassPhrase > /etc/wpa_supplicant/wpa_supplicant.conf
wpa_supplicant -ieth1 -c /etc/wpa_supplicant/wpa_supplicant.conf &
ifup eth1
pump -i eth1

Replace eth1 in the list above with whatever network interface name Knoppix has assigned to your wireless card (reading through the output of dmesg should help you with this). Don’t forget to double-check that your router has the MAC address of the card (use ifconfig to find this) if you’re using MAC address filtering.

It took me a while to figure out how to get my data out of the Palm. Palm don’t provide a decent export to anything other than Palm formats for the datebook, so a third-party exporter was called for. The web is seemingly full of dodgy Windows shareware to do the job, but jpilot (which I already, but rarely, use on my Linux machine) turned up trumps. It exports directly to the modern iCalendar standard, fully supported by Google Calendar. Hurrah! Since this solution doesn’t seem to be well-documented, I thought I’d write it down.

This is when I wish I’d done Chemistry A-Level rather than Further Maths.

Update 2007-01-13: I should add that some feedreaders will treat all the items in the feed as new, because the GUID will have changed. Just mark them all as read. Apologies for the inconvenience.

It’s normally recommended to include the line if error_message then finish endif in your .forward filter file, to make sure error messages don’t cause recursive problems in your mail system. I have found that this doesn’t work in practice if you normally receive a lot of spam, because spammers are increasingly using your email address to spam other people. This causes bounces back to you, which bypass your spam filter because of that line (at least in my setup, using sa-exim). Removing that line greatly reduced the amount of spam reaching my inbox.

As the more capable e-mail spammers seem to be figuring out that anti-spam technologies are getting pretty good at filtering out the crap they send, normally using rule-based detection, Bayesian learning, or a combination of the two, a lot of spam now being sent out is image-based – and anti-spammers are now using OCR to fight back against this new tide.

As I’ve mentioned before, I have a huge spam problem on my personal e-mail account (~4,000/week) – due to a combination of bad luck and some foolish naivety at a few points – and so I have a fairly highly-tuned SpamAssassin installation running at home, with plenty of custom rules and plugins. I’ve seen a rising amount of image spam on it, so I decided to give FuzzyOcr, a plugin for SpamAssassin, a try. So far, the results are pretty impressive. FuzzyOcr uses the open-source gocr program as the engine, and ties it to with SpamAssassin and some logic. The OCR is fairly CPU-intensive, so unlike most SpamAssassin plugins, it only kicks in if the message is otherwise going to be below a certain scoring threshold. So far it has roughly halved the volume of spam that slips through into my inbox (previously ~40-50/day), which is a welcome improvement.

However, fun though they are as a technical challenge, technical approaches such as these always feel like fighting a losing battle. I might write a lengthier article on this at a later date, but I’d like to see ISPs take a far more hardline attitude with their peers that host spammers. There are also compelling economic solutions to the problem, mostly related to micro-payments for sending email. There are problems with those too (how do you roll them out gradually?), but you rarely see graphs of spam that have a downward trend – a solution to the spam problem would be most welcome.

A few weeks ago I replaced some of the bulbs in my house with energy-saving ones. However, the ceiling light in my hall behaves in a very odd manner. Occasionally, after I switch it off at the wall, it flickers on very briefly (for about 1/10 second) about once every minute – even though the power is (allegedly) off. The flicker is fairly dim, so I only notice it at night. If I take the bulb out of the socket, the flicker stops. If I put it back, it starts again. This behaviour happily continues for hours – to the extent that I remove the bulb when it happens because it’s too distracting when trying to sleep.

Perhaps it’s some kind of residual charge in the bulb. But this doesn’t really seem to explain why it only flickers when the bulb is in the socket (even though the switch is off). It also doesn’t explain why it doesn’t happen in the rest of the house (they have the same brand of bulb). The only difference is that the hall has two switches – but they aren’t dimmer switches or anything special.

Any thoughts?

(Of course, once a year, I lose an hour – but I prefer not to talk about that…)