Softwareengineering

I use Firefly (previously called mt-daapd) as a media server for my Roku Soundbridge. It has a feature called ‘Smart Playlists’ that dynamically create playlists based on certain criteria, but they aren’t that powerful - they don’t support sorting or other more advanced query features.

Fortunately, underlying Firefly is a sqlite database, which can be queried using standard SQL syntax. This enables a technique of creating static playlists that are automatically re-generated periodically instead.

I’ve thought for a while that build, source-code management, and bug tracking software (which I’m collectively calling meta-software) could, and should, be so much simpler. I’ve written previously about my contention that bugs and features are the same thing, but the problem is wider. Software has a tendency to acquire features over time, and software that’s used to make other software is no exception. Here are some assorted thoughts about how to improve the situation:

I’ve written about synchronicity vs. asynchronicity before, but I wanted to revisit the subject because it seems to be so key to modern services; as more and more communication mechanisms evolve out of available technology and entrepreneurs’ imagination, understanding customer’s usage patterns will be important when developing businesses around them. An excellent article by Gregor Hohpe, Starbucks Does Not Use Two-Phase Commit (included in Joel Spolsky’s Best Software Writing Vol. 1), is an examination of why understanding computer science concepts such as 2PC (and, I would argue, synchronicity) is important when engaging in business process engineering. There’s a large overlap between business and software engineering here, and this is why IBM sells products like WebSphere Process Server together with business consultants to help customers implement them. There are a number of other essays in Spolsky’s excellent book which also discuss related subjects.

It’s strange how the same techniques can be used to attack both sides of a problem. For some time now, some of the more sophisticated web spammers have been using OCR techniques to circumvent CAPTCHAs on websites in order to hijack free email accounts, submit comment spam on blogs, and similar forms of mischievousness.

As the more capable e-mail spammers seem to be figuring out that anti-spam technologies are getting pretty good at filtering out the crap they send, normally using rule-based detection, Bayesian learning, or a combination of the two, a lot of spam now being sent out is image-based - and anti-spammers are now using OCR to fight back against this new tide.

Google have created a powerful brand based on creating simplicity from complexity (what all good IT is about). Their tools aren’t perfect, but they’ve made life easier for billions, and so I think they still deserve some free feedback from time-to-time. So, a few thoughts:

• Mr. Google, please develop a podcast search engine. So much interesting content is now being released as podcasts (quick plug for my favourite: EconTalk), that it would be useful to be able to search them. All you have to do is invent a speech-to-text interpreter that actually works reliably. Simple. [Note: as I sometimes do, I wrote this post in advance of it being published. I’ve since discovered that such a tool already exists. However, I thought I’d leave the original prose here: Google, if you get one out soon, you could still corner the market]

  More ...

Marchitecture. I shamelessly stole this from a presentation I attended the other day (names withheld to protect the innocent). If it resonates with you, it probably doesn’t need explaining, but marchitecture is IT architecture that is used for marketing reasons rather than technical ones. Sometimes the marchitecture looks the same as the ‘real’ architecture, sometimes not. Wikipedia’s definition seems a bit narrow (I’m not sure what electronic architecture is anyway), but hey. No original research seems to one of the more widely violated Wikipedian principles.

IBM Hursley invited three final-year MEng students from Imperial College to give us presentations on their individual MEng projects today (mine, from several years ago, can be found here). They were:

• Marc Hull, who talked about his project on Balancing simplicity and efficiency in web applications. Marc’s work focused on improving the development of stateful web applications, and in particular on object-relational mapping in Java, in an attempt to allow more straightforward persistence of objects to databases. This has always seemed to me to be an area lacking in usability and ease (see J2EE for plenty of examples), so anything that moves us closer is welcome.

  More ...

One of the things I’ve felt IBM’s been strong at in recent years is the way we embrace open-source as a development model, and as a model for providing software to our customers. Eclipse, which has a lot of support from IBM, is a well-known example, but there are plenty of others. I honestly believe that the provision of open-source software gives us a competitive edge over IT organisations of comparable size. Sure, we don’t do it for all our products, but for toolchains like Eclipse it has delivered real benefits - the number of plugins developed for it is testament to that.

Joe McKendrick discusses SOA and reuse in a recent blog entry, essentially drawing on some comments from David Chappell that reuse didn’t do as well as predicted in the era of object-orientation, and that SOA isn’t faring well in this department either. Dave Linthicum, in his latest podcast, also discusses this topic.

I’m not sure I can comment that widely on the state of current SOA projects, and I would agree that SOA may suffer from similar management problems to that of object-orientation: if developers of SOA systems aren’t rewarded for saving time with a reuse strategy, they won’t be enthused to do so. This is an important part of any software project, and encouraging reuse is a best practice that shouldn’t be restricted to object-orientation or SOA.

I spent last Thursday and Friday in London at the Google offices in Victoria for the first Google Test Automation Conference. The presentation topics ranged widely, considering the relatively narrow scope of the conference, but most were well developed and interesting, even if some retrod familiar topics. Some of the highlights included:

• Steve Loughran and Julio Guijarro, HP Labs. This presentation was about Smartfrog, a system deployment framework, which Steve and Julio were working on as part of a strategy for system testing. They demonstrated several examples of how the system might work in practice. Smartfrog looks pretty flexible, and I plan to spend some time looking into it. Frameworks for deployment have an inherent problem in catering to the wide variety of platforms, configuration mechanisms, deployment combinations and so on that are necessary in practice. Anything that gets closer to this is therefore welcome. Smartfrog also has the interesting property that the XHTML it produces as output is sufficiently well-formed that, although it has an embedded CSS stylesheet for presentation in a web browser, it can also be parsed as XML data without much effort, and thus act as a machine-readable data source as well. This might seem obvious to some folks, and I’m willing to bet it’s not the first time it’s been done, but it seemed novel to me.

  More ...