The latest book batch

I’ve just ordered:

  • Programming Pearls by Jon Bentley
  • Masterminds of Programming: Conversations with the Creators of Major Programming Languages by Federico Biancuzzi
  • Practical API Design: Confessions of a Java Framework Architect by Jaroslav Tulach
  • Coders at Work: Reflections on the Craft of Programming by P Seibel
  • Rapid Problem Solving with Post-it Notes by David Straker
  • Dependency Injection: With Examples in Java, Ruby, and C# by Dhanji Prasanna

To put a bit of (positive) pressure on myself I hereby pledge to read and write reviews for these at a rate of two books per month! 🙂

Restoring PC keyboard sanity on OS X

Purist Mac-heads thinks that pressing Alt+Shift ⇑+8, Alt+Shift ⇑+9 and Alt+7 to produce {, } and | respectively is just the way of nature. As a Mac OS X user and programmer of C syntax programming languages I beg to differ. Personally I very much prefer the less finger acrobatic Alt+7, Alt+9 and Alt Gr+< routines I’m familiar with from the Windows and Linux world for producing the same characters.

I’d not define myself as a Mac-head, my background is mainly in PC:s and Linux even though a SSD-equipped Mac Mini is my primary at-home-computer. OS/X is elegant, fast and has that so desirable it-just-works quality that even current state of the art Linux distributions seem to not yet fully have achieved. Hooking up a PC-keyboard with a Swedish layout to the Mini just works – as expected. But for the the left curly brace, the right curly brace and the pipe character the just works ends there if your are not willing to submit to the arcane finger acrobatics described above.

The fix

  1. Download Ukulele.
  2. Ukulele is a keyboard layout editor, but for our purposes we do not need to edit anything, just grab the .keylayout file for your locale from the Logitech Keyboard Layouts directory of the Ukele disk image and copy it to ~/Library/Keyboard Layouts
  3. Tick the keyboard layout in the Input Sources tab of the Language & Text pane in System Preferences.
  4. Rejoice – curly brace and pipe character frustrations are now a thing of the past!

N.B. the keys mentioned in this post are those found on a PC keyboard with a Swedish language keyboard layout. The general problem described is likely similar for other Nordic and European keyboard layouts, even though there may be some differences to the character↔key mapping.

4 cool things you can do with a distributed version control system

With the Git-craze currently flourishing it’s hard to see that there’s competition in distributed versions control systems (DVCS) space. In fact there is a strong contender that I’ve had the privilege of getting to know more intimately during the last couple of months. Mercurial was born around the same time as Git, in 2005 when an open and free distributed version control system for maintaining the Linux kernel was in urgent need. It was finally Git that got the honor, but that doesn’t mean Mercurial hasn’t been entrusted any projects of significance. Among the Mercurial users today are OpenJDK, Netbeans and Mozilla. Open source project hosting provider Google Code adopted Mercurial rather than Git after careful analysis.

I’m a bit of “late adopter” when it comes to DCVS:s, but in the unlikely case there is even later adopters – here’s a brief summary of the whole “distributed” thing:  version control systems are something all developer are (should?) be very familiar with. The most common ones such as old rusty CVS and the more modern Subversion are based on the client-server model – developers work on a working copy of a project stored on a central repository on a server. The client working copy is updated from the remote repository and the fruits of the developer’s labor are committed the other way around. The central repo stores all revisions of the project ever committed including commit comments, tags and branches. The working copy is essentially little more than a particular revision from the central repo plus the developer’s outstanding uncommitted changes against this version.

Enter distributed version systems.  The idea is that developers no longer work with plain working copies of projects, instead they work with full repositories. These distributed repositories are no longer heavy weight, singular entities on the lines of a Subversion server, rather they are small manageable things that lives beside the working copy in the file system (in the case of Mercurial in the .hg sub directory of the working copy folder). The repo can easily be zipped up and mailed somewhere. DVCS:s however provide native functionality for cloning repos between different computers (over a network) or just between different file paths on the same machine.  Developers still commit and update the working copy as usual but the counter part is now the private local repo. Changes to the repo are pushed to other repos (again over the network or on the same machine). The other way around is known as a pull. There is no pre-determined repository topology, developers are free to push and pull between different repos in any direction, at any time.

A distributed version system is a generalization of your typically vanilla client-server version system. You could use a DVCS in a completely centralized way, always pushing changes to the local repo to a central repo after each commit. A distributed version control system however lends itself to a distributed development process well suited for open source projects, and provides some other cool possibilities:

1. Carry around entire projects, including the complete commit history, tags and branches, on a flash stick. Ideal in academic/corporate environments where restrictive firewalls makes it problematic to connect to remote repositories.

2. Work the same project with two different IDE:s. Having two of the Java heavy-weights (Eclipse, Netbeans or IntelliJ) share project folder is not something I’d recommend.  A better approach is to clone separate repos for each IDE and push/pull deltas between them to heart’s content.  I’ve used this approach with Java Swing apps – utilizing the excellent GUI builder Matisse in Netbeans to lay out the screens while doing the rest of the coding in my preferred IDE, Eclipse.

3. Speed up deployments. Modern web application frameworks tend to have many dependencies and thus produce voluminous deployment artifacts. On a typical ADSL-hookup a 50MB WAR (a realistic figure for a Grails app the uses a couple of plugins) could take several minutes to upload to a deployment server. A better approach is to clone a repo on the deployment server(s) and simply push the delta since the previous deploy. The WAR can then be built on the server faster than it would have taken to build it locally and then upload it.

4. Add version control to anything, anywhere with ease. Transforming an existing directory structure into the working copy of a full-blown pushable, taggable, revertable and diffable repository is trivial and unobtrusive. This can be immensely powerful.

Want to start tracking the system configuration of a Unix system? Simply:

~# cd /etc
/etc# hg init

Changed your mind and want to revert to the previous non-version-controlled state and remove all traces of the version control?

/etc# rm -rf .hg/

The 10 minute guide to distributed version control with Mercurial

Mercurial is a piece of distributed version control (DVCS) software which is remarkably easy to work with from the command line¹. Users of the grand-daddy of version control software, CVS, or its follow-up – Subversion, will be pleased to notice that the Mercurial developers have gone to great length to provide a similar syntax. Where Mercurial deviates it’s mainly due to the nature of DVCS which brings with it a couple of concepts not used in traditional client/server version control systems.

This guide is designed to give a DVCS novice a feel for distributed version control in general and Mercurial in particular in under 10 minutes. Unix and client/server version control familiarity is assumed.

800px-Pouring_liquid_mercury_bionerd

The Mercurial executable is named for the chemical symbol of the element mercury - Hg. Photo credit: Bionerd

Start with installing Mercurial:
Visit the Mercurial download page and grab an installation package of your choice.

Create a repository:
~# mkdir repo1

~/repo1# cd repo1
~/repo1# hg init

Add a file to the repo:
~/repo1# touch file.txt
~/repo1# hg add file.txt

Commit the file to the repo:
~/repo1# hg commit

Clone the repo:
~/repo1# cd ~
~/# hg clone repo1 repo2

Make a change in the working copy of the cloned repo and commit it:
~/# cd ~/repo2
~/repo2# echo "DOH" > file.txt
~/repo2# hg commit

Push the change to repo1:
~/repo2# hg push

Update repo1 with the delta from repo2:
~/repo2# cd ~/repo1
~/repo1# hg update

Let’s try pulling changes from repo1 to repo2 :
~/repo1# echo "DOH2" >> file.txt
~/repo1# hg commit
~/repo1# cd ~/repo2
~/repo2# hg pull
~/repo2# hg update

Note that good help is never far away :
~/# hg help
~/# hg help pull

For more on what you can do with with Mercurial read up on 4 cool things you can do with a distributed version control system.

¹) If you are so inclined, there are good graphical Mercurial interfaces such as TortoiseHg available.

The dark side of Grails

Grails can be an extremely enjoyable framework to work with. Some days productivity is super-high, code keeps flowing and I find new plugins which fits just like a glove. Grails automatic reloading of changed classes minimizes distractions. And then – bang! – a nasty SQLException with its accompanying 200 line stack trace is dumped on my terminal and brutally interrupts my flow.

The status of the Grails Bug Tracking system.

The status of the Grails Bug Tracking system October 13, 2009.

During my three months with Grails I’ve stumbled on several confusing bugs in the framework, the majority of which were related to the Grails’ ORM-layer GORM – particularly troubling for me personally as I’ve only limited SQL experience and practically no knowledge of Hibernate (the underpinnings of GORM) at all. Frankly speaking I’d very much prefer it to stay that way, with GORM doing the database heavy lifting behind the scenes and me knowing very little about these mundane tasks. This may be wishful thinking, it certainly is with the current (1.1) version of Grails. I’ve now adopted the routine to always check the Grails JIRA before proceeding with debugging of my own code when I encounter SQLExceptions.

Dean Del Ponte asks if bugs in grails are hurting adoption? The answer is a resounding yes, grails is simply to buggy!

But what does really “too buggy” mean? Its meaning lies of course it the eye of the beholder. Grails is not too buggy to be used in a production setting (if you are willing to accept a fair amount of hacking to get around bugs in the framework and the accompanying loss of developer productivity and morale). Grails is not too buggy for me, its advantages (convention over configuration, brevity, exciting plugins, CRUD scaffolding) far outshines the bugs. Grails is however way too buggy to provide for the smooth learning curve it really deserves.

I finish with some of my pet peeves:

  • GRAILS-2735 – Criteria.list() with paginate and sort order causes SQLException
    With us since 1.0.2, 13 watchers as of today.
  • GRAILS-4089 Default Sort Order for associations causes java.sql.SQLException: Column not found
    With us since 1.1-RC1, 16 watchers as of today.
  • GRAILS-2475 Criteria Builder causes SQL errors when using pagination and sort
    With us since 1.0, 2 watchers as of today.

The monster Mini : A study of rejuvenation through surgery

Mac mini.

The Mini is about the size of a stack of CD cases.

As a proud owner of a Mac Mini of 2007 vintage I’ve over the years grown accustomed to its elegance, quietness, stability and diminutive dimensions. It still handles causal web browsing and media center duties with Plex with grace, but heavier work e.g. Grails development in Jetbrains’ notoriously memory hungry integrated development environment IntelliJ IDEA is pushing it far out of its (and mine) comfort zone. The Mini’s specs – 1.83GHz Core Duo/1GB RAM/80GB 2.5″4200RPM HDD – are simply not up to the task.

Originally I was thinking about getting a new PC and relegating the Mini to media center duties only. But I had a hard time parting from the sympathetic little companion on my desk. Finally I went for a radical upgrade instead. The shopping list:

The SSD pushed the total up to a neighborhood where I could have gotten a pretty decent PC with a fast magnetic hard drive instead, but I was betting on the solid state drive to give my Mini a competitive edge – especially for I/O heavy work loads such as compilation.

The surgery

My Mini in gutted state with no secondary or primary memory installed.

My Mini in gutted state with no secondary or primary memory installed.

A Mini is a bit challenging to upgrade compared to larger PC desktops – which shouldn’t come as a surprise considering its form factor. There is no way to “open” the case as with your run-off-the-mill PC desktop. In fact the case is one solid piece of aluminum. There are plenty of guides (such as this one) on how to pry off the Mini’s aluminum casing from its contents. I’d suggest the presumptive Mac Mini upgrader to carefully study some of these, and watch a Youtube video or two on the same subject, mainly to gather moral strength before brutally gutting the poor Mini with the help of a so-called putty knife.

Don’t feel too intimidated though – I’m no expert on hardware hacking, but found it surprisingly easy to get to the innards of the Mini! When inside the only problem I had was related to the seating of the two new RAM modules – remember to boot up the Mini and check that everything is okay before putting the case back on again!

The outcome

The upgrade has breathed new life into the Mini. Intellij IDEA 8.1.3 is fully usable, perhaps not enjoyable – but that I’m willing to attribute to my general Eclipse preference rather than lack of computer horse power. Web browsing feels snappy, applications that I used to have to wait for now starts up virtually instantly. Overall I’m very satisfied with the upgrade! 🙂

Don’t flash your flash!

I’m a Flash skeptic. There are plenty of useful and valuable applications of Flash technologies on the web – Parleys and Youtube, just to name two. Problem is that the majority of the most annoying, useless and resource consuming web content also is Flash-based. Ads tend to rely on Flash for crude animations, silly games and blinking lights that divert our attention from the content which we really want to consume. Some sites, notably news sites, have gotten increasingly desperate to increase ad revenue allowing more, larger and more annoying flash ads. Some front pages are so full of Flash – was about to write “full of shit” – that slower computers almost grinds to a halt. The fact that advertisers tend to rely on Flash for their most ominous content is however something which can be turned against them!

The first thing I always do with a fresh Firefox installation is to install the excellent Flashblock extension.  Flashblock takes a pessimistic approach to Flash content on the web, and per default blocks everything replacing flash contents with a placeholder one can click on to show the contents. It’s also possible, using a convenient right-click, to white list a site and allow its flash content to be visible by default.

flashblockMore than 7.000.000 Firefox users have installed Flashblock and hence started to associate its place holder logo with annoying content and sites desperate for ad revenue. They have realized that Flashblock can make previously painful browsing experiences speedy and enjoyable.

Front page of http://www.aftonbladet.se with Flashblock enabled.

http://www.aftonbladet.se with Flashblock installed.

The Addthis service provide an effortless option for web developers to add attractive and comprehensive social network sharing features to sites. However, Addthis also per default use Flash for statistics keeping purposes. The somewhat unprofessional-looking end result for a Flashblock user is something like this:

Flashblock render of the default Addthis flash.

Flashblock rendering of the default Addthis flash.

The remedy is trivial.  The extensive javascript configuration API of Addthis provides for:


<script type="text/javascript">
 var addthis_config =
 {
 data_use_flash : false
 }

</script>

Rackspace Cloud has some things going for it

I’ve long been looking around for some sort of internet connected home host for purposes such as experimenting with web application technologies and backing up source code.  After much consideration I finally decided on Mosso – which promised affordable fully featured virtual machines in their cloud.

I’ve never looked back since. The spin-up of the machine was a breeze, the installation of my preferred Linux distribution quick and for only about $10/month I had root access to my very own private Ubuntu server, with its own IP-address. Motto provides a handy management console for such tasks as taking backups of the instance, fiddling with the DNS configuration and migrating between their different plans. Initially I went for the dirt cheap basic plan which goes with 256 of RAM, 10GB of HDD and a guaranteed 1/64 part of the CPU power of  a 2-way quad core server. Memory demands stemming from Spotifun – a Grails based web application I’ve currently deployed to the instance – forced me to go one step higher on the plan ladder: 512 of RAM, 20GB of HDD and a 1/32 part CPU guarantee for ~$20/month. The migration was trivial, my virtual hard drive was simply grown and after a OS restart I was set.

Comparative evaluation

Competitor such as Amazon EC2 offer more assuring CPU-sharing guarantees but appear much more expensive than Mosso, an important consideration for my mostly hobbyist endeavors.  Mosso’s paltry CPU guarantees have in practice never troubled me, CPU-bounded work-loads typically flies. However I cannot judge out the possibility that I was simply lucky and got deployed to a lightly loaded physical server, the backdrop to this posting is only my personal experience with a single instance.

The only negative thing I want to add is that Mosso hosts its servers in the USA, causing an unavoidable round trip time of up to 200 ms for Europeans such as yours truly. The lag when hacking in a SSH console is acceptable for me, I’m not a fast typist anyway. 🙂 I’m not aware of any other cloud hosting service which currently allows the customer to decide the continent where the instance is to be hosted. However, the rumour mill has it that Microsoft Azure will have this feature when they are out of beta.

Mosso is since a couple of months back the Rackspace cloud.

64 bit JVM:s may come with severe penalties on memory constrained systems

I’ve been toying around with Spotifun – a Grails based webapp  – on a Rackspace cloud virtual machine.

The entry level rackspace offering is an instance with just 256 MB memory. I quickly discovered that 256 of RAM doesn’t cut it for running Tomcat with a single Grails app with a few grails plugins (~50MB WAR file size). The WAR barely deployed before the JVM ran OutOfMemory.

Decided to migrate my instance to a Rackspace plan with 512 MB of RAM.  This offered a sort-of reasonable experience. But just sort-of,  frequently episodes which excessive 10+ second response times would occur. Heavy garbage collection and/or swapping activity appeared to be a likely culprit, but as I was to cheap to just upgrade to a plan with even more memory I decided to whip out JConsole and have a look on the running JVM via JMX. I discovered that the JVM on the Rackspace instance was far more memory hungry than the JVM on my development work station.  Almost twice the amount of heap memory was used (measured post Tomcat startup)! And this on a system constrained to 512 MB (of which mysql and the base Ubuntu OS will obviously claim a significant share). Further investigation led me to the reason:

# uname -a
Linux 2.6.24-23-xen #1 SMP Mon Jan 26 03:09:12 UTC 2009 x86_64 GNU/Linux

Rackspace is obviously running on x64 hardware, causing Ubuntu apt-get to install a 64-bit JVM when java is installed. 64-bit software generally occupy more memory than their 32-bit counterparts, see the Wikipedia article on the subject for further details. The remedy is in this case however only a:

apt-get install ia32-sun-java6-bin

…away. A quick change to JAVA_HOME in Tomcat’s startup script to ensure that the correct JVM is used and memory consumption was back to sane levels. After reverting to a 32 bit JVM I’ve yet to see abnormal response times from Spotifun.

Maven repo full? Groovy comes to the rescue!

Working with Maven and continuous integration practices, the maven snapshot repository can grow in size at an alarming rate. Typically each commit to version control will trigger a build, test execution and (if the two previous step succeeds) a deployment of the project artifact to the the snapshot repo. Pretty soon, particularly if a large multi module project is the order of the day, the hard drive of the build server will be full with old snapshot releases of interest only to software archaeologists.

As I understand it maven repository managers such as Sonatype’s Nexus allow for specifying policies with regard to retention of outdated snapshot releases but I’ve unfortunately no experience with such beasts. For simpler scenarios where maven is configured to dump snapshots directly to the file system I’ve prepared a small shell script in groovy which  produces a list of the absolute path to repo files which can be safely nuked.  Set up your favorite build server (i.e. Hudson) to execute:

snapshotCleaner.groovy | xargs rm

weekly and you will never be troubled with running out disk space again!  (You may want to do a trial run without the rm part before trusting Hudson with erasing the files for you on autopilot, though. :-))

The script uses a couple of groovy niceties including (obviously) the ability to execute groovy as a unix shell script, the GDK java.io.File extension eachFileRecurse and the regex find operator =~ in a boolean context. Here’s the snapshot cleaner source:

#!/usr/bin/env /opt/groovy/bin/groovy

def snapshotRepoPath = '/depot/maven_repo/snapshot-repository'

long size = 0

new File(snapshotRepoPath).eachFileRecurse{File f ->
  try {
  if (isPartOfSnapshotRelease(f)) {
    if (!(f.getName() =~ getLatestDatePatternFromMavenMetaData(f))) {
      println "$f"
      size += f.size()
    }
  }
  } catch (Exception e) {
    System.err.print("For $f received exception $e")
  }
}

System.err.println "  Total disk space consumed by returned files ${size/(1024*1204)} MB."

def boolean isPartOfSnapshotRelease(File snapshotCandidateFile) {
  if (snapshotCandidateFile.isDirectory()) {
    return false;
  }

  if (snapshotCandidateFile =~ /maven-metadata/) {
    return false
  }

  boolean hasSubDirInParentDir = false
  snapshotCandidateFile.getParentFile().eachFile {File f ->
    if (f.isDirectory())
      hasSubDirInParentDir = true;
  }

  return !hasSubDirInParentDir
}

def String getLatestDatePatternFromMavenMetaData(File file) {
  def x = new XmlSlurper().parse(new File(file.getParent() + File.separator + "maven-metadata.xml"))

  "${x.versioning.snapshot.timestamp}.${x.versioning.snapshot.buildNumber}"
}