Effective QR-scanning with the Raspberry Pi and Camera Module

For a recent project I had a need for a reliable and fast QR-code scanner. I had an Raspberry Pi 2 single-board computer on hand and coupled with the Pi Camera Module and some elbow grease I managed to come up with a rather sweet setup.


The official Raspberry Pi version 2 Camera Module consists of a 8-megapixel Sony IMX219 sensor mounted behind a fixed focus lens on a small circuit board. The board connects to the computer CSI connector via a flexible ribbon cable.


The stock ribbon cable supplied with the Camera module is too short (150 mm) to allow for a comfortable positioning of the module for QR-scanning. The are available replacement flex cables of various length that can be easily swapped in — I went with a 500 mm cable.

Focus point

The camera system has a huge depth of field but the fixed lens is still focused too far away to get sharp shots at the distances one would want in a QR-scanner. The lens is fixed with some glue but when this is broken or scratched off is it actually possible to rotate it in its assembly thus adjusting the focus point of the system.


Tools of the trade.

There are many instruction videos on the net showcasing various techniques to achieve this, what worked for me was a small pliers to keep the non-rotating part of the assembly secure while applying force on the lens knobs with a tiny pointed screwdriver to force the lens to rotate counter-clockwise.

rotateReal force is actually required to break off the glue, but after this can the lens be adjusted rather easily. To determine the amount of rotation required I’d recommend to take frequent snapshots with the raspistill CLI tool shipped with Raspbian during the operation, stopping when sharp shots at 10 cm are achieved.

According to reports should it be possible to unscrew the lens from the assembly if it’s rotated too much. I’d like to think it would be quite finicky to re-attach so be careful! It’s only a matter of ~ 360° of rotation to achieve the desired focus point.


The first step after attaching the camera is to enable it in  /boot/config.txt  (can be done with simple raspi-config utility), rebooting and verifying basic camera functionality with raspistill.

For the next steps are there a plethora of  write-ups. Many solutions appear to be quite fragile. I wanted something more robust, something not depending on python libraries that’s not installable with pip or downloads of binaries from unmaintained SourceForge-projects.

I finally ended up with agilerule’s QRScanner.java which utilizes the proven zxing java QR-code library (appears to be very popular in the Android eco-system) to detect QR-codes in snapshots taken by the standard raspistill command line tool. I threw in some optimizations:

  • The time consumed by zxing to detect a QR-code is proportional to the resolution of the image. The full 8 megapixel sensor resolution is very much overkill, I finally settled for 400×300 pixel snapshots (0,12 megapixel) as a compromise between detection speed and accuracy. This low resolution means that the image-acquisition, QR-detection cycle is dominated by the acquisition phase and that it’s possible to  detect ~1 QR-code/second.
  • Rather than dumping the acquired images to the root file system one can utilize tmpfs, thus saving some wear-and-tear on the SD-card and also perhaps achieving a minuscule performance improvement.

To try it:

  1. Clone the repo: git clone https://github.com/nordstrand/qr-scanner-for-raspberry-pi
  2. Install maven
  3. Run: mvn clean install exec:java


This QR-scanning setup is quite usable and responsive but cannot be compared with what you’d find at the super market checkout line when it comes to speed and accuracy.

The item to be scanned should ideally be positioned as parallel to the camera film plane as possible. Sharp shadow lines falling on the scanned item can cause problems. So can light sources shining directly into the lens. What worked best for me was an inverted mount, with the camera pointed down towards a dark surface. I’d also recommend to add a buzzer that can make some beeps after succesful scans.


~10 cm from the camera film plane to the scan target will work fine for a wide variety of QR-code sizes.

My main focus of improvement would be to increase the scanning speed. That would entail using something other than raspistill still image acquisitions, perhaps time lapses or trying to access the camera through a lower-level C interface.




Docker tips for java developers

Latest comes with new guarantees

Unlike the SNAPSHOT-version concept of the ubiquitous Apache Maven dependency manager the so-called “latest” tag in Docker comes with no effort to ensure that a somewhat recently published artifact/image is used. Thus will:

$ docker run myrepo.com/myimage:latest

only pull the latest  image from the repository when run the first time. On subsequent executions will Docker just re-use the myimage:latest image that was pulled into the local repository. An explicit:

$ docker pull myrepo.com/myimage:latest

is required to get the latest image.

Docker Compose up is actually quite smart

Re-running the up command on an already “up” project will check for any deviation between the running services and the configuration in the docker-compose.yml file.  A delta wrt. e.g. an environment variable value will automatically be reconciled by Docker Compose by stopping and removing any outdated container(s) followed by relaunching with the updated configuration set. This can save serious time compared the more brute-force method of tearing down and starting up all containers in the project after every docker-compose.yml change.

Docker volumes requires explicit cleanup (for now)

Removing volumes associated with containers requires an extra parameter:

docker rm --volumes mycontainer

If –volumes is left out any associated volumes will remain for ever, hogging disk space.

The Docker volume subsystem appears to be significantly reworked in the upcoming Docker 1.9 version with new features such as the listing and removal of orphaned volumes.

Spark Core: an experience report

I was recently introduced to Spark Core – a diminutive Arduino clone with integrated WiFi and extensive cloud support.

These are some observations I made while porting some Arduino 433Mhz RF code to it.

If one for some reason lacks WiFi (the integrated on-chip antenna offers quite good coverage) user code will in general never execute. The core will try for-ever to connect to the Spark Cloud while blinking a rainbow of confusing colors. The remedy is either to setup WiFi immediately (and give up on the thought of using the core in an un-connected none Internet-of-things setting) or use a recent addition to the Spark API:


The semi part of the system mode will make the core refrain from attempting to connect the cloud.

Spark Core ships with “special” bootstrap firmware, named tinker, that is factory-installed on all cores. If one, like me, skips the “newbie”-only step-by-step guide and immediately flashes over this firmware with something of ones own creation it’s no longer possible to get the core into the “Listening mode” required to set it up with credentials to the local WiFi network. How-ever it’s easy to reflash with tinker.

The core’s singular color status LED is a train wreck. It’s both in many cases the (only?) key to unlock what the core really is up to and, at the same time, painfully hard to decipher. I mean – what’s the difference between pulsing and “breathing”? Can you differentiate between light blue and cyan in a well lit room?

When porting Arduino code I stumbled on some interesting incompatibilities:

  • The Ardunio binary literal syntax, e.g. B01010101, is unsupported. It’s recommended to use gcc syntax, e.g 0b01010101, instead.
  • The Spark Core has a 32 bit CPU while much Arduino code tends to be 16 bit, this can cause problems with code that does bitwise operations (a perennial C language favourite).
  • When the Core is connected to the cloud one can expect some latency outside of the main loop function (that’s where WiFi system code is run). This can cause problems with timing-sensitive Arduino code.
  • Debugging by means of logging through the serial interface can be tricky – serial communcation appears to be seriously asynchronous even when data is explicitly flushed.

Spark Cores are quite cool and small. 🙂


Configuration management on the Raspberry Pi

Raspberry Pi computers running the Raspbian flavour of the venerable Debian Linux distribution make excellent headless home servers. Inexpensive, small, quiet, reliable, power-efficient and powerful enought to be employed as NAS-appliances, BitTorrent boxes, AirPlay receivers, blog hosts, beer fermentation controllers etc etc.. Setting up a Raspberry Pi for these application are seldom as trivial as a:

$ apt-get something

More typical are laborious sessions at the console cloning git repos, running DDL scripts, tweaking cron tables as directed by long step-by-step tutorials in blog posts.

If  one had to endure this once and then be set for the foreseeable future things might not be too bad.

However, as stable the Pi hardware is it does have one Achilles’ heel – persistent storage. The SDHC-cards used for holding the root file system are  infamous for not holding up very well in the long-term (perhaps not too unexpected considering the much less demanding camera applications SD cards typically are designed for). Eventual file system corruption appears  to be almost unavoidable, as the plethora of online problem reports bears wittnes to [1], [2].

This problem leads to frequent (re)imaging of cards. If one wasn’t for foresighted enough to keep a full backup image hanging around you’ll have to start from scratch with an install image, toiling away at the step-by-step guides again.

Enter modern configuration automation tools. The prefered silver bullet of the fledging devops movement promises big saving in the data center automating all kinds of system administration task on hordes of virtual machines in private or public clouds.

Recently I’ve experimented with applying these tools on a different kind of platform: the humble Raspberry Pi.

The tools provides ways to specify the desired configuration of the nodes under control in various domain specific languages. These languages typically allows for powerful encapsulation and modularization abstractions thus managing complexity and promoting reusability. The tools offers several importants benefits compared to manual or semi-manual configuration :

  • Large libraries of ready-to-use modules for tasks such as controlling databases, installing packages, etc
  • Convenient mass application of specification on multiple nodes
  • Idempotency. Specifications can be re-run repeatablity ensuring that controlled nodes are up-to-date with the latest addition.
  • More friendly syntax than bash.

I’ve taken a closer look att Puppet, Pallet and Ansible.

Puppet is a well-known Ruby based tool that has been around for a while. The Puppet ecosystem is huge and there is a great assortment of third-party modules ready to use for almost any task. I’ve used Puppet for implementing a manifest that installs the Tiny Tiny RSS reader on PostgreSQL with data backups, feed updates, etc.

Pallet is a newer Clojure based option. In the trio it is the option which currently clearly enjoys the least amount of community traction, the available documentation is also sparse and disorganized. Even though I’m attracted to Clojure as a language these factors caused me to abandon Pallet for now.

Both Puppet and Pallet are heavily geared toward operation in large data centers, offerings features to control and organize hundreds of nodes. Puppet also in general relies on central master nodes that agents on controlled nodes periodically pull new configuration specs from. This is a huge impedance mismatch when one only wants to control  a couple of a Raspberry Pi servers in a living room.

Ansible is a relatively new Python based tool hitting 1.0 as recently as February 2013. Compared to Puppet there is less documentation available and the selection of third party modules is still a little behind. However is Ansible under heavy development with features and new modules added at a furious pace, the latest release being 1.3 as of August 2013.

Ansible is by design push based and agent less, requiring no running daemon on controlled hosts. Since Ansible uses standard ssh as transport to control hosts it is perfectly possible to control hosts without installing any additional software at all on them. The apt modules however requires python-apt and aptitute to be installed (easily achieved remotely with the Ansible shell module).

This means that it’s possible to assume control of a freshly imaged Pi without ever manually logging in!

Ansible is remarkably quick and executes a typical playbook much faster than a corresponding Puppet manifest on the modest Pi hardware. Ansible comes with a great deal of instant gratification, the learning curve beeing  smoother than that of Puppet, much of which probably can be attributed to a simpler specification language. Ansible playbooks are YAML, Puppets manifest is a Ruby DSL. Yaml is definitely less ceremony, and more cool. Puppets manifest are declarative in nature, one defines the desired actions to be taken and relationships between them (like before/after) leaving Puppet to figure out the correct execution sequence, while Ansible’s style is more imperative. Puppets style of specification may offer better optimization possibilities on the tool level and perhaps specs that are easier to maintain in the long run, but so far I’m very happy with Ansible’s simpler approach.

The conclusion is simple: to control a couple of Pi:s go for Ansible!


DD-WRT wireless TKIP woes

The DD-WRT firmware has always been rock stable for me and I’ve been on various builds of it on a total of three work horse Linksys 54G routers for at least five years. In addition, lately DD-WRT have done VPN-duty for me on a more recent Netgear WNR3500L.

With no warning one of my trusty old 54’s suddenly dropped wireless – the SSID was no longer to be found.

Some long debugging later revealed the that the router had changed wireless security settings to WPA2+TKIP while WPA2+AES is the only recommended wireless security setting on DD-WRT. Resetting to WPA2+AES fixed the issue and attributing the strange change of setting to some freak issue during a recent reboot I was once again a happy camper.

Some weeks later, after another reboot, wi-fi was gone again. And this time it was impossible to change back to WPA2+AES under the Wireless security tab. I tried different options for saving the correct setting but every time the router immediately reverted back to TKIP again.

During a second round of debugging the root caused was spelled out for me in a forum thread – my trusty 54 had simply run out of NVRAM. This type of memory is used for storing router settings and statistics, stuff that must be non-volatile during reboots. Running out of NVRAM will cause DD-WRT to fail to persist settings (with no indication of the cause). The amount of free NVRAM on a device can easily be checked by executing

nvram show > /dev/null

either through the web interface (Administration → Commands) or by logging in via ssh or telnet (user=root, pswd=WiFi password).

There are multiple options for freeing up NVRAM, the one I used was clearing WAN bandwidth historical data.


Seven popular causes of unit test indeterminism

This post was almost in its entirety written in 2010. I found it recently in the Drafts folder and decided to publish it now. Most of the content is still relevant.

Most people want their test to be determinate and reproducible. Most people don’t want the added excitement that comes with a unit test beeing sooo green when run in the IDE and failing so hard on the continuous integration server.  In xUnit Test Patterns: Refactoring Test Code Gerard Meszaros calls a test exhibiting this kind of behaviour for an Erratic test.

Root causes for erratic tests include platform differences (causing tests to exhibit different behavour when executed on different computers), lack of state isolation (causing build failures only when tests are executed in an unfortunate order) and lack of proper synchronization (causing test to fail completely randomly).

I think we all can agree that Erratic tests are anti patterns of the first order. Hence here goes (in no particular order):

Seven popular causes of unit test indeterminism

1. Usage of Set instead of List. The java.util.Set contract doesn’t define any order for its elements, thus is new HashSet<String>("a","b").equals(new HashSet<String>("a","b")) not always true. Most times it is though. Though far less often true on the CI server than on the work station, in my experience. Advice: only use set colections in production code if it really makes sense (the set specific properties  has a meaning in the domain).

2. Line break representation. A rather common scenario in corporate settings are the devs (reluctantly) running Windows work stations while the integration server is hosted on a Linux box. Line breaks is represented quite differently on these systems and can cause problems both with production and test code. System.getProperty("line.separator") to the rescue!

3. Default character set. An InputStreamReader per default treats the InputStream it wraps as a byte stream pumping characters encoded in the platform default character set. The default character set varies wildly from operating system to operating system (and locale to locale). If your test data includes high bit characters (i.e. characters not contained within the the 2^7 symbols of the original ASCII spec) – as it should – you better be specific about the encoding of your InputStream. Advise: always specify character set encoding explicitly when working with character streams e.g.  new InputStreamReader(is, CharacterSet.valueOf("UTF-8")).

4. Case insensitive file names. Another platform dependent favourite: Windows is perfectly happy to new FileInputStream("ListOfAccountsToProcess.data") even though the file happens to have been christened  ListOfaccountsToProcess.data. When the code is moved to Unix enviroment the lower case a in the file name will suddenly start to matter.

5. Static state is something best avoided like the plague. It really kills test isolation, requiring pain staking (and easily broken) manual clean-up code to be inserted everywhere. Even if you have done your best to avoid static data, one sometimes have to work with Java system properties. These are inherently static – setting a property can potentielly influence any test run thereafter.

6) Time flies. Failure to take account of the most intrinsic property of time: time advances. Now is not the same time as a split second ago. The expression new LocalDate().isBefore(new LocalDate(2013,12,24)), while perfectly true as of this writing, will not be particularly true after this christmas. I prefer to trigger CI build not only after commits but also periodically for, among other reasons, to expose these kinds of deficiencies as early as possible.When testing time dependent logic, be careful about hard coding dates in the test data, prefer to work with dates relative to now. Or, even better, ensure that all access to the current time in application code is through an easily mocked interface.

7) Database integrations tests. Programatic integration tests are controversial, in a perfect world we may not have a need to these slow and fragile misfits but for now few application can go with out them – particularly wrt. persistence. On the many problems with integration tests is the question of how the test fixture is managed. In unit test preconditions are typically set up in memory within the scope of each tests, thus each tests a guaranteed a fresh pristine test harness. This approach is typically not feasible when implementing data base integration tests, even when working with in memory database the cost of dropping, creating and populating the database with test data is comes with a prohibitive cost.  Other options such as manually trying to reset the database to its original state after each tests or writing test methods that shares a common fixures are laborious and unattractive and can lead to poor defect isolation.

The venerable Spring Framework provides a TestRunner that promise a luring solution. By virtue of a @Transactional annotation the frame work starts a fresh transaction before execution of the test method, a transaction that is rolled backed post test method execution – thus guaranteeing perfect test fixture isolation even though the database is reused.

The problem is only that many features of the the typical ORM stack is not properly exercised if the transactions is not commited! In fact it is only until the transaction commits the JPA PersistenceContext/Hibernate session flushes to database. Additionally are database constraints typically deferred util commit time.


OAuth – a brief summary

OAuth is a simple standard for allowing an end user to authorize an application to access a third party service on behalf of said user.

Access is authorized on two levels by the third party as the application needs to be identified as does the the user on behalf of which it is acting.

Application authorization

The application obtains through an out-of-band channel, typically a web form at the third party service where the application developer submits an application for access, the following pair of credentials:

  1. Consumer key (a.k.a. API key, public key, application key). Transmitted to third party as oauth_consumer_key.
  2. Consumer secret (a.k.a API secret, private key, consumer secret key, application secret)

The consumer secret is never directly transmitted to the the third party, as it is used to calculate a signature for requests.

User authorization

The user typically authorizes the application to access the service using a 3-Legged OAuth process whereupon its completion the application obtains an access token consisting of:

  1. Token (a.k.a. access token). Transmitted to third party as oauth_token.
  2. Secret (a.k.a. access token secret, oauth token secret).

The secret is never directly transmitted to the the third party, as it is used to calculate a signature for requests.


OAuth authorized http requests to the third party adds several OAuth specific parameters the most important of which are oauth_consumer_key, oauth_token and oauth_signature. The value of  oauth_signature is a SHA1 calculated hash of the consumer secret, access token secret and all the parameters sent in the request. OAuth parameters can be sent as standard URL parameters or as the value of the Authorization http header.