GAE automated python testing with Nose and Growl

Let's say we have an app written in Python to be run on Google App Engine, and we want to write unit tests for it. ...or, wait. Let's even say we don't have an app (yet) and we want to start from writing the tests (AKA TDD).

Anyway, here's a simple environment:

• Growl
• Nose
• snort and NoseGAE plugins for Nose
• nosy or tdaemon - a tiny little program that would run in the background, run your tests as soon as one of the source files changes and notify you about tests results using Growl.

First, make sure you have Growl installed and running. 

Next, install python packages:

pip install nose NoseGAE snort nosy

or if you prefer tdaemon, replace "nosy" above with "tdaemon". I'm going to show you both in a bit. 

If "pip" gives you "command not found", replace it with "easy_install", or install pip python package with "easy_install pip" and then execute the above again.

A side note: why snort and not nose-growl? Well, it's been a nice plugin for Nose but it looks like the latest Mac OS, Lion, and the plugin doesn't play nice together. In fact, there's a blog post describing the setup in details. I couldn't make it work with Mac OS Lion though. So, yeah,  I'm using Mac OS here but this should work for a linux distribution too. Not sure about MS Windows.

Back to the point. First off, let's just make sure the tests are running:

nosetests --with-gae --without-sandbox --logging-filter=-root

You should be able to see an output in the terminal similar to the following:

..........
------------------------------------------------------
Ran 10 tests in 0.938s


OK

Now, let's make sure we have growlnotify working properly (that's what snort plugin for Nose is using to display test results notifications). Just type this in the terminal:

growlnotify -m test

You should be able to see a notification saying "test".

- that means Growl and growlnotify are installed and working properly. 

Almost done. All we need to do now is connect nosetests results that were displayed in the terminal earlier with growlnotify (1) and make it go over and over again whenever a source file of the app is changed, i.e. we want to make it listen to the filesystem changes (2) related to our application directory.

The (1) is pretty easy: add "--with-snort" option to "nosetests" command, e.g. right before "--with-gae". That's what we installed snort package for at the beginning.

Now, to make it watch filesystem changes (2), there are two options (at least, the options I want to show in this post) - nosy or tdaemon. They are pretty similar in the functionality.

Here's how to make nosy work.

Create a file, let's name it "nosy.cfg" somewhere in your app directory with the following content:

# config file section for nosy
# run nosy from the APP_ROOT with: nosy -c tests/nosy.cfg
#
# Including this file in the paths to check allows you to change
# nose's behaviour on the fly.

[nosy]
# Paths to check for changed files; changes cause nose to be run
base_path = ./
glob_patterns = *.py
# exclude_patterns = *_flymake.*
# extra_paths = setup.cfg

# Command line options to pass to nose
# add --processes=num_of_cores to options (num_of_cores is a number, like 4) 
# if you want your test suite to run faster
options = --with-snort --with-gae --without-sandbox --logging-filter=-root

# Command line arguments to pass to nose; e.g. part of test suite to run
# tests = tests/unit_tests.py

I like putting all the testing stuff in a "./tests" subdirectory, so I placed it in ./tests/nosy.cfg.

To start the whole thing, execute the following in a terminal:

nosy -c tests/nosy.cfg

You should probably replace tests/nosy.cfg with the correct path to your nosy.cfg file we've just created.

Once you've started nosy, you'll see your test results appearing in Growl notifications:

If you look at the terminal, note that nosy didn't exit. It's sitting there and waiting for any file changes. Try modifying any of your python source code and save that file. You'll see that the tests begin to rerun automatically. I'm going to make a test fail on purpose now, just to make another screenshot:

That's it.  Make sure you have all tests pass (or fail, on purpose :) as you keep on writing the code.

How to make it work with tdaemon

There's no really need for a config file (e.g. a nosy.cfg) to make tdaemon work. 

Simply execute this:

tdaemon --custom-args="--with-snort --with-gae --without-sandbox --logging-filter=-root"

and it'll do the same effect like "nosy -c tests/nosy.cfg".

Happy testing!