Effortless Continuous Integration and Deployment with Node.js, Travis CI & Capistrano

At Red Badger there has been a significant transition to open source technologies which are better suited to rapid prototyping and highly scalable solutions. The largest area of growth for us as a company is in Node.js development, there are a number of active Node projects, some of which are now in production.

Node.js has gained enormous traction since its inception in 2009, yet it is still an immature technology (although maturing rapidly) therefore 'best practices' in the context of Node do not really exist yet. Initially, we did not have the streamlined Continuous Integration and deployment process we were used to from the .NET development world, so we began to look for a solution.

The Tools

Historically, we made constant use of JetBrains TeamCity as a CI server on our .NET projects. TeamCity is an excellent solution for these types of projects, which we would wholeheartedly recommend. However, it was hosted and maintained by us, running on a cloud instance of Windows Server 2008. It was both a heavyweight solution for our now much simpler requirements (no lengthly compile step!) and also not ideal for building and testing Node.js and other open source technologies which run much better in Linux based environments.

In searching for a new solution, we considered:

• Jenkins - a well established, powerful and complex Java based CI server
• Travis CI - Extremely popular in open source, particularly among the Ruby community. Travis CI is a lightweight hosted build server which typically only works on public GitHub repositories, although this is changing with its paid service, Travis Pro
• Concrete - an extremely minimal open source CI server we found on GitHub, written in CoffeeScript by @ryankee

Driven by our desire for simplicity in our tools (and our new-found affection for CoffeeScript), we opted for Concrete. 

687474703a2f2f646c2e64726f70626f782e636f6d2f752f313135323937302f636f6e63726574655f73637265656e73686f745f68692e706e67

After making a few modifications to concrete, we deployed it to a (micro!) EC2 instance, set up some GitHub service hooks and began reaping the rewards of Continuous Integration once again! We set up build-success and build-failure bash scripts to manage deployment and failure logging, and all was working well. After running Concrete for a couple of weeks on a real project, we started to miss some fundamental features of more well established CI solutions, such as a clean, isolated build environment and even basics like email notifications. There were also a number of occasions where tests would time out, or builds would seemingly never start or get lost in the process. It became apparent that such a simple CI solution wouldn't cut it for a real project, and we should look to a more reliable hosted solution.

Travis CI

Travis CI is a hosted CI server predominantly aimed at open source projects. It can very easily be integrated into a public GitHub repository with the addition of a simple .travis.yml config file which looks something like this:

[gist]https://gist.github.com/4606090[/gist]

Travis have recently launched a paid service for private repositories called Travis Pro. We decided to give it a try after being impressed by their free version. It is currently in beta but our experience so far has been very positive. Configuration is a matter of adding the .travis.yml config file to source control, and flicking a switch in the Travis dashboard to set up post-commit hooks to start triggering builds.

Travis runs a build from within an isolated VM, eliminating the side effects of previous builds and creating a much stricter environment in which every dependency must be installed from scratch. This is perfect for catching bugs or deployment mistakes before they make their way to the staging server. Travis also provides a great user interface to view the current build status, with a live tail of console output, which we find very useful during testing.

Additionally, Travis provides some nice features such as pre-installed test databases and headless browser testing with

PhantomJS

. Both of these features could prove extremely useful when testing the entire stack of your web application.

Capistrano

On a number of our node projects, we were performing deployments with a simple makefile which executed a git checkout over SSH to our staging server. Whilst this worked fine initially, it seemed rather low level and error prone, with no support for rollbacks and cleanups required to remove artifacts produced at runtime on the server. We also needed the opportunity to pre-compile and minify our CoffeeScript and didn't think that the staging server was the right place to be performing these tasks.

After a small amount of research, We found

Capistrano

. It quickly became apparent that Capistrano is a very refined and popular tool for deployment - particularly in the Ruby on Rails community. Capistrano is another gem (literally - in the Ruby sense) from the 37signals gang. Despite it's popularity in the RoR community, the tool is very generic and flexible and merely provides sensible defaults which suit a RoR project out of the box. It can be very easily adapted to deploy all kinds of applications, ranging from Node.js to Python (in our internal usage).

Installing Capistrano is very easy, simply run the command

gem install capistrano

. This will install two commands, '

cap

' and '

capify

'. You can prepare your project for Capistrano deployment using the command '

capify . '

, this will place a Capfile in your project root which tells capistrano where to find the deployment configuration file.

The heart of Capistrano is the DSL based deploy.rb config file. It specifies servers and provides a way to override deployment specific tasks such as starting and stopping processes. Our deploy.rb customized for Node.js looks something like this:

[gist]https://gist.github.com/4633276[/gist]

We use the

Forever

utility provided by Nodejitsu to ensure that Node processes are relaunched if they crash. Forever also deals with log file redirection and provides a nice command line interface for checking on your processes, so is also definitely worth a look if you haven't already.

Once this is all configured, all it takes is a simple '

cap deploy

' to push new code onto a remote server. Rollbacks are just as simple, '

cap deploy:rollback

'.

Continuous Deployment

Hooking Travis CI and Capistrano together to automatically deploy upon a successful build is trivial. Travis provides a number of "hooks" which allow you to run arbitrary commands at various stages in the build process. The

after_success

hook is the right choice for deployment tasks.

Capistrano requires an SSH key to your staging server to be present, so commit this to your source control. Then simply add the following to your .travis.yml configuration file:

[gist]https://gist.github.com/4606129[/gist]

Where deployment/key.pem is the path to your SSH key.

End result

Fast and dependable Continuous Integration which allows an efficient flow of features through development, testing and staging. With speedy test suites, you can expect to see deployments complete in under a minute after a '

git push

'.