Failing with style – improved Clojure test summaries on CircleCI

Continuous integration servers like Jenkins and CircleCI can display summaries of test results, surfacing information like “which of our tests are slow?” and, crucially, “which of our tests are fai

Red Badger Team

Feb 27, 2017

Continuous integration servers like Jenkins and CircleCI can display summaries of test results, surfacing information like "which of our tests are slow?" and, crucially, "which of our tests are failing?", without forcing developers to spelunk through the logs.

These summaries are produced by parsing test output, the de-facto standard for which is the JUnit XML format. Although its origins lie in the Java ecosystem, this format can be produced by plugins for many test runners in a variety of languages. For Clojure projects, there is a Leiningen plugin called test2junit.

After setting up test2junit on CircleCI, we ran into an issue with all failures being reported to come from line 153 of junit.clj, rather than the actual location in our test code. We also discovered that the context added in 'testing' blocks is omitted from the test summary, so decided to roll our own plugin to improve the output and make it easier to identify and diagnose test failures.

The result is test-report-junit-xml. As our example project shows, it's easy to get it set up with CircleCI – just add an environment variable to circle.yml:

  
 
 
 
 
    
  
  
  
  machine:
  
 
 
 
 
  
 
 
 
 
    
  
  
  
    environment:
  
 
 
 
 
  
 
 
 
 
    
  
  
  
      TEST_REPORT_JUNIT_XML_OUTPUT_DIR: $CIRCLE_TEST_REPORTS/lein

and add the plugin to project.clj:

  
 
 
 
 
    
  
  
  
  :plugins [[lein-test-report-junit-xml "0.2.0"]]

Now you can sit back and enjoy your detailed test failures.

On second thought… maybe you should go and fix them…

With test-report-junit-xml, the error message is visible in tests that throw exceptions, context from 'testing' blocks is included, and failures are traced back to the correct file and line number.

Like so much of the tech we use and love, test-report-junit-xml is open source. Suggestions? Problems? Head over to GitHub and let us know.

Failing with style – improved Clojure test summaries on CircleCI

Similar posts

Are you looking to build a digital capability?