Scheduled Testing with Cruise Control

There are a number of posts in the blogosphere around using Cruise Control with Salesforce.  However, these generally have a pure Continuous Integration focus, in that the code is checked out from source control repository, deployed to a dedicated Salesforce instance, compiled and tested.  This is a useful mechanism when in the development phase of a project, but not always the best solution once a customer is live.

At BrightGen we have a Service Management offering for Salesforce (and others), which means that from time to time we look after a solution that we haven't developed.  Even if we did build the original system, we will often be second or third line of support after local power users and administrators.  In these situations, we are interested in detecting if changes have been made that may cause problems - for example, a validation rule being applied that may preclude programmatic object creation.  To that end, we have used Cruise Control to set up daily execution of production unit tests.

If you don't already have Java installed on your target machine, install the JDK from the Oracle Download page.

Installing Cruise Control is pretty straightforward - a link to the latest version is available at the top of the download page. If you are using windows you can simply download an executable file and run that - its a stripped down version of full Cruise Control, but I've found its ideal for my needs.  Follow the defaults and it will install into c:\Program Files\CruiseControl and set itself up as a service.

Cruise Control uses Apache Ant to execute builds, and this is included in the installation.  In my install its located at C:\Program Files\CruiseControl\apache-ant-1.7.0.

The next step is to add the Force.com migration tool to Ant.  You can access this by logging into your Salesforce instance, navigating to the Setup page and opening the App Setup -> Develop -> Tools menu.  Click the Force.com Migration Tools link to start the download.  Once the download is complete, extract to a temporary directory - I used C:\temp.    Navigate to the temporary directory and you will find a file named ant-salesforce.jar.  Copy this file to the lib directory under the Cruise Control Ant installation.  In my case the command was

> copy c:\temp\ant-salesforce.jar "c:\Program Files\CruiseControl\apache-ant-1.7.0\lib"

Note that if you already have Ant installed and you have set up your ANT_HOME environment variable, Cruise Control will use that, so you should copy ant-salesforce.jar to lib directory of your existing Ant installation.

The next step is to create a project - navigate to your CruiseControl\Projects folder, and create a new folder - I've called mine BobBuzzard.  This folder needs to contain a couple of files.  Firstly build.xml:

<project name="Bob Buzzard Salesforce Automated Testing" default="compTest" basedir="." xmlns:sf="antlib:com.salesforce">

    <property file="build.properties"/>
    <property environment="env"/>

    <target name="compTest">
      <echo message="Executing tests on Salesforce Server ....."/>
      <sf:compileAndTest username="${sf.username}" password="${sf.password}" serverurl="${sf.serverurl}">
       <runTests allTests="true"/>
      </sf:compileAndTest>
      <echo message="Tests completed" />
    </target>

</project>

This is the XML file that controls the Ant build for the project.  The compTest target is the key element - this connects to the Salesforce instance and executes all tests.

Note that the user id/password and server url are parameterized rather than harcoded.  These are populated from the second file that needs to be created, build.properties:

# build.properties
#

# Specify the login credentials for the desired Salesforce organization
sf.username = <your username>
sf.password = <your password here>

# Use 'https://www.salesforce.com' for production or developer edition (the default if not specified).
# Use 'https://test.salesforce.com for sandbox.
sf.serverurl = https://login.salesforce.com

# If your network requires an HTTP proxy, see http://ant.apache.org/manual/proxy.html for configuration.
#

Finally, Cruise Control must be configured to build the project, via the config.xml file present in the CruiseControl directory.  My sample build file is shown below.  Note the publishers section at the bottom of the file - this sends out an email success/failure notification to my google mail account.

<cruisecontrol>
    <property name="BuildTime" value="0005"/>
    <project name="BobBuzzard" requireModification="false">
        <listeners>
            <currentbuildstatuslistener file="logs/${project.name}/status.txt"/>
        </listeners>

        <schedule>
            <ant time="${BuildTime}" anthome="apache-ant-1.7.0" buildfile="projects/${project.name}/build.xml" target="compTest"/>
        </schedule>

        <log>
            <merge dir="projects/${project.name}/target/test-results"/>
        </log>

   <publishers>
  <email mailhost="smtp.gmail.com"
                username="..."
                password="..."
                mailport="465"
                usessl="true"
  returnaddress="keir.bowden@googlemail.com"
                subjectprefix="[CruiseControl]"
  buildresultsurl="http://localhost:8080/cruisecontrol/buildresults/BobBuzzard">
     <always address="keir.bowden@googlemail.com" />
  </email>
      </publishers>    
   </project>
</cruisecontrol>

Once all this is done, you can then fire up the Cruise Control service via the Windows Control Panel. New projects are built as soon as the service starts, after which they will be built at the time specified in the config.xml file.

As the installation includes its own Apache Tomcat server, you can navigate to http://localhost:8080/dashboard and see the results of the build. If all has gone well, your dashboard will show a green block for a successful build, which can be hovered over to see a summary:

The Cruise Control Dashboard

In the event of problems, the block will be red.  You can see the details of the build by clicking the block, selecting the Errors and Warnings tab on the resulting page and expanding the Errors and Warnings section as shown below:

If any exceptions were thrown running tests, or indeed connecting to Salesforce, they will appear here.

One word of caution - if a build fails, always check why.  It may be that the Salesforce password has been changed, in which case Cruise Control will happily retry it every day which may cause user lockout.