An Interface Developer's guide to build tools

What are build tools to us?

Interface Developers don't traditionally have much of a relationship with build tools. HTML, CSS and JavaScript require no compilation and the Application Developers on a project will often be in charge of the build.

This is changing. Our JavaScript needs validating (via JSLint) and, along with our CSS, compressing and concatenating. If you use SASS (I'm not quite convinced enough to yet) you will already be generating the final version of your CSS. Even HTML, when split into modules can need combining into flat templates.

It's becoming more a part of our jobs but it can be a sharper learning curve than for, say, Application Developers. Build tools are written in languages unfamiliar to the average ID and most of their documentation is written for developers who work in those languages, which usually isn't us.

Having hit the above walls I thought it was worth putting my understanding down, based on the tools I've worked with to date.

What is a build tool?

Programs, in a very basic way, are us writing down a process in a way a computer understands. With that in mind it will always seem obvious to programmers to have as much of their working processes also being programs. That way, you write it once and run it when you need to.

Builds are a way to formalise this; you write a build file describing your process and the build tool runs it.

Build scripts are usually centered around a few things:

• The chaining together of a series of processes
• The configuration of this being separate from those processes
• The sequence in which the processes are run is also part of the configuration

One idea of how to do it

There should be a natural workflow for using build tools:

• Write a plan of your process (not a build script)
• Turn this into a build script
• Configure the use of your build tool so you can run it in one step (like clicking a button on a web interface)

The posts

I'm going to write about Apache's Ant and Maven, both Java-based tools. This is because those are the ones used by JSLint4Java (see my post) and Rhino, mentioned in my post on using it to run JSLint.

So far, we have an article on Ant. The Maven article will follow soon when I can do it justice :)

I'm also planning to actually write some build scripts for each tool to complement the articles so keep an eye out in future.

An Interface developer's guide to Ant

Apache Ant

I love Ant, it's kind of like building Java programs out of Lego (and we all like Lego).

Overview

Ant was made by the creators of Tomcat to automate it's build in a way that worked across OS's. With heritage like that it's not surprising that it takes the approach of keeping its config in an XML file. As most tags allowed in this file are well-written XML (tag names, attributes and nesting all in the right order) it's all pretty easy to see what everything is doing once you get the main principles.

Those principles

Inside the build file processes, called tasks, are grouped into targets. The sequence in which these targets are run is defined by their dependancies to each other. This mechanism is also really simple, each task will run any other tasks that are listed as dependancies before it runs it's own processes.

For example, this is a build file:

<project name="MyProject" default="init" basedir=".">
       <description>
              simple example build file
       </description>
       <property name="build" value="build"/>

       <!-- make the build folder -->
       <target name="setup">
              <echo message="Creating build folder"/>
              <mkdir dir="${build}" />
       </target>

       <!-- copy the files across -->
       <target name="init" depends="setup">
              <echo message="Moving all .js project files to build folder"/>
              <copy todir="${build}">
                     <fileset dir="./">
                            <include name="**/*.js"/>
                     </fileset>
              </copy>
       </target>
</project>

How it works

You will run the above build by entering this into your command-line/terminal:

ant

That's it, no options to set, Ant will default to the assumption that you want to use a file called build.xml which is located in the folder you are in when you run the command.

So it looks in build.xml and sees the XML above. The default attribute in the project tag specifies the init task as the one to start with & the basedir attribute that the build will execute in this folder (. means the current directory, in case you were wondering):

<project name="MyProject" default="init" basedir=".">

So Ant looks at the init target and sees that it is dependant on the setup task (see its depends attribute. As mentioned before, this means it executes the setup task before the contents of the init task.

So the setup task contains a single mkdir process to make the build folder we need for the main task.

<target name="setup">
       <echo message="Creating build folder"/>
       <mkdir dir="${build}" />
</target>

Now the main init task can run. This has a copy task that operates on files specified by its child fileset tag. This tag includes a child tag of its own that specifies a pattern files need to match to be used.

<target name="init" depends="setup">
       <echo message="Moving all .js project files to build folder"/>
       <copy todir="${build}">
              <fileset dir="./">
                     <include name="**/*.js"/>
              </fileset>
       </copy>
</target>        

A word about properties

You might have noticed the <property> tag being used. Anything in your build file that is referenced many times and stays the same should be kept in a property. This is good practice but also means if you need to change a property, you do it only once, not every time it is used in the file.

most properties are simple name:value pairs (with each set in an attribute on the property tag but they can hold a lot more

Also note that every build file has some properties already available so to avoid duplication, have a look.

So pretty straight forward really. Ant tasks are available for most things you will need to do from Apache with loads of others available from the open source community.

Good ideas

When you're writing build files it's a good idea to follow a few rules to keep it all nice and clean. To be honest, these apply to the writing of any program but I digress…

• Keep any values you're using in multiple places as properties (like ${build} in the code above)
• Use the echo task liberally. It is how you will know what your build is doing at each step
• Comment your build file for clarity

What else can I do?

Everything really.

How you run it matters

You can run the ant command with a load of different options. To see all of them, type:
ant -help

You can use different build files, set properties that are used inside the build file and lots of other things that can change how the build runs.

Reducing duplication even more

Giving a tag an id attribute means you can use it as a reference in other targets. For example, tags like fileset can be given an id

       <fileset id="jsfiles" dir="./">
              <include name="**/*.js"/>
       </fileset>

…and referenced across your file instead of copying the same thing multiple times:

<fileset refid="jsfiles">

And

I'm not the biggest fan of Apache documentation but the Ant manual is worth a read for all the other things you can do with Ant. Try not to get too bogged down with all the Java bits (which is what spun my head out a bit at first).

Julien Lecomte of Yahoo has a nice post on using Ant for processes IDs would commonly use. You may not end up using all of his approach (it's quite tied into Yahoo's approach to development which may not match yours) but the good thing about Ant is you can grab the tasks you do want and just plug them in.

JSLint in your workflow

Primer

JSLint is a code validation tool for JavaScript

Using it will help ensure JavaScript on a project is produced in a consistent way that avoids syntax errors.

The option to 'JSLint' a JavaScript file is everywhere but the options that it is being run under are usually not shown. This means you're validating your code against invisible settings. It also means members of a team can be validating their JavaScript against settings different to yours, which is dangerous.

Goal

Be able to lint code against the same options on everyone's IDE across the project

A bit more specifically, we need to run a set version of JSLint, with set options and we need to be able to run this on any OS and with any IDE.

Programs used

• Java (Java Developer Kit (JDK) or Java Runtime Executable (JRE) - used to run Java programs)
• JSLint4Java - a Java program used to run jslint
• Rhino - Mozilla's Java implementation of JavaScript. Can run any JavaScript program.
• JSLint

Using Java on your computer

Check if you have a version of Java on your computer:

Windows

1. Go to Start menu
2. Go to 'Run'
3. Type 'cmd' to enter the command line
4. Type 'java -version'

Mac

1. Open the Terminal App (in Applications -> Utilities)
2. Type 'java -version'

If you get:'java is not recognised as an internal or external command, operable program or batch file' you don't have Java.

If you get output that starts 'java version' you have it.

If you haven't got a version of java on your computer, download the JRE or JDK from Oracle

On both Windows & Mac, the installer will take care of everything to do with the installation. On Mac, installing a new version of Java will not always update your core version. If this happens, use the Java Preferences App (Applications -> Utilities -> Java Preferences App) to switch versions.

Running JSLint through Java

We are interested in running JSLint using Java because:

1. Java is supported cross-platform and cross-OS
2. Running JSLint through a non-browser-based program allows a lot of flexibility.
3. Jars are supported by most Continuous Integration Processes.

So, in short, what we are trying to do is run a program that checks a file against JSLint validation and outputs the results of this. There are a few options of how to do this.

Options for doing this

The first time I tried this (under the tutelage of @agentdeal using Mozilla's Rhino, a Java program that runs JavaScript programs. It does this by running jslint.js and sending it the JavaScript file you are testing as a parameter.

java -jar js.jar jslint.js filetotest.js

The results are produced by Rhino as standard output meaning it's up to you what you do with them.

Even now this is still a good and solid method and benefits from being very simple and also by no part of its process hidden. It also doesn't pack JSLint into the jar meaning you can grab the latest version from Douglas Crockford's repo on Github

If you prefer a jar with all this as options, rather than file dependancies (which is a lost less hassle to plug into your Maven or Ant config file, you may like JSLint4Java.

I'm not going to tell you which to chose but I do list each approach on separate pages:

• Rhino
• JSLint4Java

I personally prefer to leave the options and globals settings in the JavaScript files themselves so people can see what their code is being validated against.

A word of caution

How ever you go about running it JSLint can, to quote Mr Crockford, Hurt your feelings with its bluntness. This is what you want (clarity) but it will only work if you don't confuse this directness your own set up errors.

So make sure the options you are setting confirm the idea of valid code your developers are working to and be aware that any options you don't set values for will inherit them from the version of jslint.js you are using.

Also ensure you use an external version of jslint.js, preferably the latest from github. Without specifying this with JSlint4Java you will be using the one baked into the jar.

Running JSLint through JSLint4Java

Note: this post assumes you already have a version of Java installed on your OS. If not, refer to this post for how to get it.

JSLint4Java is a Jar that allows us to run JSLint & output the results.

Installation

Get the latest JSLint4Java. Make sure you download a file that ends .dist.zip as those ending in .src.zip are the source files for development, not end users.

Unzip the downloaded file and store the resulting folder in a sensible location (like C:\Program Files\Java if Windows, or /usr/local/bin if Mac). It doesn't matter where it goes, just that you know the path to that location as you'll need it later on.

Inside the folder you'll find a jar file that's pretty clearly named, for example jslint4java-2.0.0.jar. This is the JSLint4Java jar file you'll be using.

Get the latest version of JSLint. (Usually saved as jslint.js.) If you have a version you are required to use, you can substitute it for jslint.js.

Running JSLint4Java

This is fairly simple but, in my opinion, made difficult by not being documented very well on JSLint4Java's Google code page. If you're like me and want to run it from the command line before you go plugging it into any CI, doing this is not well explained.

To use it in this way you will always be running it like this:

java -jar [path to jslint4java jar file] [options] [file to validate]

So the only bits of this that vary are the options and the file to validate. To see which options are available, run one of these 2 commands:

Windows

java -jar jslint4java-2.0.0.jar --help

Mac

man jslint4java-2.0.0.jar

Core options

Most of the options are duplicates of the JSLint ones with a few important exceptions.

Setting jslint version

The --jslint option allows you to specify the jslint.js file to use:

java -jar jslint4java-2.0.0.jar --jslint jslint.js

Controlling output

The --report option allows you to specify the format the results are in. The options are:

• plain (plain text output)
• xml
• junit (in junit xml)
• checkstyle (in checkstyle xml)
• report (in HTML)

Basic usage is typically like this:

java -jar jslint4java-2.0.0.jar --report xml

This is usually only useful if you can send the output to a file. This is done like so (assuming the output file is in the same folder as jslint4java-2.0.0.jar):

Windows

java -jar jslint4java-2.0.0.jar --report xml > results.xml

Mac

java -jar jslint4java-2.0.0.jar --report xml | results.xml

Running JSLint using Mozilla Rhino

Note: this post assumes you already have a version of Java installed on your OS. If not, refer to this post for how to get it.

Mozilla's Rhino is a Java program that runs JavaScript. What this means to you is you can use it to run JavaScript programs outside of the browser, like Python, Perl, PHP, etc...

Because we are trying to run a JavaScript program (JSLint) on the command-line that takes arguments and outputs results in text form, it's quite useful.

Installation

Rhino takes a little longer to install, mainly because the downloaded files then need building. What I mean by this is that some of the installation process is done by a build tool called ant.

This sounds excessive, but if you have a Mac you'll probably already have it. If you don't, you should, it's amazing. (It's also used by many Java programs as part of their installation so you'll need it in future.)

To check if you have it, try typing ant on either your commmand-line (Windows) or in your Terminal (Mac).

If you get this: Buildfile: build.xml does not exist!

...then you have it. (Ant uses build.xml as its default configuration file so this message is just saying there isn't one.)

If you get: ant is not recognised as an internal or external command, operable program or batch file ...or a similar message then you haven't got Ant.

Installing Ant

Getting the files

If you're on Windows, it's a good idea at this stage to create a folder called 'Apache Software Foundation' in your C:\Program FIles folder. (This is just for organisational purposes, to keep all software you may use from Apache in one place.)

Download the latest .zip file of Apache Ant from here (It will be called something like apache-ant-1.8.2-bin.zip)

On Windows, extract it into the 'Apache Software Foundation' folder you made before. This should create a folder called what the .zip is but without the -bin.zip bit. Something like 'apache-ant-1.8.2'.

On Mac it's better to keep it somewhere like /usr/local/bin so you should do the following in your Terminal app (Applications -> Utilities -> Terminal).

Assuming you downloaded it to your Downloads folder, navigate to it like so: cd Downloads (This assumes this is your 1st command & therefore being executed in your home folder).

Type mv apache-ant-1.8.2.zip /usr/bin/apache-ant-1.8.2.zip to move it to the /usr/bin folder.

Type: tar -xvf /usr/bin/apache-ant-1.8.2.zip ...to extract the zip archive into a folder of the same name, using the tar utility program. Options used are to 'extract', 'use a verbose style to report the results' & 'use the file specified by the given path'.

Type: rm /usr/bin/apache-ant-1.8.2.zip ...to remove the original zip file.

If you have issues with any of the commands above you may need to preface them with the sudo command, i.e. sudo tar -xvf /usr/bin/apache-ant-1.8.2.zip

Note: do not use the sudo prefix without understanding it, it removes safe-guards to any commands you run.

Installation

In Windows, in your explorer window, copy the path to this folder (i.e. C:\Program Files\Apache Software Foundation\apache-ant-1.8.2)

Ant uses enviroment variables (variables that are available to all programs on an OS, used point to common resources) to work.

On Windows, these are set up like so:

1. Go to your desktop and right-click on the 'My Computer' icon.
2. Select 'properties' on the menu.
3. Click on the 'Advanced' tab.
4. Inside this tab there is a button called 'Environment Variables'. Click on it to bring up another menu.
5. In the section at the bottom called 'System variables', click the 'New' button.
6. Enter the variable name as ANT_HOME and the variable value as the path to your ant folder you copied before. (i.e. C:\Program Files\Apache Software Foundation\apache-ant-1.8.2) This creates an environment variable pointing to where ant is on your system.
7. In the section at the bottom called 'System variables' scroll the list until you find an entry called 'Path' and click on it to select.
8. Click the 'Edit' button and add this to the end: ;%ANT_HOME%\bin. The semi-colon adds a new sub-section to the path, %ANT_HOME% uses the enviroment variable you created and adding \bin to the end. So you're effectively adding C:\Program Files\Apache Software Foundation\apache-ant-1.8.2\bin to it.*

* This means that when you type 'ant' at the command prompt, Windows will search the local directory for a program called 'ant' and when it doesn't find one will search all folders listed in the Path string. As the bin folder (inside your ant folder) is now one of these, it will look inside there and run the 'ant' program.

Note: Running the JDK/JRE installer should have added an environment variable called JAVA_HOME. If you don't have this, you'll need to add it, pointing to the JDK or JRE folder in C:\Program Files\Java, i.e. C:\Program Files\Java\jdk1.7.0'

On Mac, open the Terminal app & type these commands:

export ANT_HOME=/usr/bin/apache-ant-1.8.2

export PATH=${PATH};${ANT_HOME}/bin

In the above commands, we are assuming your Apache Ant folder is as we specified above & we are then adding that environment variable to your path.

If the ANT_HOME environment variable is set to /usr/bin/apache-ant-1.8.2 then PATH=${PATH};${ANT_HOME}/bin is adding /usr/bin/apache-ant-1.8.2/bin to the end of the PATH variable string.

After all this, close and reopen the command prompt and type ant. If you get Buildfile: build.xml does not exist! ...you now have ant. Otherwise, check the stages above to make sure all is set up ok.

Installing Rhino

You can get the files for Rhino from here.

Place the zip where ever you want the Rhino program to sit (i.e. on Windows it might sit in C:\Program Files\Java). Unzipping the file should leave you with a similarly-named folder, for example, something like C:\Program Files\Java\Rhino1_7R3 from Rhino1_7R3.zip.

Now use either command-line (on Windows) or Terminal (on Mac) to move into the new folder:

(Windows) cd C:\Program Files\Java\Rhino1_7R3

(Mac) cd /usr/local/bin/Rhino1_7R3

Type ant and Ant should build Rhino. If it works you should get no errors and be left with a file called js.jar in the folder.

Using Rhino

Rhino runs like any other jar file, taking the JavaScript file you want to run as the 1st argument and any following arguments are passed to the JavaScript file. So this:

java -jar C:\Program Files\Java\Rhino1_7R3\js.jar fulljslint.js filetotest.js

Uses Java to run the C:\Program Files\Java\Rhino1_7R3\js.jar jar program, sending it the JavaScript file fulljslint.js and sending that file filetotest.js as its first parameter.

Now hold on a minute

You may have spotted that there's something I've left out. I've not explained about the file you're sending to Rhino (fulljslint.js). It turns out you can't just send Rhino jslint.js and there's a good reason for this.

Rhino will run any JavaScript you send it but that's all it will do. jslint.js is pretty simple, it takes 2 arguments: the JavaScript file to validate as a string and an object of options to validate against. It can't output the results to the command-line/terminal, or anything else you tell it to because it doesn't know how (JavaScript programs generally have no access to the output stream). So fulljslint.js is actually jslint.js with some extra code at the end to print the results it produces.

I made a version of fulljslint.js with the latest version of jslint.js from github (dated 2011-07-19) here.

If you want to update the version of jslint.js in it, just get the version you need and add this file on the end.

In closing

Well done you made it to the end, that took ages didn't it? In getting this far you have:

• Got Apache ant on your machine which is brilliant.
• Got Rhino, which can be used to run any JavaScript file outside of the browser.
• Understand how it all works so you can change it.

A quick note. I got the Rhino code via http://hustoknow.blogspot.com/2011/02/jslint-and-rhino-support.html so thanks to Roger Hu for his work on this.