If the customer could live with the strange workflow, a quirky UI, or some other abnormality, so could I. It was more important to get the usable code out there. We’ll smooth the edges out later. Which is fine if it actually happens. But, it never really seems to work out that way.

So, nowadays I am not so sure. Even if the customer wants the functionality, I wonder if we do them a disservice by giving it to them. Perhaps it is a matter of who says it is “done-done”, but I am beginning to wonder if it is better to deliver the tiniest bit of functionality (iteratively of course) until it is freaking awesome. No rough edges, no quirks. Just awesomeness. Then, we move on.

This entails finding out what the heart of the functionality is and only doing that. Also, we need a customer that acknowledges the rough edges whether they face the user or are in the code. Hmmmm.

Well, like I said, I am not so sure. But I am beginning to wonder.

Does all of the code compile?

Number of automated tests?

What is your code coverage?

What is your branch coverage?

What is your average cyclomatic complexity?

If working with legacy code, how long has it been since it was modified?

If working with legacy code, who wrote it, an employee or contractor?

Are you familiar with the Joel Test (or something like it)? What’s your score?

If you have the basic infrastructure:
What type of source control do you use?
What tool(s) do you use for continuous/daily builds?
What tool(s) you use for bug tracking?
What tool(s) do you use for deployment?

How long does it take to get a user environment set up?

What type of machines (RAM, CPU, monitor(s) size)?

What type of phones? Obvious how to hang up, redial, etc?

How easy is it to get conference area?

Do conference areas have projectors?

Do conference areas have whiteboards?

Do cubes have whiteboards?

How many managers have programming background?

How many managers to a person?

How are people organized? Matrix, hierarchical, team?

Who makes the estimates?

What is the basic unit of measurement for estimates? Hours, days, points, etc?

Type of browser? Are daily tools dependent upon browser?

And, of course,

Is coffee provided?

Problems and Solutions Expanded

Let’s expand a little on the table above. First, the problems are in decreasing severity to a the long term success of project. So, not having a set of tests is the biggest (technical) detriment to the project. Without automated and continuously running tests, we essentially have a black box. We can’t easily make changes without being certain that we are not introducing bugs.

Given the choice between some tests and no tests, I’ll take some tests. If we have some tests, we have a lot of the hard stuff out of the way. But, how do we make it better? We add tests until we have all code paths covered. There are number of tools out there for measuring code coverage. Hook the coverage tool up with your tests. Remember, it is not the percentage of code covered in a class that is important. It is the branch coverage. We want to know that we have tested all the paths through the code. If you have good branch coverage, you’ll have good line coverage. Good line coverage does not necessarily mean good branch coverage.

If we have complete tests, the next issue is most likely that the tests are hard to understand. To fix this, we keep in mind that not only prove that the code is working as expected, but also act as documentation. Effective documentation means exposing only what is necessary important for a certain outcome. We want enough context to understand what is going, but not so much information that the essentials are drowned out. Achieving this is a difficult balancing act. Refactoring test code (extracting methods and classes, using proper setup and tear down) and patterns like the test data builder help us achieve this goal.

Now I must come clean: the assumption all along has been that by starting at the tests we have made changes to the production to make it testable. This means that if we have all the other problems licked, we should be at a stage where the public API for the production code is settled. We have all the code tested, and we know what the code is supposed to do. So, if the production code is difficult to understand, it is a matter of refactoring it.

All to often, when dealing with software, we stop after something is working. But, as those in the know know, we ain’t done. We better have tests at this point, and we need to refactor. This is where we go beyond making something work. This is where we reflect on its usage and how to make it better, how we make it easy to do the Right Thing and hard to do the Wrong Thing.

I don’t want this to become yet another “You Need To Have Tests” post. But, our tests make sure the code still does what we want while we are making the code easy to work with. We need to have a safety net if we are going make changes to the code that will allow us to move beyond “something that works”.

Think about code you’ve worked with. Think about the code bases you liked and the ones you didn’t. Strive to get your code like the ones you enjoyed, the ones you could intuit what you needed to do, the ones that guided you in the right direction. Most likely, that code made made it difficult to do the Wrong Thing, but was so easy to work with that doing the Right Thing was effortless.

Don’t fall int the trap of just recreating code in the style that you were subjected to. Rise above that crap and create something makes it easy to do the Right Thing!

The same can be said for code. Messy (poorly formatted code) can increase the difficulty of understanding the intent and/or modifying the code.

Look at the code I recently came across. The code has been changed to protect the innocent and not-so-innocent. I had to make the font size really tiny to get everything on one line like it was originally. But, the spirit remains intact.

String string = (CyberProvince.findByCountryId((CountryParameters.findCountry()).getCountryId())).getProvinceNumber().toString()

So, image we have some objects that represent part of a hierarchy. In this case, we have provinces which are part of a country. And, to make things more interesting, we have the notion of a virtual province (CyberProvince). Anyway, from the code above it is a really difficult to see what is going on and who the actors are.

So, let’s make this readable. First, the parenthesis:

String string = (CyberProvince.findByCountryId( ( CountryParameters.findCountry() ).getCountryId() )).getProvinceNumber().toString()

In the very least, if you are using a buttload of parenthesis, then use some space to expose the “pieces”. Next, let’s extract a variable:

CountryParameters countryParameters = CountryParameters.findCountry();
String string = (CyberProvince.findByCountryId( ( countryParameters ).getCountryId() )).getProvinceNumber().toString()

A little better. Now, lets clean up some of the parenthesis, and extract a variable.

CountryParameters countryParameters = CountryParameters.findCountry();
Integer countryId = countryParameters.getCountryId()
String string = (CyberProvince.findByCountryId( countryId )).getProvinceNumber().toString()

Let’s extract another variable.

CountryParameters countryParameters = CountryParameters.findCountry();
Integer countryId = countryParameters.getCountryId();
CyberProvince cyberProvince = CyberProvince.findByCountryId( countryId );
String string = (cyberProvince).getProvinceNumber().toString()

Ok. Let’s do another round of parenthesis fixing.

CountryParameters countryParameters = CountryParameters.findCountry();
Integer countryId = countryParameters.getCountryId();
CyberProvince cyberProvince = CyberProvince.findByCountryId( countryId );
String string = cyberProvince.getProvinceNumber().toString()

Before the changes, understanding the code and considering modifications was much more difficult. Now, we can actually understand the intent. We can begin to analyze the code, to weigh the pros and cons of further refactoring of said code. Plus, we have the added benefit of being able to more easily debug the code if needed.

Once again: neatness counts!

So, in addition to the Jetty-related jars mentioned in the earlier post, we will need the following Selenium jars:

selenium-java-[version].jar
selenium-server-standalone-[version].jar

They are available here.

We have a JSP:

<%@ page contentType="text/html;charset=UTF-8" language="java" %>
<html>
<body>
<h1>From The JSP</h1>

<h2>h2 text</h2>

</body>
</html>

This page is served when one tries to go to http://localhost:8080.

And, we also have a simple test class:

package com.bitmotif.example;

import org.eclipse.jetty.server.Server;
import org.eclipse.jetty.webapp.WebAppContext;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.server.SeleniumServer;

import static org.junit.Assert.assertEquals;

/**
 * User: pjberry
 * Date: 3/22/11
 * Time: 6:29 PM
 */
public class ExampleTest {

    private SeleniumServer seleniumServer;
    private Server applicationServer;
    private WebDriver driver;

    @Before
    public void setUp() throws Exception {
        startApplicationServer();
        startSeleniumServer();
        driver = new FirefoxDriver();
    }

    @After
    public void tearDown() throws Exception {
        driver.quit();
        seleniumServer.stop();
        applicationServer.stop();
    }

    @Test
    public void testPageContents() {
        driver.get("http://localhost:8080");

        WebElement element = driver.findElement(By.tagName("h2"));

        assertEquals("h2 text", element.getText());
    }

    private void startSeleniumServer() throws Exception {
        seleniumServer = new SeleniumServer();
        seleniumServer.start();
    }

    private void startApplicationServer() throws Exception {
        applicationServer = buildServer( buildWebAppContext() );
        applicationServer.start();
    }

    private Server buildServer(WebAppContext webApp) {
        Server server = new Server(8080);
        server.setHandler(webApp);
        return server;
    }

    private WebAppContext buildWebAppContext() {
        WebAppContext webApp = new WebAppContext();
        webApp.setContextPath("/");
        webApp.setResourceBase("./src/main/webapp");
        return webApp;
    }

}

So, what do we have here? Simply, we fire up Jetty, then fire up the Selenium server, and lastly, we fire up the WebDriver (client). With our server running and proxy running, we use our client to load a page. Next, we load a page and get the h2 tag. Then we check the text in the h2 tag.

This is freaking awesome. Basically, we have a way to test against a container without having to do a deployment! Extending this idea, we can then do things like create tests for Javascript without leaving the IDE or automate acceptance tests for continuous integration without a bunch of tedious configuration. That’s pretty powerful stuff.

Let’s face it, this just isn’t working. I wish I could say it’s me, but it’s really you. I’ve tried and tried to make you happy, but in the end, it has just made me unhappy.

Look, you really are amazing. The things you know and the things you can do–damn! But, when we disagree, it is awful. When it happens, it makes me wonder why we are together.

So, we should break up. I can use the space and time to catch up with old friends, to do things that I used to do before I met you.

Maybe we’ll cross paths again in the future, but for now, we should go our separate ways.

As I became more senior, the knowledge I acquired by investigation and asking meant that I became a person providing answers to questions. In fact, there are times when that is all that I do–so much so that I sometimes I forget that I can ask questions. Nowadays, (in the worst case) I’ll flail for a while making progress, but at too slow of a pace. Then, I’ll either eventually figure it out or eventually realize I need to ask some questions. Either way, it isn’t the most efficient use of my time.

I need to remember as I progress, it is still OK to ask questions. I can’t always know all of the answers and, in some cases, figuring out the answer does not have a good return on the time invested.

Using what you have to get what you need

That definition has always stuck with me. I find it to be the simplest, most straight-forward definition of what an engineer does. Admittedly, it is vague. But, I think it is vague in a good way. It doesn’t overspecify, it just gives us a conceptual framework.

How does this tie into what we sometimes call software engineering? Most of the time, we are trying to solve some problem (to get what we want) with software (which we already have or will have).

Ideally, we have an idea of what we want. But, in reality, that’s rarely the case. Much of software development is discovering what is actually needed. An easy way to do this is to have some software, let the users work with it, get their feedback, make changes, and repeat.

This is where many organizations go astray. Rather than doing software work (engineering) themselves, many organizations decide to buy a tool. On the surface, this isn’t a bad a idea–why reinvent the wheel? But, in order for a tool to solve a problem, we have to fully understand the problem.

And, this is where the title of the post comes from. If there is a piece of software that solves the problem, it would be unprofessional to propose creating our own. However, it is also unprofessional to propose using a tool when we do not know enough about the problem. Without the engineering exercise, we are going to have a tougher time selecting a tool.

In the end, even if we think it is possible to use a tool to get what we want, it is worth while to do some engineering.

Tests should be written first

Let the test help direct the design
Make the code easy to test

Each test must be independent of other tests

Running tests in isolation provide faster feedback and makes testing easier
Junit assumes that tests can be run arbitrary order
Depending on other tests indicates tightly coupled code

Tests should not be dependent upon upon production code to be tested

For example, don’t use your dao to insert data for testing
Failing to do so requires proving that the dao works, which then creates a dependency on another test

Tests should demonstrate behavior for expected and unexpected behavior

Tests serve as documentation for the happy path and the sad path

Red, Green, Refactor

This is where we remove duplication, simplify, and in general make things better
Applies to the test code as well

Each class merits its own test class

Applies to “simple” classes, too
Not only is it a test, it is documentation for the future

Organize test methods in three chunks arrange, act, assert

This makes your intent much easier to understand

Isolate layers

You shouldn’t need a database connection to test a class that doesn’t directly use the database
Mocks, Stubs, and Dependency Injection are your friends

Each test method should test one thing

This makes your intent much easier to understand
This makes it easier to understand what is and isn’t working
Ideally it would have one assert

Prefer naming conventions that describe the scenario and outcome

One example is methodName_Scenario_Outcome
Use the language of the domain

Know what you are testing and how to test it

Prefer state-based testing for checking return values and outcomes
Prefer behavior-based testing for testing interactions

Keep tests as quiet as possible

Avoid printing out statements, logging, etc.
Your tests should only be vocal when something interesting (usually bad) happens

Tests should be as small and as fast as possible

Do one thing, do it fast
Fast feedback is the name of the game

Tests should be run regularly

In the very least, every time there is a code change
Automate it
Get notifications if tests are failing

Fix tests ASAP

Even if you aren’t the on who created the issue

Write tests to reproduce bugs

Use this to guide fixing code.

Obviously, this isn’t the alpha and the omega of unit test practices, but I think it captures most of what’s critical.

The following links were influential (but not the only influence) for this list.
http://exubero.com/junit/antipatterns.html
http://geosoft.no/development/unittesting.html
http://stackoverflow.com/questions/106800/unit-testing-guidelines
http://blog.jayfields.com/2005/11/unit-test-guidelines.html
http://developer.apple.com/library/mac/#documentation/DeveloperTools/Conceptual/UnitTesting/1-Articles/UTGuidelines.html No Longer Valid
http://pragprog.com/the-pragmatic-programmer/extracts/software-entropy
http://www.robertbeal.com/145/thoughts-about-unit-testing-private-methods-and-100-coverage
http://martinfowler.com/articles/mocksArentStubs.html