development

Remember Who Is In Control

We are. Not the computer, not the software.

If a piece of software doesn’t behave the way we’d like, if it doesn’t complete the desired task, it is broken. Acknowledge that is broken, acknowledge that we performing work arounds, then fix it.

To do anything else is to relinquish control. We become slaves to a master that doesn’t even know we exist. If that happens, nothing gets better.

Remember, we are in control.

Don’t Program In The Database

On a particular project many moons ago, I was tasked with creating a reporting system for a small number of retail stores. It was bunch of stuff about inventory, sales, and the like. Something that has been done a million times. And, unfortunately, will be done a million times more.

As this was one of my first programming jobs using the Java/J2EE stack, I tended to stick with what I knew. Or, maybe it is better to say I stuck with what I knew would work. In this case, it was the database. The bulk of creating a report was done in the database with some rather huge and gnarly queries. The queries did their jobs, I took the results pushed them up a layer (a *very* thin layer) for processing, and then presented the data to the user.

Things went well. I even did that new-fangled thing called “refactoring” on the processing layer. I created some very broad and generalized code for taking results from the database and putting them in a UI. Look at how awesome I am!

Then, of course, things went to Hell. The users wanted more information in their reports that couldn’t easily be obtained by modifying the aforementioned gnarly queries. I couldn’t capitalize on the processing layer simply because there really wasn’t much of one. And, the one that was there was uber-coupled to how results were pulled from the database.

After much investigation, I determined that the modifications were really going to have be rewrites. I just couldn’t justify “making it work” when I knew there was eventually going to be yet another round of changes. I needed to split the responsibilities of the processing and the database appropriately. It was going to be work.

Luckily, the customers were OK with the rewrite. I laid out all the mistakes I had made, and what I really needed to do, and they said “Do it.” I was lucky to be working with these folks. Otherwise, this would be a story about losing a job.

In the end, I did the rewrite. I am not going to claim that the rewrite was the end-all, be-all for what we needed. But, it worked. It handled changes over a couple of years, and there was only that single rewrite.

The pain I experienced taught me a lot about layering and how to think about using databases. If there was one thing I took away from the experience, it was this: don’t program in the database!

Fail Fast

For the last two months I have been on a project where I’ve violated a number of core development principles. Right now, I want to talk about one of those principles. (The others could be their own post or series of posts).

FAIL-FAST

When people talk about failing fast, they are usually talking about it in the context of an application’s runtime. Instead of letting something bad continue, end it. Rather than propagating errors, fail as fast as possible so we can recover as fast as possible.

Another context in which fail-fast is applicable is in the notion of development progress. That’s what I am going to talk about. You see, on this particular project I made the code, well, err, more forgiving than I normally would. I did this so I could get feed back on some other aspects of the code and to make what should have been only throw-away code “work”.

You can probably see where I am going with this. Because things “worked”, there was the illusion of a mostly-finished application. I am not going to lie—even I started to believe it. So, when I tried to explain that something wasn’t finished, or a fix for something would take X amount of effort, it didn’t register with the non-technical people. It looked like we were pretty much done.

I realized I did my customer and myself a disservice. An impression had been made. The code looked like it was almost done. If I had stuck with a fail-fast mindset, I would have had feedback for the customer to understand where we are. And, I would have the failures as a guide for what the next development steps should be.

Sometimes, a spectacular failure is what’s needed.

What A Little Communication Can Do

Over the last few weeks, my team has spent time refocusing on communication within the team as well as with our customer proxy. For various reasons, we had become a very siloed. There was little communication with our customer and only little more communication between team members–primarily among individuals that had to make their pieces of code talk to each other.

We’ve gone back to tasking out work on a regular basis. In the course of doing this, we all are talking about what’s there, what’s needed, what we understand, what we don’t understand. With this, we were able to figure out what we could comfortably commit to. Then, we worked with our customer proxy while we developed. This was done in conjunction with our normal code review/refactoring sessions. Lo and behold, the iteration went smoothly.

I hope the rest of the team feels the same way. I know from a lead/senior development perspective, it was less painful (almost fun!) than previous iterations. I’d almost forgot what a little communication can do.

Testing Tools Need To Be Sharp

Too often, I hear people say “It’s test code, don’t worry about it.” Well, I disagree. I think the quality of your test code is reflective of the code base in general. The same issues that we have have with the test code we will have with production code. If you see duplication, high complexity, low cohesion, etc. in the tests, odds are you are going to see it in the production code base.

The tests and the supporting code need to be well factored. If a “small” change in the production code is going to result in a large amount of painful changes in test code we are less likely to make the proper changes. And thus begins the downward slide.

It is better to apply DRY and SOLID principles every time we write test code. We get more time practicing the discipline needed to write quality code, and we set ourselves up for an easier time in the future.

A Software Commons

Where are the places where all stakeholders can meet? Not a physical place, but artifacts, tools, etc.

Assuming that our stakeholders are comprised of customers, developers, testers, database, support, and deployment working in their own silos, what is the common ground between any two?

Worst case:

Customer Developer Tester Database Suppport
Customer - Requirements Requirements Requirements Requirements
Developer Application - Application DB Schema Application
Tester Bug Report Bug Report - Bug Report Bug Report
Database DB Schema DB Schema DB Schema - DB Schema
Support Application Application Application DB Schema -

We can see that in a worst-case, siloed organization, the conduit for communication can differ. Even between two stake holders, the conduit can differ depending on the flow of information (e.g., the conduit between Customer and Support is not the same as Support and Customer). Each conduit is a chance for miscommunication.

So, the common ground between any two can vary. What if we have an idea or issue that spans several silos? More and possibly very different artifacts will have to be taken into account. Because these artifacts were created by different groups, there’s a good chance there is going to be miscommunication and wasted effort.

Now, let’s tweak the terms like “Requirements”, “Application”, “Bug Report”, and “DB Schema” for a best-case description of said artifacts/conduits:

Requirements: Description of what the Application should do with runnable examples and text
Application: The system that will be deployed for users, as well as all the tests, set up scripts, etc
Bug Report: Automated, report-as-soon-as-you-find-it test failure
DB Schema: The DDL and DML used in the database in support of the Application

What we really need is something that takes the best-case description of the artifacts and holds them in a common area for all of the stake holders. This common area would be the spot where we all can talk the same talk, manipulate the necessary artifacts, and if need be, see the appropriate “view” (as a Customer, as Tester, etc…) of the software as a whole.

Where/what is this commons? Well, it doesn’t fully exist yet. Things like Fit, FitNesse, and BDD almost get us there. The next generation of tools such as Twist get us even closer.

Extension is about Consumption

The next time you are about to extend a class, stop and think about who is going to be consuming this extended class. If the extended class (instead of the base class) is in the contract amongst collaborators, why the extension?

If, somewhere along the line, you want your extended class to be used (consumed) as the base class, go ahead and extend. If that isn’t the case, then maybe composition should be preferred.

It really comes down to this:

Extension is about consumption.

Disconnect!

Quick–how many of your tests would fail if you were not connected to a network (for database, files, services, etc)? How many of these are labeled as “unit tests”?

Do you have a way of quickly determining which tests are dependent upon that network connection? Can you easily run all the tests that aren’t dependent upon the network? Do you find that there are errors/failures in surprising areas?

Try it and see. You may learn something interesting about your code’s dependencies.

Purge!

Quick–when was the last time you just deleted the directory for you project(s)? Could you do it right now without any problems? What would you need to recreate your set up?

Once you have your setup, it seems pointless to spend any more time trying to make it easier/faster to create said setup. And maybe it is for *you*. But, the next person coming along probably won’t consider it pointless.

Be hardcore. Purge that stuff right now and see what you need to do to get going again. You’ll find out quickly what you (and your new team member, your CI, your deployment team, etc) need.

Be Glad It Is Difficult

In the past couple of months, I have run into a few nay-sayers while attempting to build some test infrastructure. Many times, when I propose a change, the response I get is basically this:

We could probabaly do that, but it would be hard.

Well, duh. That is why we get paid. If it was easy, we wouldn’t have a job. If at any given time you know the solution to all the problems facing you at work, you can be automated out of a job.

Maybe the reluctance is a form of job security. But, I don’t think that holds, either. Why? Because there are always problems to solve. Solving a difficult problem frees you up to solve even more difficult problems. If you are always doing that, I don’t think you’ll have a hard time keeping your job.

Now, I’ll do the disclaimer-type thing. I am not saying we should do something just because it is hard to do. But, in life many things worth doing are difficult. And, if somebody doesn’t know how to do this difficult thing, they hire people like us.

So, be glad it is difficult.