Testing smackdown, part 2

Following up yesterday's post about the disagreement between Joel, Jeff, and Robert about testing and code quality, I was reminded about some code changes we had to make last year.

As I mention in my bio, I work at an investment bank. After the market closed one day, at 5 or 6 pm, we got a call from one of the business managers.

"We need to stop trading with Bear Stearns. Starting immediately, if Bear tries to buy something from us, reject the trade." This was back when everyone became seriously concerned that Bear Stearns would go out of business. (Eventually, JP Morgan bought them.)

None of us was even aware that the trading feed had a feature that allowed us to reject a trade based on the buyer, because no one on the development team had been around when the trade feed was written 8 years before. And Wall Street had never stopped trading with another major bank in our lifetimes.

With the clock ticking, we investigated how the feature worked. After a couple hours of talking to business people and developers at the exchange (where we send our trades), we figured out what fields to check, how to reject the trade, etc.

And then we hard-coded Bear Stearns into our trade feed. Hard-coded. Ran a couple unit tests ourselves, dropped it in for unit testing.

It was coming up on 9 pm, so the business manager ran a couple of very quick tests, gave us the thumbs up. Developers at the exchange confirmed our messages looked good. We pushed it out to prod.

That was fine, until a few weeks later, when Lehman started wobbling. "We need to stop trading with Lehman."

Again, we hard-coded it.

Does this make you cringe? In this case, I think what we did was exactly right.

Now, you might say that we got lucky. This was major business functionality, and it went in with maybe a half hour of testing all together. And you might also say that the fact that we had to go back and break the code open again later for Lehman shows the problem with our rushed implementation.

But, frankly, part of doing your job is knowing your code and understanding the risks you're taking. That's what Joel is talking about when he says he wants to step back from hard rules because he fears that people will stop thinking.

What risks did we take? The main thing we considered is, "What is the cost of hard-coding this vs the cost of doing something well-designed?" Knowing our code, we decided hard-coding would save a lot of time with not much downside. The main reason: you only have a small number of trading partners that you could remove from the feed before you'd end up not needing the feed anymore! Yes, we had to bust it open again for Lehman, but that was maybe another half hour, since this time we knew exactly where to put that code.

And, of course, as it turned out, we've never had to remove this code.

So, from a time perspective, we spent less than 2 hours coding this and never touched it again. It'd be impossible to get a better return on a richer, more robust implementation.

How about the testing? The two best testers are the business manager and the developers at the exchange -- they know the features and exactly how they want it to behave. Due to the time of day, every minute we delayed meant less likelihood we would have the these experts around, if not physically then mentally. So, I wanted to get the results into their hands as quickly as possible, even though that meant shorting our own testing.

What would have been completely unacceptable would be to say that we couldn't implement it in time, before start of trading the next day. And that's what Jeff's getting at when he says

Because the unit of measure that really matters to me is, how quickly you can respond to real business needs with your code. And by that I mean, how well are you serving the people that are using your code. To me that's what it's all about.

This is in the paragraph that leads up to the infamous quote. It's important to think about that when you consider his later remarks.