Portable dev env, cont'd

So far, the portability has been very useful.  I dropped the whole thing onto my work computer in a couple mins.  On the whole, saved me a lot of hassle.

The main tweaks I've made so far:

1. Environment variables.  For Java you need JAVA_HOME, etc.  Typically, the installer creates system vars, so you could manually do this, but I do something even better: create small shell files that sets it as a temporary user var.  Then you add a call to this file anywhere you need it.  For example, instead of executing Eclipse directly, create a shell file that first calls the file to set the vars, then executes Eclipse.  The beauty is that you can run multiple environments at the same time.  For example, because you're using user vars and not using system vars, you can run your app with JAVA_HOME pointed to 1.5 version, and then run it again with JAVA_HOME pointed to 1.6.

You need versions for each OS you run on, but that is pretty basic shell writing. 

2. Registration settings.  The installer typically sets the extension to automatically invoke the executable; i.e., .rb files can be run directly, without pre-pending with "ruby".  Minor issue to me, though you could easily write a script to registers the association wherever you are, but you might compromise your ability to run multiple environments.

A bigger deal is that in Windows, you lose right-click shortcuts.  In some cases, like 7zip and Vim, it's an inconvenience, but some others, like gitTortoise, you're out of luck.  No gitTortoise for you, just the git command line or built-in UI.  I do like Tortoise from my SVN days, so you could go with a mixed strategy: install locally where you can, but keep the portable version as part of the env in case you find yourself in a situation where you can't install an app.

3. gVim settings.  One of the pleasant surprises with Vim is that everything seems to be configurable, and thus it supports every feature I've thrown at it.  Here's the ones I've added to my default settings, critical to me for usability:
colorscheme murphy " you can get very granular, but I just chose one of the included themes
set number " turn on line numbers for all doc types
set expandtab " convert tabs into spaces
set tabstop=2 " set tab width
set shiftwidth=2 " indentation
set lines=60 " width of Vim window
set columns=80 " height of Vim window
set directory=..\..\temp " location of backup files
set backupdir=..\..\temp " location of temporary files
vnoremap < " de-indentation of highlighted text block using <
vnoremap > >gv " indentation of highlighted text block using >

4. network drives.  If you keep the portable env on a local drive, you won't have the problem discussed here.  Because you don't ever know whether you'll be dropping the env on the C: or the J:, on the root or in "/dev/my env/dir/subdir", you have to use relative paths and directory variables when.  So far, this has never been insurmountable, but the most difficult challenge was when I dropped the env on a Windows network directory. 

If you run a .bat file on a network drive, it can't change directory to there, so it defaults to the local Windows system directory, and all your relative paths are messed up.  Here's some good documentation on it, but the quick answer was using pushd "%~dp0" at the beginning of the batch file.  This creates a drive letter and points to the dir that you're executing the batch file in.  Now all your relatives paths are intact. 

-----
It might seem that getting the portable environment right sucks up a bunch of time.  That's true, but actually whenever you want a good personalized environment, you spend quite a bit of time installing and configuring your tools.  The great part about a portable environment is that you won't have to do it again!

Building a portable dev environment

I started a little coding side-project, and came to the idea of putting the whole dev environment on a usb drive.  That way, I could plug it in wherever I am, on any computer, and have all the tools I needed without any additional setup.

An additional benefit: to bring another developer onto the project, I could just drop the contents of the drive on their computer.  They'd be instantly ready to roll, with close to zero start-up effort.

I have a 8GB drive, but so far the environment uses only about 500MB.  Here's what I installed:

* JRuby.  The project calls for Ruby, and I went with JRuby because it's extremely portable: just extract the binary zip to wherever you need it.  You might be able to setup Ruby on a flash drive, but the options sounded more experimental. 

* Java SDK.  Turns out you can direct the java installer to point right at your USB drive.  Here's more detailed instructions.  Very easy, ran across only one bug -- closed by Sun though not fixed -- that didn't have any effect on performance / function.

* Eventually, I'll need a j2ee container, but I feel pretty confident I could get resin or tomcat or glassfish working on a drive.  Installing webapps on a j2ee container involve nothing more than just a .war file drop, of course, so dropping in Rails should be a snap.  I don't need Rails just yet, but I'll post an update when I try that.

* Git.  Git offers a portable version, and is more appropriate for distributed source control than something like subversion.  If I used subversion, that would mess up the whole goal of keeping it on a flash drive.  I wouldn't be able to drop the env to a friend at a moment's notice, and then we'd need a central server stored somewhere.  Git might need some additional config if others join, but for now it's just me.

Of course, using Git might seem like overkill to some.  I like to save versions frequently, and I quickly get tired of and confused by all the .old and .dated files and directories lying around.  Git will eliminate all that clutter.

* Firefox.  While coding, you always end up with a ton of reference and investigation links.  I thought taking my browser with me would be better than storing links on a 3rd party site, which I've never really liked.  IE is not a possibility, and I prefer Firefox for the add-ins anyway.  This ended up working very well (as long as you bump up the cache, which defaults at 0), except for one problem, which might be considered serious.  If you run the portable version, with all your dev-related links, and then you try to run your local version, Firefox just runs another instance of the portable version.  So, you can't get to any locally-installed links without closing the portable version.  One workaround would be to store all links on all your Firefox installations, but I wasn't really looking to store, for example, my company's HR website on my dev env, especially if I want to drop the whole thing to another developer at some point.  Will have to think about this one a bit more.

* gVim.  I want to use the same code / text editor on any computer; your productivity increases as your expertise increases.  Yes, going to gVim feels a little like using WordPerfect on the dawn of Word's impending domination, but gVim is much more robust than Notepad++, which might be the only other possible contender meeting these two criteria: free and portable.  Frankly, I found very few features that gVim didn't have or couldn't be added to the default configuration.  A robust text editor decreases the likelihood of needing to fall back to a full-blown IDE, which I'm not sure would run well on a flash drive (though some say it works).

That's where I am so far.  Other than the remaining Firefox issue, the results have exceeded what I thought was possible.  I'll provide updates as my env requirements grow or unexpected issues pop up.

Happy holidays coding

Every year, I pull out the holiday lights and always (so far) receive a pleasant surprise.  "Wow, look how organized these lines of lights are!"  Bundled, neatly folded up, tied in key points. 

And that's when I remember that the year before, when I put them away, I took extra time to do it right.  Undoubtedly, other needs pressed -- lots of cleanup, snow to be shoveled, an urgent need for more exercise -- which is why, honestly, it never fails to surprise me when I take them out again.

Why don't we treat our code equally well?  People shove code in, sometimes with little regard to ever using it again.  Bad names, broken formatting, long run-on functions, no design considerations, etc.  It's a giant knot of holiday lights the next time someone opens the box.

There are 3 main reasons.  The reason you might hear most often -- "I was in a hurry" -- doesn't hold up.  Time pressures rarely put a gun to our head.  When NYSE goes down -- now those coders have an excuse!

The rest fall into these buckets:

1.  Tragedy of the commons.  Whether you do your part or not doesn't affect you that much.  Rather, it contributes or detracts from the group good.  So, you prefer to spend your time on things that will benefit you more directly.  In other words, your box of lights goes into a community pile, from which you'll pick a random one next year.  The best way to combat this is by cultivating a group culture that values well-written code.

2.  Discipline is difficult.  The temptation to slide by, just this once, draws you in.  You worked hard at it every day this month, you've earned the right to just shove something together and go home early this time.  If you're feeling this way, you're probably a bit burned out.  Take a break.  Take a day off if possible.

3.  I just don't care.  Often because the gain is too far away.  In other words, anyone who shoves their lights in the box without any effort to be organized, knowing they'll be the one pulling it out in a year.  This is the hardest one to overcome; the guy is crapping in his own yard, what can you do?  Sometimes, appealing to professionalism can help.

Of course, it goes almost without saying that the extra time I spent putting the lights in carefully was easily repaid by the short time it took to get them strung up again this year.  And now I can relax.

Forcing focus

Recently, I found myself struggling to focus. I think the change of pace to coding threw me off.

As a manager, I zipped around constantly; frankly, there were too many conversations and issues all up in the air, so I had to choose, but there was always something. As a developer, I spend a much greater amount of my time staring at the screen. Plenty of things to do, of course, but at my own pace. Which is good, just different.

So, I implemented something that forced me to stay on track: I log the time and a very, very brief description whenever I start doing a new task. Says here today I started at 8:50 reading email, got some breakfast at 9, did some data scripting from 9:05, etc.

After about three days of this, I learned two things very quickly.

First, forcing myself to note the time whenever I switched tasks made me accountable. If I took an hour lunch break, it's there in the log. I know where my day went. When I'm spending too much time chit-chatting or doing non-productive tasks, I become conscious of it quickly, and feel pressured (by myself) to get focused on real work.

Second, I was hopping between tasks too often. I still do, but now I'm better about trying to bundle my interruptions. If I'm in the middle of coding, then I'll wait until I have to go get food before I also reply to some emails. This should be intuitive, but I'm so used to interruptions that I was distracting myself constantly.

I don't find anything wrong with realizing that I might need help to stay disciplined. Sure, I'd rather be ultra-disciplined all the time, but better to realize the need and take action than to continue failing. As they say, if you can't stop eating all the ice cream, don't keep any around the house.

(Side note: logging all my tasks is quite a bit like Dave Winer's suggestion to Narrate Your Work. Though my list remains private, I still accrue to myself some of the same benefits.)

Old friends

For the first time in a while, I am spending the majority of my day coding. Very little management going on for me right now.

I have some other thoughts about it, but one thing that really struck me is totally not code related: listening to music again at work. Since I rarely listen to music outside of work, I enjoy it immensely.

When you're managing, you need to talk to people constantly. Interruptions come frequently, or you go off to interrupt someone else.

Listening to music again has been like re-acquainting myself with old friends. Friends with funny names like Radiohead, but still.

Backgammon and good code

I'm not yet done with How We Decide, but this example of deliberate practice jumped out at me.

Robartie didn't become a world champion just by playing a lot of backgammon. "It's not the quantity of practice, it's the quality," he says. According to Robertie, the most effective way to get better is to focus on your mistakes.

You might recall that deliberate practice appears in Talent Is Overrated as the key to high performance. Focusing on one's mistakes reminds me, again, of Test Driven Development. I'm not a TDD nut -- I have never personally worked on a devoted TDD team -- I'm just pulling these threads together. TDD focuses on your mistakes before you make them. You'll still make some, and so your TDD gets better and better, along with your results and productivity.

This sentence also struck me:

After a few years of intense practice, Robertie had turned himself into one of the best backgammon players in the world. "I knew I was good when I could just glance at a board and know what I should do," Robertie says. "The game started to become very much a matter of aesthetics. My decisions increasingly depended on the look of things, so that I could contemplate a move and see right away if it made my position look better or worse."

I have long believed that good code is clean code, exemplified by Joshua Bloch's axiom that good code reads like prose.

Concern for aesthetically-pleasing code strikes some coders as just unnecessary housekeeping chore. But I think programmers who consistently produce solid code write clean and organized code as a matter of routine. And those who don't do it routinely, but have to go back after the fact and clean things up, maybe refactor a bit, will improve over time and eventually get to the point of routine. It no longer becomes a chore, but just a standard way of doing it.

The counter-argument is a whistle-clean, beautifully modular code that fails miserably at what the users wanted. That's actually a different problem: bad communication or bad specs. But I bet that code did exactly what the programmer wanted it to do.

Slow software practices

Alex Tabarrok highlighted this chart about eating. Somewhat paradoxically, more time spent eating actually resulted in lower obesity.

I wondered whether a similar effect might exist for software.

It's easy to think of a hypothesis where this could be true. Quickly built software, with fingers flying, usually allows little time for planning. The goal is to get the code out as soon as possible, and lots of code is what happens.

All that code begets more code fixes. Also, the code likely doesn't involve a lot of abstraction, having been built for some specific cause, the emergency of the day. Thus the next emergency requires new, specific code for it, as well.

Perhaps a slow software movement is what we need. Of course, our company owners and business managers would freak out if we started saying that we're adopting "slow software practices".

So, of course, we call it other things -- requirements gathering or architecture review or quality control. But at the heart, one of the main elements of all of them is to slow things down, to create room to stop and think.

Keep that in mind the next time you're facing a project. Yes, it needs to get done right away, but invest in yourself and the project, and slow down.

What else I hate about Microsoft "programming"

Answer: I never know what's actually going on!

I work with some guys who are pretty knowledgeable about Microsoft development, very solid guys.

Still, when something goes wrong -- the data load craps out, the website doesn't show some reports, the reports server doesn't respond quickly -- the usual answer is to simply restart. "Restart IIS" or "delete the reports and then re-deploy them" or "delete all the data from that table and reload it from scratch". Sometimes, I'll have an error that will persist for half a day and then suddenly disappear, and they'll say, "Yeah, that happens sometimes, you just keep trying."

Restarting frequently works. If you've ever worked on a support desk, you know that one of your friends is the "restart your PC". That often resolves whatever weird error your user has.

But that isn't solving the problem, it's just erasing the situation that created it, in the hopes that it doesn't occur again.

I think part of the problem is that Microsoft targets users (in this case developers) who just wants to get it done, and not users who want to know the dirty details. Creating anything involves going through a wizard process. They spend lots of resources making sure things happen automatically.

What they don't spend their resources on is making the underlying mechanics visible. Typically, you'll get an error, and then the strategy involves looking up the cryptic error message in Google. From that, many different suggestions will be returned, and you look through each one to see if it applies to you. This is not debugging.

As a result, I never feel confident that things will work correctly. I never feel confident that when things fail, I will understand why they're failing. I never feel confident that if I don't know why things are failing, I can debug it to the point where I can find a root cause.

It takes impressive effort to automate so much, and for a lot of people -- Microsoft's target audience, presumably -- this gets them to the point they need to get to. So, this is not intended to be a Microsoft bashing exercise.

At the end of the day, though, as I mentioned before, if all programming was strictly programming on the Microsoft platform, I doubt I would have lasted long as a programmer.

What I hate about Microsoft "programming"

Answer: it doesn't feel like programming!

For the first time in a very long time, I'm doing development on a pure-Microsoft app: sql server, asp.net front end, sharepoint, etc. This is all new technology to me, though of course it is similar to other things I've done.

What's brilliant about Microsoft stuff is it can generate a multi-layer, multiple-technology application in a very short time.

Along the way, I write a few queries and such, but mostly I find myself configuring. Configure the reports server to talk to the database. Configure the web server so it doesn't have port conflicts. Configure the deployment tool to generate to the right location. Draw lines between two tables to configure the data mapping.

I'm not suggesting this is easy. The details behind each configuration holds many hours of potential problems, and expertise pays for itself.

In the end, though, I feel more like a sys admin than a programmer. Perhaps it is a compliment to Microsoft that entire apps can be built without knowing much about code. But, just as I was never interested in being a sys admin, I never would have become a programmer if this is all it consisted of.

Comprehensive Agile survey

Jurgen Appelo is running a big survey to try to identify what Agile practices actually get used.

I like the idea of the survey because I've often wondered how much real pair programming actually occurs. I've never run into it personally, nor have I run into someone who had done it.

Mostly, I wonder how many implementations match what I read to be the definitive way to do it: 2 coders partnering, then switching partners, and thus tasks, every 4 hours. Everyday, a coder would have 2 partners and up to 2 tasks. I believe I read this in Robert Martin's Agile Principles, Patterns, and Practices (one of the best programming books I've ever read), though I could be wrong. 2 partners and 2 tasks everyday seems crazy to me.

Unfortunately, after taking the survey, it doesn't seem like I will get a good answer to my question, as the questions are not that granular. I will note, however, that pair programming seems to have one of the lowest adoption rates among the well-known, often-mentioned Agile practices (e.g., user stories, scrum, TDD, etc.)

But that's okay, it's still useful to take the survey, so go ahead.

Is software evil?

Michael Osinski compares his creation -- software that enabled the creation and selling of complex mortgage products -- to another "evil engineering" feat, The Manhattan Project.

I have been called the devil by strangers and “the Facilitator” by friends. It’s not uncommon for people, when I tell them what I used to do, to ask if I feel guilty.

Osinski claims his software, Intex, dominates all the investment bank mortgage departments, and I can confirm that my investment bank used and continues to use it. So, Intex certainly played a role in the financial crisis.

It remains far short of the old Manhattan Project though on the evil scale. A better metaphor would be the creation of some math formulas that the engineers used in the Manhattan Project. The toxicity resides in the product, not so much in the tool that created the product.

But, still I find the question intriguing: should software engineers consider the ethical ramifications of their work?

Would you have any problems being hired to write software that:

• recorded key entries and mouse clicks without the user's knowledge?
• tracked people's on-line browsing behavior without their knowledge?
• breaks passwords through repeated guess submissions?
• connected adults looking to cheat on their spouses?
• scanned public email of all Americans, not just those suspected of illicit behavior?
• simply stored and queried the data from the database of public email?
• infiltrated government servers and stole political information, such as GhostNet?
• disabled thousands of servers, i.e., a worm or virus?
• stole money from people's checking accounts?

Somewhere between the first and the last, harming someone else becomes explicit and intentional. Osinski's case sits close to the first, where the ultimate damage caused was far removed from the purpose of the software. (Excel was probably just as much to blame, if not more.)

Also, somewhere between the first and the last, everyone finds a line that they wouldn't cross (or at least think they wouldn't).

Think about the application you work on. Could someone likely be hurt by it? Is it likely to be used for illegal purposes? Should you care?

Data hell, part 2

A question came in, What do I mean by data hell exactly? (Presumably from a UI guy :-)

Here are some of the characteristics:

1. duplicate data in multiple tables. Of course, data from one table doesn't match the others -- it never does because that's what happens with duplicate data.

2. no primary keys on tables. How to identify programmatically the exact row that you want? Not easily. Sometimes filters work in the right context -- e.g., "I know know there's only one manager in New Jersey, so I can filter to New Jersey and get the right manager." Painful.

3. columns that act as pseudo-primary keys vary by table, even if it's the same data. For example, every car has a unique license plate. That's the primary key for Table1. Wait, every car also has a VIN. That's the primary key for Table 2. Now join them together. Exactly.

4. the pseudo-primary keys aren't reliable. The column in Table1 that holds license plates has VINs for some rows. The join suggested in #3 to Table2 actually works -- sometimes. When? Hard to say.

5. cross-keys aren't reliable. To solve the problem, VIN is often added to Table1 or license to Table2. If you're really "lucky", as I am, you'll have both in both. Unfortunately, sometimes the two keys are out of sync, as predicted in #1: a car gets in Table1 with the one license and VIN, the VIN matches to a VIN in Table2, but that row in Table2 has a difference license.

6. multiple naming conventions. Can't remember if the column is spelled "licenseNum", "LicenseNum", "license_num" or other variations with "number", "no", etc.? That's because each table has a different variation. Often, a table will be consistent with itself (though often it won't be), but different programmers add new tables and bring their own convention without concern for fitting in.

7. just lots and lots of bad or missing data. Data entry just not happening reliably.

What do you do about it? It depends on how much time you have.

If it's a long project, you painstakingly fix the data in some key tables, make sure the input processes are clean. Then you build out from there, either refactoring tables or fixing data at each point along the way. All the while, you try to fit in other business requests as you can.

Sometimes, the business can't wait, and the requests pile up faster than you can take them off. Your only hope is to hire some good database developers and work like crazy. Keep at least one or two of them away from the firing lines, so they can fix things in the background.

Worst is if it's a short-term project. You might think that, being short-term, that is the easiest. Ride it out until it winds down, right? If you've been an app developer for any amount of time, you know that apps very rarely die. If the app works, someone will find a use for it. But, because it's considered short-term, no funds will be invested. It will go on living like a zombie. The people who keep these things going are real heroes.

I had forgotten what data hell felt like

Now I remember, thanks to the new databases I have inherited!

That is all.

10 papers every programmer should read

Michael Feathers says everyone should read these at least twice. Pointers like this -- filtering the many papers he's read over the years down to these ten -- are invaluable.

I found all of them online for free, except for #8, so I added links for them.

1. On the criteria to be used in decomposing systems into modules - David Parnas html pdf

2. A Note On Distributed Computing - Jim Waldo, Geoff Wyant, Ann Wollrath, Sam Kendall pdf

3. The Next 700 Programming Languages - P. J. Landin pdf

4. Can Programming Be Liberated from the von Neumann Style? - John Backus pdf

5. Reflections on Trusting Trust - Ken Thompson html pdf

6. Lisp: Good News, Bad News, How to Win Big - Richard Gabriel html pdf

7. An experimental evaluation of the assumption of independence in multiversion programming - John Knight and Nancy Leveson pdf

8. Arguments and Results - James Noble

9. A Laboratory For Teaching Object-Oriented Thinking - Kent Beck, Ward Cunningham html pdf

10. Programming as an Experience: the inspiration for Self - David Ungar, Randall B. Smith pdf

New rules: no more rules

This escalator sign doesn't really make sense. But at least they have the excuse of English not being their native language. Presumably, the Chinese makes perfect sense.

On the subway I take, they have no such excuse.

The tunnel where I get out is very deep. A very long escalator takes you from the subway to the top, where the ticket booth and street exits are. The escalator only goes up; to go down, you have to take the stairs or the elevator. Once you arrive at the top of the escalator, an LED sign greets you with this message:

"Escalator is for subway patrons only."*

What? Only people coming from the subway are on the escalator. The only way to be in violation would be to buy a subway ticket at this station, walk down the stairs, and then take the escalator back up. Just for fun. Even then, you paid for a ticket, so shouldn't you be allowed a free escalator ride?

Why is this sign there? The only reason I can think of is that, in the design stage, someone put in this "feature" thinking abstractly that it could be of benefit. Later, when it came time to actually use the sign, the only thing anyone could think of was mindless manager-ese -- something they read somewhere else.

"Restrooms are for customers only" on a restroom at the back of the store is another example. Whoever posted that sign probably saw a similar sign somewhere else, but failed to note that they probably saw it at the front door.

In software development, we should be following principles and practices, not rules. Principles are almost philosophical; it's what you believe.

Practices are not rules. Practices derive from principles. This is where confusion creeps in. Is "Make your classes final, do not allow extensions" a rule or a practice? How about "All classes should have a unit test"?

The key is whether you're being motivated by principle. If you understand the principle, and that's why you're following the rule, you're really following a practice. If you don't understand the principle, or you're following the practice regardless of it, you're actually following a rule.

I see now that Robert Martin also has a related post yesterday, specifically about SOLID principles. Scott Bain also has an good write-up on this in Emergent Design.

* I would take a picture, but that is verboten in this high security era.

Testing smackdown, part 4

Not to keep cross-posting Jeff Atwood (a sign of respect, anyway), but he responds to Robert Martin's criticisms directly.

At what point do you stop having a set of basic, reasonable programming
guidelines -- and start being a Ferengi programmer, an imperfect manifestation
of the ruleset?

This has expanded beyond testing and into development principles, but TDD is still a good straw man for the conflict.

Note that Jeff's argument here is not the same one I was making before about why TDD or another development principles might be pushed aside sometimes (basically: real-world constraints get in the way).

In fact, even though Jeff and I seem to reach the same conclusion, I disagree with Jeff here. This might not be his only reason for discounting the importance of TDD, but at least this particular argument seems weak.

I'm not sure you could follow Robert's principles "mindlessly", so I don't see any problem with putting them out as design goals.

Several years ago, when I started coded heavily in Java, I read that you should only code to interfaces. It also said that behavior used by multiple implementations should be kept in abstract classes; in anticipation of that, it recommended extending abstract classes as a rule, so that specific behavior could be easily moved up if you wanted to generalize. And, actually, those rules make a lot of sense to me.

So, on my next assignment, I did exactly that. Every single class extended an abstract class. Every instantiation was to an interface. To some degree, mindlessly.

But, to some degree, thinking was always required. Do I need this method in the interface? Which part of the behavior goes in the abstract class and which in the implementation?

The real downside was the massive number of files. More files had to be maintained. Creating a class took more time, because I had to create 3 files every time I wanted one. Once that was done, however, the burden wasn't too great, and I did see some benefits later when I went back for modifications and extensions.

On subsequent projects, I created interfaces and abstract classes more selectively. This is closer to the agile idea of limiting complexity to what you know you will need.

With this practice, or pick a principle, say the Single Responsibility Principle, it's much easier to ignore it and just plow forward with the code.

I haven't yet run into a case where someone was absurdly over-applying solid development principles or practices; the incentives run the other way.

Testing smackdown, part 3

Robert Martin further responds to Joel Spolsky and Jeff Atwood:

If you had read through any of the articles and/or books I’ve written on these principles over the last 15 years, you’d have found that I don’t recommend the religious or dogmatic approach you blamed me for.

I'm following this confrontation because I think it represents one of the primary benefits of our current age. The two parties aren't just blowing hot air: one is one of the greatest software architects living, and the other two are extremely successful professionals with large numbers of followers.

If people are misquoting or misinterpreting you, by all means, take them out back behind the shed. Perhaps they have been irresponsible. Perhaps these are wide-spread inaccuracies, and this is a chance to set the record straight. That's what Robert's trying to do.

On the other hand, perhaps real problems emerge with the methodology in some circumstances.

It's all out in public, so everyone can understand the nuances of each argument and make their own conclusions.

Think if George Washington were blogging to correct Supreme Court justices trying to interpret the "founding fathers' original intent". Or John Maynard Keynes were here to set the economists straight about what to do with our current financial mess. (The economic blogging battles going on make this software discussion seem very polite by comparison.)

Hopefully we'll get more details from all sides about both interpretation and implementation problems around test-driven development, SOLID, Agile, and other principles.

Dilbert + agile

These are a bit old, but the Agile implementation at Dilbert's office is amusing.

Dilbert pairs up.

We need more programmers.

Thanks to Jose for the pointer.

Testing smackdown, part 2

Following up yesterday's post about the disagreement between Joel, Jeff, and Robert about testing and code quality, I was reminded about some code changes we had to make last year.

As I mention in my bio, I work at an investment bank. After the market closed one day, at 5 or 6 pm, we got a call from one of the business managers.

"We need to stop trading with Bear Stearns. Starting immediately, if Bear tries to buy something from us, reject the trade." This was back when everyone became seriously concerned that Bear Stearns would go out of business. (Eventually, JP Morgan bought them.)

None of us was even aware that the trading feed had a feature that allowed us to reject a trade based on the buyer, because no one on the development team had been around when the trade feed was written 8 years before. And Wall Street had never stopped trading with another major bank in our lifetimes.

With the clock ticking, we investigated how the feature worked. After a couple hours of talking to business people and developers at the exchange (where we send our trades), we figured out what fields to check, how to reject the trade, etc.

And then we hard-coded Bear Stearns into our trade feed. Hard-coded. Ran a couple unit tests ourselves, dropped it in for unit testing.

It was coming up on 9 pm, so the business manager ran a couple of very quick tests, gave us the thumbs up. Developers at the exchange confirmed our messages looked good. We pushed it out to prod.

That was fine, until a few weeks later, when Lehman started wobbling. "We need to stop trading with Lehman."

Again, we hard-coded it.

Does this make you cringe? In this case, I think what we did was exactly right.

Now, you might say that we got lucky. This was major business functionality, and it went in with maybe a half hour of testing all together. And you might also say that the fact that we had to go back and break the code open again later for Lehman shows the problem with our rushed implementation.

But, frankly, part of doing your job is knowing your code and understanding the risks you're taking. That's what Joel is talking about when he says he wants to step back from hard rules because he fears that people will stop thinking.

What risks did we take? The main thing we considered is, "What is the cost of hard-coding this vs the cost of doing something well-designed?" Knowing our code, we decided hard-coding would save a lot of time with not much downside. The main reason: you only have a small number of trading partners that you could remove from the feed before you'd end up not needing the feed anymore! Yes, we had to bust it open again for Lehman, but that was maybe another half hour, since this time we knew exactly where to put that code.

And, of course, as it turned out, we've never had to remove this code.

So, from a time perspective, we spent less than 2 hours coding this and never touched it again. It'd be impossible to get a better return on a richer, more robust implementation.

How about the testing? The two best testers are the business manager and the developers at the exchange -- they know the features and exactly how they want it to behave. Due to the time of day, every minute we delayed meant less likelihood we would have the these experts around, if not physically then mentally. So, I wanted to get the results into their hands as quickly as possible, even though that meant shorting our own testing.

What would have been completely unacceptable would be to say that we couldn't implement it in time, before start of trading the next day. And that's what Jeff's getting at when he says

Because the unit of measure that really matters to me is, how quickly you can respond to real business needs with your code. And by that I mean, how well are you serving the people that are using your code. To me that's what it's all about.

This is in the paragraph that leads up to the infamous quote. It's important to think about that when you consider his later remarks.

Testing smackdown, part 1

It's not often you see the heavyweights go straight at each other, so when they do, it's worth taking note -- not for the spectacle, but to try to understand how they could be so far apart on something fundamental.

Robert C. Martin is deeply troubled by something Joel Spolsky and Jeff Atwood said. Yes, the Robert that wrote one of the greatest dev books ever. Yes, the same Joel and Jeff that inhabit spots 1 and 3 in Jurgen Appelo's top 100 software dev blogs.

Here's the offending quote, as spoken by Jeff:

I don't want to come out and say I'm against [unit] testing, because I'm really not. Anything that improves quality is good. But there's multiple axes you're working on here; quality is just one axis. And I find, sadly, to be completely honest with everybody listening, quality really doesn't matter that much, in the big scheme of things.

Here's Robert's reaction:

I was riding my exercise bike, listening to Stack Overflow #38 when I heard Jeff Atwood and Joel Spolsky say “Quality just doesn’t matter that much.” I nearly fell off my bike.

I mean this is Joel Spolsky right? His buddy Jeff makes this incredibly dumb statement and Joel grunts his approval. WTF?

This is the kind of idiotic statement I would expect to hear from junior hackers, or people who haven’t written very much code.

The condemnation would have been much harsher if "Uncle Bob" hadn't interpreted the offending comments generously.

Joel may be aware of the controversy, because he promptly posted on his own blog part of the transcript from the podcast, which I don't recall him ever doing before. Read it and form your own opinion about the context of Jeff's comments.

I will have my own opinion about it tomorrow after you've had a chance to consider your own.