Is software evil?

Michael Osinski compares his creation -- software that enabled the creation and selling of complex mortgage products -- to another "evil engineering" feat, The Manhattan Project.

I have been called the devil by strangers and “the Facilitator” by friends. It’s not uncommon for people, when I tell them what I used to do, to ask if I feel guilty.

Osinski claims his software, Intex, dominates all the investment bank mortgage departments, and I can confirm that my investment bank used and continues to use it. So, Intex certainly played a role in the financial crisis.

It remains far short of the old Manhattan Project though on the evil scale. A better metaphor would be the creation of some math formulas that the engineers used in the Manhattan Project. The toxicity resides in the product, not so much in the tool that created the product.

But, still I find the question intriguing: should software engineers consider the ethical ramifications of their work?

Would you have any problems being hired to write software that:

• recorded key entries and mouse clicks without the user's knowledge?
• tracked people's on-line browsing behavior without their knowledge?
• breaks passwords through repeated guess submissions?
• connected adults looking to cheat on their spouses?
• scanned public email of all Americans, not just those suspected of illicit behavior?
• simply stored and queried the data from the database of public email?
• infiltrated government servers and stole political information, such as GhostNet?
• disabled thousands of servers, i.e., a worm or virus?
• stole money from people's checking accounts?

Somewhere between the first and the last, harming someone else becomes explicit and intentional. Osinski's case sits close to the first, where the ultimate damage caused was far removed from the purpose of the software. (Excel was probably just as much to blame, if not more.)

Also, somewhere between the first and the last, everyone finds a line that they wouldn't cross (or at least think they wouldn't).

Think about the application you work on. Could someone likely be hurt by it? Is it likely to be used for illegal purposes? Should you care?

Better software through connecting

This past weekend, I went to the dreaded DMV -- Dept of Motor Vehicles. If any government agency brings forth horrifying images of incompetence, bureaucracy, and disaffected employees, it's the DMV. Even though I was just renewing my license, I took a huge book, in preparation for long waits and lines.

I never even cracked the book. I was in and out within 15 minutes.

Actually, I can't remember ever having a bad experience at the DMV. Mediocre, maybe, but not bad. Yet, the imagery persists in my mind, and I would guess most people's.

I was thinking about this because it seems like a large percentage of IT shops have similar reputations. They do decent work, pump out a good share of deliverables, chip away at the list of problems. Yet, they are perceived as slow, lazy, and second-rate.

Some problems are about basic management, all of which practices like Agile addresses: expectation setting, progress tracking, frequent user feedback, quick error identification.

But, it's often not a competence issue. At the DMV, where things were effective enough, what I noticed is that none of the employees seemed happy. Occasionally, you'd see them joke with each other, but the customers were like fish they had to process. All interactions were perfunctory, not personal.

Government agencies don't have a lot of leeway to show personality, but there's no real excuse for a software dev team.

No matter how good your applications are, a weak relationship with your users will doom you to a negative or, at best, mediocre reputation.

If your users work in the same building or nearby, one of your primary jobs is to get physically in front of them regularly. Daily is about the right frequency, for every person on the dev team from the intern to the manager. Ask how things are going. Get to know them as people. Connect.

As technical people, we often focus on the technical specifics that our users request. We neglect the personal aspect of the job. Yet, the definition of a service is that it's personal. The products get outsourced.

It costs nothing to be friendly and engaged with your users. The payoff is that your apps will be better, even if they're the same.

7 things that make a good manager

My manager being out on jury duty has led me to continue to muse about what makes a good manager. I'm thinking about it in terms of software teams, which means managers on car production lines could be very different.

I figure I've had about 20 jobs in my life, starting with the afternoon paper route I took on when I was 10. So, I must have had around 30 direct managers during that time. Surprisingly, I can only think of one or two that I absolutely could not work with. Of course, some were mediocre, but still had valuable good qualities. Several have been outstanding. Perhaps I have been lucky.

Here's my list of the most important things that a manager does, in order:

1. creates an environment that enables you to succeed. This can include both positive attributes (e.g., gives you enough authority to make the decisions that you need to, provides access to the required resources) and negative (e.g., beats back and filters scope changes and new requests, shields you from the political noise).

2. challenges you with "stretch" assignments in new areas. This could be new technology, leadership, or new responsibilities. In any case, you face the requirement to develop new skills, and that keeps it interesting. Of course, you have to be ready for it, and a good manager knows how far to stretch you.

3. ensures you are properly rewarded for your efforts. Many developers I've worked with have a difficult time asking for a raise or expressing their desire to be promoted. A good manager takes care of that and fights hard to get you recognized, while you are toiling away behind the scenes, away from the spotlight.

Both 2 and 3 are really flip sides of the same coin: motivation. There are numerous ways to motivate people, but they essentially boil down to personal development or public recognition. Everyone wants both, but everyone also has priorities, so some people will prefer 2 to 3, and others the reverse.

4. mentors and teaches. I have really appreciated managers that were vastly better than me, and I loved to step back and watch them work. The great ones explained their methods, identified areas where I could improve, and presented these issues to me in a way that encouraged me to do better -- as opposed to feeling worse and inadequate.

5. provides vision. At the "worker bee" level, it can be hard to see where the department is heading, much less the company. You might feel your efforts are Sisyphean, pointless. A good manager provides strategic direction and explains how your efforts contribute to that, while maintaining honesty and authenticity.

6. organizes the team to encourage interaction. Your co-workers represent your best learning opportunities. A group that works together will be more powerful than a bunch of individuals pursuing their own goals. Frequent and positive interactions often require the right structure or the right incentives to encourage them.

7. acts as a friend. Some people have told me they didn't want to be too "chummy" with their workers out of fear that they wouldn't get the respect they needed on the job. I've never understood this. I consider myself friends with all of the best managers I've had -- people I would want to invite to my sister's wedding. I maintain that it's always easier to work with people that you have a friendly rapport with than those you have only a business relationship with. Especially in software development, respect comes from being effective, useful, and knowledgeable, not from having a commanding personality.

Software development as group activity

The stereotype of a software developer is a kind of loner cowboy who gets the job done.

What bugs me about this is that, occasionally, a developer will create something from scratch that the guy in the next cube built a couple years ago. Or the implementation will be ill-conceived, even though the best architect sits no farther than a paper airplane could fly.

This makes me appreciate Agile development. I find it gets developers talking, even if it's simply an extended version of, "I'll do this, you do that." Developers who talk about their work are more likely to generate new ideas or solve problems effectively.

Teams are powerful.

In my experience, the basic tenets of Agile development -- e.g., daily scrum, rapid iterations, burndown, backlog -- together provide a useful scaffolding for getting individuals to work as a team. The first time I got all the pieces working, I was actually amazed at how the team took off on its own. The individual parts of Agile practices actually serve as a metaphor for a team: each part can be ineffective by itself, but together they become very powerful.

I only wish for better alternatives to pair programming. The guidelines for pair programming seem oppressive -- switching pairs every 4 hours! I haven't yet met anyone who actually does it as specified, and I haven't been courageous enough to try it myself (nor strong enough to convince any of my bosses).

But I definitely like the idea of getting programmers to work even closer together.

Jeff Atwood mentioned a very clever idea (which he apparently had 4 years ago) for how to screen for developers who will be more inclined to work with others:

I have my own theory about the ideal way to interview developers: have the candidate give a 10 minute watercooler presentation to your team on something they've worked on. I think this is a far better indicator of success than a traditional interview. You'll quickly ascertain:

• Is this person passionate about what they are doing?
• Can they communicate effectively to a small group?
• Do they have a good handle on their area of expertise?
• Would your team enjoy working with this person?

This is absolutely something I will try.

Are you a good manager?

One simple measure of your value as an employee: when you say you're taking a few days off, how concerned is your manager?

Does the opposite also hold true? Yes, it's a crude measure, and it's just one, but I think it goes the other way, too.

If you manage people, do you think they're relieved when you're out of the office, or are they worried? Are they more productive, or less?

My boss has jury duty, so she's out for at least a couple days, and possible more. That worries me. Every day that she's out, I get increasingly bogged down by administrivia, "emergency of the hour" fire drills, unfocused and not well considered requests, and thorny problems that don't seem to have good answers. These issues grow until, after a couple weeks, it takes most of my day.

One of the most important jobs of a manager is clearing and filtering impediments, interruptions, and distractions so that the people that work for you can do their jobs. Clearly, my manager does that.

On the other hand, we've all heard of the opposite type, too -- the kind you wish would just leave you alone. The kind that everyone's glad when he's on vacation. I've been fortunate not to have very many of these in long career.

Do you manage people? Which type are you?

Data hell, part 2

A question came in, What do I mean by data hell exactly? (Presumably from a UI guy :-)

Here are some of the characteristics:

1. duplicate data in multiple tables. Of course, data from one table doesn't match the others -- it never does because that's what happens with duplicate data.

2. no primary keys on tables. How to identify programmatically the exact row that you want? Not easily. Sometimes filters work in the right context -- e.g., "I know know there's only one manager in New Jersey, so I can filter to New Jersey and get the right manager." Painful.

3. columns that act as pseudo-primary keys vary by table, even if it's the same data. For example, every car has a unique license plate. That's the primary key for Table1. Wait, every car also has a VIN. That's the primary key for Table 2. Now join them together. Exactly.

4. the pseudo-primary keys aren't reliable. The column in Table1 that holds license plates has VINs for some rows. The join suggested in #3 to Table2 actually works -- sometimes. When? Hard to say.

5. cross-keys aren't reliable. To solve the problem, VIN is often added to Table1 or license to Table2. If you're really "lucky", as I am, you'll have both in both. Unfortunately, sometimes the two keys are out of sync, as predicted in #1: a car gets in Table1 with the one license and VIN, the VIN matches to a VIN in Table2, but that row in Table2 has a difference license.

6. multiple naming conventions. Can't remember if the column is spelled "licenseNum", "LicenseNum", "license_num" or other variations with "number", "no", etc.? That's because each table has a different variation. Often, a table will be consistent with itself (though often it won't be), but different programmers add new tables and bring their own convention without concern for fitting in.

7. just lots and lots of bad or missing data. Data entry just not happening reliably.

What do you do about it? It depends on how much time you have.

If it's a long project, you painstakingly fix the data in some key tables, make sure the input processes are clean. Then you build out from there, either refactoring tables or fixing data at each point along the way. All the while, you try to fit in other business requests as you can.

Sometimes, the business can't wait, and the requests pile up faster than you can take them off. Your only hope is to hire some good database developers and work like crazy. Keep at least one or two of them away from the firing lines, so they can fix things in the background.

Worst is if it's a short-term project. You might think that, being short-term, that is the easiest. Ride it out until it winds down, right? If you've been an app developer for any amount of time, you know that apps very rarely die. If the app works, someone will find a use for it. But, because it's considered short-term, no funds will be invested. It will go on living like a zombie. The people who keep these things going are real heroes.

I had forgotten what data hell felt like

Now I remember, thanks to the new databases I have inherited!

That is all.

Bonuses bonus

I was done talking about bonuses, but the subject refuses to go away.

I'm not outraged by AIG's bonuses mainly because Merrill's bonuses outraged me far more. (Apologies to my friends at Merrill.) But, getting them back wouldn't bother me either.

One person on my trading floor said, upon reading that a bill would be passed to tax the bonuses at 90%, essentially clawing them back, "It's the end of capitalism. Some of these guys work 100 hours a week."

These guys do work hard, but I hardly think capitalism is at stake. I once worked over a 100 hours a week, including shifts that exceeded 30 consecutive hours, over a summer for an average of $7.50 an hour. If anything, capitalism will be healthier if we have a smaller range of salaries.

While many of the people who will be giving their bonuses back did not cause the problems, their bonus guarantees were probably born of continued expectations of fat profits through financial derivatives.

Not causing the problem is not really the issue, though, is it? What if developers started a job and refused to fix bugs because "I had nothing to do with the creation of those problems"?

More importantly, at the end of the day, everyone should be expected to work hard. While I will say, having worked with both bankers and software developers, that the former work much longer hours on average, a good percentage of highly-skilled developers work 60 hours as week (and, of course, some work more). They might earn between 100 and 150k. Does working another 50% more hours justify 1000 to 2000% the salary?

Bursting any bubble hurts. Right now we're witnessing not just the burst of the housing and financial markets bubble, but also financial industry salaries. Ouch.

Simple things to help keep your job

These days, lots of people worry about losing their job.

You might feel more comfortable if you knew all the source code intimately, were a technical genius in every architecture and language, and knew as much about the business that your app is used in as the people who use your app.

All good things to work on, but this post is about things you can do that take almost no time, effort, or money.

1. Show up on time. Most dev shops have fuzzy start times and fuzzy end times. Most developers start late and work late. Yet, consistently showing up on time signals that you are reliable. It helps others include you in important conversations or events. Few of us are morning people, but this is a small price to pay.

2. Call ahead when you're going to be late. I once got a call from someone at 11 a.m. saying they were going to be late and they didn't want me to wonder if they were coming in. Uh, I was already wondering. Again, the primary issue here is that you want to give the impression of being reliable. In addition, there's no excuse to call when you're already half way to work; call when you get up and it's already obvious you'll be late.

3. Attend group social events. If the team is getting a beer after work, you could get a soda if you don't drink. You don't have to spend a lot of money, just buy one drink and nurse it. This is actually a great way to strengthen your work relationships. Your interactions become more natural, and that can help you when, for example, you need to bother someone to get help sorting out your java threading issues.

4. Volunteer for internal groups. If your department needs a social planning committee, or the company is looking for members to join a diversity group, sign up. This will take a few hours of your time, which you shouldn't let interfere with your work, but you'll be making contacts outside your team. You can also try out some new responsibilities, like project management or dealing with vendors.

5. Give a presentation. What have you been working on? Offer to give a training. It will seem as if you are doing the group a favor; meanwhile, you're actually practicing your public speaking skills, sharpening your thought organization skills, testing yourself on how well you know your work, creating documentation, and interacting with people you normally might not or at least in different ways.

6. Be courteous and treat people with respect.

7. Tell your manager as soon as you think you might take a day off. Someone told me once at 5 pm on Thursday that, "oh by the way, I booked myself on a weekend getaway, so I'm taking tomorrow [Friday] off". Again, perception of reliability is being hurt, and you might be causing your manager real scheduling pain that could easily have been avoided. Maybe you're afraid that you'll change your plans? I'm sure you're boss won't mind if you decide to work on that after all.

In a sense, these things are free, always have been, and yet it surprises me how often people don't do them. Maybe the current job climate will generate a few converts.

One way to know you are good

Everyone thinks they're exceptional. After all, 70% consider themselves above average.

You can test how good you are by taking a job where you are expected to be a senior person: project or team lead, architect, technical expert. The newer the better: new industry, new programming language, new tools, new responsibilities.

Now, how long does it take for you to make a contribution to the team, to go from being a net receiver of information to a contributor, or, said another way, becoming someone who answers more questions than asks? How long does it take to generate unique insights about the application or the team's problems? How long until you become the go-to person in certain areas? How long before things emerge that only you can do?

If you have truly exceptional skills, the answers to all these questions will be "very quickly" or thereabouts.

Sometimes, you find out that what made you strong was just having been in your previous position for a long time or you've just been doing it longer, not that you were particularly gifted. Nothing wrong with that -- that's highly valued by any manager.

But keep that in mind when new people come, and they need your help or they don't seem to be as quick as you are.

"Forced entrepreneurship"

I think of software teams as entrepreneurs. Not just because so many software companies are start-ups trying to find a niche. Even at big companies like where I work now, the entrepreneurial spirit drives the success stories.

Read this NY Times article about laid-off workers putting their creativity to do new things. The subject of entrepreneurialism fascinates me, because the good IT departments at my company, as well as the good small companies where I have worked, all have that same spirit.

At one start-up I worked at back in the 90s, we had a huge client-server installed base. Eventually, Java and web service type applications showed too much benefit, and we jumped with two feet into that space. What made that a great time was not the relief that I could strengthen my resume, but being surrounded by individuals who eagerly threw away everything they knew, where we were experts, for the chance to try some new things, where we were rookies.

At larger companies, organizational bureaucracy may crush the entrepreneurial spirit. But, I also find that a large number of the people who come to larger companies no longer have a hunger to learn, or even a curiosity about what is going on around them.

The best IT departments in our company attack problems like something personal is at stake, as if they were laid off, and it's now up to them to find solutions, to try new things. This can be difficult -- there is a lot of organizational bureaucracy of course -- but that should be the aim of every team and department. If yours is lacking that, then figuring out how to change it becomes the top priority.

Gaming search engines for 15k a month

This article gives the inside scoop on how to manipulate search engines and blogging to generate lots of traffic and earn lots of money via AdSense. It seems one step removed from a scam.

I say "seems" because on closer read, I conclude that it is a completely legitimate business practice. In fact, it may be the future of newspapers, or, rather, what will replace them.

The author's basic strategy is this:

Your goal is to build a site which contains everything someone would ever want to know about your root keyword.... Now you have some original high quality meat for your microsite which is truly original content and is highly targeted for each long tail keyphrase.

Later, he markets the site by connecting to other authors and experts on the same topic, and creating a self-referencing blog.

Ultimately, this is exactly how people have always built businesses. They create a product -- say, "Rosetta Stone" language software -- then run infomercials about it. Or, they investigate a subject, write a book, and tour the country promoting it. (Malcolm Gladwell comes to mind.) The only difference is that this replaces monetary cost with technical know-how.

It is the kind of job that only technically savvy people -- do-it-yourself, non-corporate programmer types -- can do today. Tomorrow, it will go mainstream. But today, there are a growing number of these kind of jobs available to the right individuals.

Meanwhile, he has done a tremendous favor to the people who were interested in a somewhat obscure subject, by sorting and centralizing an ocean of information and creating new, relevant content. That's not gaming the system, that's providing a highly-focused news service.

Welcome to the golden age of information!

(Thanks to Jose for the pointer.)

The volunteers that work for you

What if you viewed the people that worked for you as volunteers?

Volunteers, of course, don't owe you anything. Treat them like crap, and they'll disappear.

How would your management techniques change? How would you motivate them? How would you get the bad, boring jobs done?

It's not impossible. I've worked at several organizations where the volunteers were willing to do mundane, thankless, even painful jobs at great personal effort. Sometimes, I was that volunteer.

It's an important question, because in a very real way -- more than ever, and even moreso in the future -- that's what they are: volunteers. They can leave, especially the ones you need the most. And "the cause" is more near and dear to their hearts than any a non-profit could offer: a job that interests and challenges them (while offering financial rewards, of course).

Who's fault is it?

Ever had a good idea? Maybe something simple, like changing the color palette of the app to shades of blue. You brought it to the team, they mostly ignored you.

Later, it turns out, you were right. The users hated the existing color palette. Responses indicate blue would have been best.

Who's fault is it?

In an ideal world, we wouldn't ask this question. Instead, we would move on to "what do we do now that we're in the situation we're in?"

But, usually the person asking this question is the person who suggested the blue colors. "Those idiots should have listened to me. It's all their fault the app looks hideous. I told them to change it to blue."

How hard did you push to change it to blue, though? Mentioned it once?

Often, you have to push an idea several times before people stop and hear you. More importantly, you have to be able to express why blue would look better or appeal to customers. If you can't do these things, then people would have to follow your idea slavishly -- hardly a good idea. If you didn't do these things, then it wasn't really an idea that you were willing to bet on.

A parallel would be the guy who says he knew a month ago Citi shares were going to drop to $1. How many shares did he short? Zero. People would pay millions for accurate advice like that. He could have quit his job, instead of working for peanuts. But he knew.

If you have an idea, you need to put your money down, put your reputation on the line. Otherwise, it's just words.

Many successful projects of all sizes -- Google Reader, Facebook, open source, to name some software examples -- faced strong opposition. They got done only because someone believed enough in it to put their reputation on the line, and communicated the brilliance of the idea convincingly.

Motivation and complaints

The other day, I met a woman who was full of complaints. Lots of people didn't submit their data correctly, and that created work for her. The building maintenance crew made some lovely improvements, but far too noisily during work hours, which distracted her. And so on.

All valid complaints actually, but I got worried because we were on the same project team. Was she going to drag her feet the whole time for one reason or another?

I kept my concerns to myself and plunged in. And came away surprised and delighted at how productive and helpful she was.

I came to realize that she just liked to have everything lined up before beginning. Work environment, data, clear sense of objective -- once these were in place, she could move mountains. The main point of her complaining was that it prevented her from getting to it and doing an awesome job. She was totally self-motivated, but in the wrong environment, her review might have contained phrases like "reluctant to jump in" or "a negative distraction".

Far from dragging her feet, she actually wanted to race, and cursed the clutter that blocked the starting line.

What's the value of 10 seconds?

I love this creative use of software.

As you must know, some sites require you to translate a distorted word and type it in when leaving comments, to filter out spambots. The guy who helped create that system "realized that he had unwittingly created a system that was frittering away, in ten-second increments, millions of hours of a most precious resource: human brain cycles."

It could be defended as a small cost and necessary evil, but he decided that surely all that power could be harnessed for something useful. So he reinvented it.

If you've visited those sites [e.g., Facebook], your squiggly-letter-reading ability has been harnessed for a massive project that aims to scan and make freely available every out-of-copyright book in the world, by deciphering words from old texts that have stumped scanning software.

For example, through this effort, every issue of the New York Times, back to 1851, will be digitized by the end of the year! Which just goes to show, the sum of those small costs was very large, indeed.

This reminded me of another fascinating application that I learned about from a former professor of mine, the excellent Panos Ipeirotis. In his class, he mentioned the esp game, a very clever way to get real people to label images for free. Google licenses the game to improve searches on its millions of images.

As with fuzzy words, people interpret pictures better than computers. Both of these applications struck me as being like manual grid super-computing networks.

No wonder, then, that I later came to learn that both applications were created by the same guy, Luis von Ahn. I also came to learn, through my professor's blog, that von Ahn has recently started a blog. Full of interesting ideas, no doubt. We are in the golden age of information!

Maybe it's the game that's the problem

Watching the season premiere of Celebrity Apprentice last Sunday, one of the contestants made a comment that shocked me. (I watched the kickoff last year too, and that was the only episode I saw.)

The comment was that Dennis Rodman should be fired from the team because Rodman didn't have a good work ethic. What?

We're talking about one of the greatest rebounders and defenders in the history of basketball. He won two Defensive Players of the Year awards. He once had 34 rebounds in a single game. His contracts often contained memberships to 24-hour weight-lifting gyms, because he worked out so much.

Clearly, he does not suffer from a work ethic problem.

But, it is also true that Rodman contributed very little to his team on Celebrity Apprentice. He required heavy coaxing for every small effort.

He stated that it was never clear to him what everyone's role was, including his own. He could have been just rationalizing his bad behavior, but there is still some truth there. Rodman's the kind of guy who needs to know his role, who likes to focus. In his basketball career, he didn't look to score very often.

The primary challenge of a software development manager is to motivate the team. People aren't successful, and often the reason is, "He wasn't motivated" or "She wasn't a hard worker." This is particularly relevant to software development because most companies want only self-motivated coders, due to the solitary nature of so much of the work. "Here's the source code, here's a problem description, go to it." Frequently, there's no training. If you're the right person, you'll figure it out.

Yet, how many Dennis Rodman's get the boot as a result? No doubt, motivating someone like Rodman is a huge challenge. (Fortunately, he would surely hate coding.) On the wrong team or with the wrong coach, Rodman was only a disruption, a "bad apple." Instead, he was a major factor in 5 championship teams.

Some people really do have a poor work ethic. A software manager has to distinguish the ones who aren't motivated from the ones aren't getting the right motivation.

So let me change that sentence. The primary challenge of a software development manager is to figure out what motivates each team member. Everything else flows from there.

10 papers every programmer should read

Michael Feathers says everyone should read these at least twice. Pointers like this -- filtering the many papers he's read over the years down to these ten -- are invaluable.

I found all of them online for free, except for #8, so I added links for them.

1. On the criteria to be used in decomposing systems into modules - David Parnas html pdf

2. A Note On Distributed Computing - Jim Waldo, Geoff Wyant, Ann Wollrath, Sam Kendall pdf

3. The Next 700 Programming Languages - P. J. Landin pdf

4. Can Programming Be Liberated from the von Neumann Style? - John Backus pdf

5. Reflections on Trusting Trust - Ken Thompson html pdf

6. Lisp: Good News, Bad News, How to Win Big - Richard Gabriel html pdf

7. An experimental evaluation of the assumption of independence in multiversion programming - John Knight and Nancy Leveson pdf

8. Arguments and Results - James Noble

9. A Laboratory For Teaching Object-Oriented Thinking - Kent Beck, Ward Cunningham html pdf

10. Programming as an Experience: the inspiration for Self - David Ungar, Randall B. Smith pdf

What's the Google secret?

A final comment on compensation, this one related to options. (Then, like the bonus cycle, we'll be done with this subject... until next year. That's the way the financial industry works -- for now.)

Google gets a lot of press about how great it is to work there and how people are lined up for interviews, like the lines for American Idol auditions. Stories about their 20% free time policy, creative culture, self-organization methods, and of course free food sound very appealing. I admire them for their innovative products and efforts to balance business with "not doing evil". And yes, some of my best friends are Googlers.

Recently, Google decided to reprice options that it had given out to employees. Due to the stock market cliff-dive, the options became worthless.

This reminded me of another tech company that was once considered the greatest in the world, hiring only the smartest of the smartest people. They minted many millionaires through stock options. People dreamed about working there.

The company? Microsoft, which many would say epitomizes everything that Google is against. It wasn't that long ago that Microsoft was the beautiful company.

Which led me to wonder: is the warm glow coming from Google really just a function of their (previously) red-hot stock price? Will Google be perceived as just another company now that the options payoff has disappeared, and their pay matches most other companies (i.e., now that even their masseuse can't become a millionaire).

Obviously, this is a simplistic view. Google represents the cloud revolution against the desktop, just as Microsoft represented the desktop revolution against big hardware. That's also important.

But what I'm interested in is whether Google is actually a fundamentally different type of company, or they just have a good business at the present time. I think if you asked Google management which they would rather have, they'd choose the former. (Of course, everyone would want both.)

The typical justification for options is to align both the long-term benefits of the employee and the employer, via the stock price of the company. At the point that the company reprices options, the options are no longer acting as an alignment tool, though. The employee benefited despite how the company fared. The company's options program is exposed as just a form of deferred compensation.

There's nothing inherently wrong with that. Any manager would be foolish not to exploit a cheap way to fund super salaries for great people, as long as the shareholders don't mind.

Going forward, it will be interesting to see whether Google's creative organization continues to be heralded as their secret weapon, or whether their exalted image will follow their stock price.

Seth's blog

Seth Godin doesn't talk at all about software development in his blog, so I was surprised it at #3 on Jurgen Appelo's list of top 100 blogs for dev managers. Presumably most of those hits are coming from marketers, not software managers.

Yet, his is one of the first blogs I read everyday, and much of what he says directly relates to having a software career based on excellence and demonstrating importance through daily effort. As he marks his 3,000th blog post, he comments:

The hard part, as you can guess, is the first 2,500 posts. After that, momentum really starts to build.

He started back in 2002, so that works out to aboue 415 posts a year. Yup, he's definitely a guy who believes in daily effort.

Here's a fairly recent post -- a typical example of why I read him. It's aimed at marketers, but highly relevant to anyone building their IT career.

As you consider marketing yourself for your next gig, consider the difference between process and content.... As the world changes ever faster, as industries shrink and others grow, process ability is priceless. Figure out which sort of process you're world-class at and get even better at it. Then, learn the domain...

In IT, it's easy to get obsessed about content. Our resumes overflow with acronyms and languages, versions 3.7 through 8.13. As we work longer, we might pick up some knowledge about the industry we're in, such as financial services, so that goes in the resume too. It's all stuff we "know".

Meanwhile, what's really valuable is the process. Capturing that in your resume or identifying which job applicants understand the process is the challenge and the goal. How do you identify what is important to your user? How do you effectively test your code?

Practices such as Agile help draw focus to the process, but sometimes that just turns into another list of domain knowledge. What's a product backlog? How many weeks do your sprints last? This is all content.

The more important questions lie underneath these details. What's the purpose of a product backlog? Name some other ways to do the same thing, and why are they better or worse? That's process.

I highly recommend Seth's blog for anyone interested in thinking about how to do their job excellently.

There's no f in bonus

Bonus season winds down here in the financial industry, and we're left to consider what it all means. The bonus pool is particularly sensitive on Wall Street because financial firms rely on it excessively, outweighing regular pay for some employees. Many workers in the industry and in my company took a zero bonus.

What is the purpose of a bonus? Reward for individual productivity? Incentive to stay? Alignment of interests between employee and employer? Profit-sharing?

If your answer is all of the above, then I think there's no chance the bonus system at your company is working. It's just another form of compensation, so there's almost no reason to have it; just give a regular salary.

The general public is fuming about Wall Street firms' bonuses this year, in light of having their received taxpayer assistance. The public views these bonuses as a type of profit-sharing; thus, no profit, no sharing.

Individual employees view these as productivity rewards. Thus, the ones who worked very hard and had measurable positive results this year are upset about low or no bonuses.

Middle management typically views it as a key employee retention tool. Thus, even though Mary Developer didn't have the opportunity to produce like in previous years (maybe a couple of her projects were canceled), and Joe Programmer out-produced Mary this year, rarely would a manager consider switching their bonuses if Mary is viewed as the foundation of the group.

I would guess that senior management most commonly views these as a way to align employee and employer interests. Unfortunately, the payouts encourage the employee's short-term interests, which may be at odds with the company's long-term interests.

Underlying all these interpretations: a bonus that doesn't grow or, worse, shrinks feels like a punishment, even if the bonus is advertised as variable or tied to profits.

I think the biggest problem is that most companies haven't really thought specifically about what the bonus should accomplish. Thus, it tries to do all of the above by default, only because everyone interprets it however they want.

One of the most important decisions a company's executives makes is how people will get paid. If they want use bonus payments effectively, they need to be crystal clear about their purpose, and drive that down through the company.

On Wall Street, they would likely have revamped the entire bonus structure, probably eliminated it. Unfortunately, the issue has now spun out of control, and the revamping will be imposed.

The end of H-1B?

Thomas Friedman notes:

the U.S. Senate unfortunately voted on Feb. 6 to restrict banks and other financial institutions that receive taxpayer bailout money from hiring high-skilled immigrants on temporary work permits known as H-1B visas.

Not only will "buy American" sentiment continue to grow, but the original purpose of the H-1B program -- to fill the shortfall in technology workers -- begins to look silly as the numbers of laid-off tech workers increase. I personally know several good technologists searching for work. If this downturn continues for any amount of time -- and it appears that it will -- then that shortfall will surely disappear.

At the same time, like everyone else in the industry, I have worked with plenty of H-1Bs, and have become good friends with many of them, so I wouldn't be in favor of shipping people out.

Ironically, it's probably in the best interest of countries who send their H-1B workers to the U.S. to keep them at home. But, what's good for the team (country) in the long-term may not be what's good for the individual in the short-term.

Currently, the U.S. acts like the New York Yankees of the world. The Yankees hire the most expensive and best players, tilting the odds of a championship in their favor. Smaller teams constantly have to re-tool with young, unproven players. Similarly, the best and brightest come here for better pay and working conditions, tilting the odds in favor of exciting new innovations being created in the U.S. They help start new enterprises that make their solutions available first to U.S. companes. Their countries of origin have to constantly train new talent to stock their own companies.

In general, I am in favor of more immigration. Economies with greater immigration prove to be stronger than those with less. Japan has suffered economically since the 90s in large part because of a lack of immigration, combined with a very low reproduction rate.

I'm not sure that targeting immigration to a specific category of worker is the best way to go about it, but it may be the only way to achieve the desire ends politically.

Thus, let me say: long live the H-1B!

Remind me I said this if I get laid off. But, you have to believe in your own value, that quality wins out in the end. As a result, I have confidence my friends, currently out of work, will find something soon, being smart and highly capable guys.

Difficult times put your beliefs to the test. That's when you find out what you truly believe.

Path to excellence

How can you excel in your job? Go deep and go wide (but not at the same time).

Going deep means becoming an expert. This can simply be a manifestation of doing something very well: learning about what you're doing, instead of just trying to get it off your plate as fast as possible.

One person who worked for me coded up some trade feeds for regulatory purposes. As she went along, she spent the time to learn a lot of fine-grained details about the regulations and what that meant for our system. Eventually, every meeting involving coding changes (because the regulations regularly change or have exceptions) required her presence.

Another guy wrote a feed to send some data to a central database for management reporting. The feed required some calculation pre-processing, but otherwise, it was pretty straightforward. In doing his work, he came to fully understand the nuances of the calculation on our side and how any changes would affect the aggregation. Eventually, he learned so much about this overall business process that he stopped working for me and started working for the business side.

By going deep, you become the go-to person on a subject. You can go deep on a technical language or a complex application module. You become the state of the art for that art. You demonstrate high competence in a public, understood way.

Going wide means picking up new skills or knowledge. The key to going wide is that the value is unlocked only when you combine the new with the old.

My friends who worked on the trade and data feeds combined their programming skills with a business process they previously knew nothing about. They did an excellent job of combining the two.

What you can't do is learn something totally unrelated to what you are doing now. Eventually, this could pay off for you, but it could be a long wait, and it won't help you excel today. You are a newbie in the new field; your only advantage is your accumulated unique experience that you bring with you.

Here's a fascinating example of a guy who repeatedly goes deep, then jumps to something new but related, then goes deep in that. He started with industrial engineering, becomes an expert in gun manufacturing ("I probably know as much, or more, as any one single person about manufacturing guns," he claims, credibly), moves into robotics. Then,

A few months ago, Baber decided to design his own mini-helicopter, one light enough to disassemble and carry in a backpack. After a successful test flight in January, he is currently busy building [modifications].

Yes, these are all related fields for him. This example shows that, if you truly master this process, you actually create your own "industry". Here's the summary (and the full article, requires free registration).

What's in your box score?

I'm not a big fan of the NBA, but an article about NBA player Shane Battier is loaded with insights about teams.

It was, and is, far easier to spot what Battier doesn’t do than what he does. His conventional statistics are unremarkable: he doesn’t score many points, snag many rebounds, block many shots, steal many balls or dish out many assists... "He can’t dribble, he’s slow and hasn’t got much body control."

Battier’s game is a weird combination of obvious weaknesses and nearly invisible strengths.... When he is on the court, his teammates get better, often a lot better, and his opponents get worse — often a lot worse. He may not grab huge numbers of rebounds, but he has an uncanny ability to improve his teammates’ rebounding.... At the same time he somehow improves the defensive efficiency of his teammates — probably, Morey surmises, by helping them out in all sorts of subtle ways. "I call him Lego," Morey says. "When he’s on the court, all the pieces start to fit together."

Weak stats on both offense and defense, yet teams he plays for become winners.

An old adage claims that you can't manage what you don't measure. Seems true enough. Is the quality of your software improving? Much easier to say if you have some statistics on number of bugs, severity, etc.

But are you measuring the right things?

For most of its history basketball has measured not so much what is important as what is easy to measure — points, rebounds, assists, steals, blocked shots — and these measurements have warped perceptions of the game. ("Someone created the box score," Morey says, "and he should be shot.")

The ongoing financial meltdown contains numerous, unfortunate examples of what happens when the wrong behaviors are rewarded based on measuring the wrong things. For one, bankers bonuses were tied to the size of their deals, regardless of quality and risk.

It's only recently that the NBA started to figure out how to measure the value of team play. At "regular" companies, the use of 360 reviews attempts to get at something similar. Feedback loops from multiple sources on multiple aspects of a person's job can also be used.

Does your team / manager / company -- do you -- encourage behavior that makes the team stronger? Or are you breeding players just padding their own stats?

The article goes into a lot of other equally relevant and interesting subjects. Read the whole thing.

Promotions

Annual promotions were announced recently at my company.

Occasionally, someone works for me who actively disdains getting promoted. "Not my objective," is what they say. I applaud the attitude, but I do sometimes wonder about the underlying assumptions.

While it is accurate to say that it is not a primary objective, it should be an objective because getting feedback should be an objective. A promotion is one of the biggest feedback mechanisms. I'm a big fan of receiving feedback.

I think these employees feel the promotion process is corrupt and inaccurate, and therefore not an accurate reflection of what they do. Another version of this complaint: "only those who are trying to get promoted actually do," the implication being that more kissing up to the boss is required. Surely that happens in some cases, but the vast majority of promotions that I participated in or had detailed information on were openly debated and resulted in the right people being chosen.

But, even if you accept the premise -- that the promotion process is useless as a feedback mechanism -- asking, "What are the specific things I would have to accomplish to be promoted?" is a very good way of finding out how to succeed. This creates a conversation about what is considered excellent work and what isn't.

At the end of the year, if you didn't get promoted, but you hit all the achievements, you'll know you did a great job, which is the primary objective. Then again, maybe you will get promoted, and you might find you like it.

Valentine's Day

Valentine's Day is a corporate holiday manufactured to boost sales during a slow time. It's a totally transparent marketing manipulation scheme catered to women.

That doesn't prevent millions of women from loving it all the same. After all, everyone likes to be appreciated.

When do you take time to tell your team how much you appreciate them? This is a serious matter, and something that doesn't happen enough. Studies show positive reinforcement is more effective and longer-lasting than negative.

The annual review doesn't count, though it should happen there as well. (Positive comments should outnumber the negative ones.)

Maybe you're thinking, "We have an event just for this already: our annual holiday party." But, that's a gift of the company, not from you, and if your company parties are anything like mine, what typically happens is people use the party to interact and bond with people outside their group. That's actually a great use of the party, and it also means you should have something separate.

How about an annual appreciation lunch with each team member? Or a party that is planned ahead and advertised specifically as a "team appreciation party"?

Sound contrived? Surely it is, but that won't stop your team from liking it and feeling appreciated all the same.

(Just don't do it on Valentine's Day.)

New rules: no more rules

This escalator sign doesn't really make sense. But at least they have the excuse of English not being their native language. Presumably, the Chinese makes perfect sense.

On the subway I take, they have no such excuse.

The tunnel where I get out is very deep. A very long escalator takes you from the subway to the top, where the ticket booth and street exits are. The escalator only goes up; to go down, you have to take the stairs or the elevator. Once you arrive at the top of the escalator, an LED sign greets you with this message:

"Escalator is for subway patrons only."*

What? Only people coming from the subway are on the escalator. The only way to be in violation would be to buy a subway ticket at this station, walk down the stairs, and then take the escalator back up. Just for fun. Even then, you paid for a ticket, so shouldn't you be allowed a free escalator ride?

Why is this sign there? The only reason I can think of is that, in the design stage, someone put in this "feature" thinking abstractly that it could be of benefit. Later, when it came time to actually use the sign, the only thing anyone could think of was mindless manager-ese -- something they read somewhere else.

"Restrooms are for customers only" on a restroom at the back of the store is another example. Whoever posted that sign probably saw a similar sign somewhere else, but failed to note that they probably saw it at the front door.

In software development, we should be following principles and practices, not rules. Principles are almost philosophical; it's what you believe.

Practices are not rules. Practices derive from principles. This is where confusion creeps in. Is "Make your classes final, do not allow extensions" a rule or a practice? How about "All classes should have a unit test"?

The key is whether you're being motivated by principle. If you understand the principle, and that's why you're following the rule, you're really following a practice. If you don't understand the principle, or you're following the practice regardless of it, you're actually following a rule.

I see now that Robert Martin also has a related post yesterday, specifically about SOLID principles. Scott Bain also has an good write-up on this in Emergent Design.

* I would take a picture, but that is verboten in this high security era.

Testing smackdown, part 4

Not to keep cross-posting Jeff Atwood (a sign of respect, anyway), but he responds to Robert Martin's criticisms directly.

At what point do you stop having a set of basic, reasonable programming
guidelines -- and start being a Ferengi programmer, an imperfect manifestation
of the ruleset?

This has expanded beyond testing and into development principles, but TDD is still a good straw man for the conflict.

Note that Jeff's argument here is not the same one I was making before about why TDD or another development principles might be pushed aside sometimes (basically: real-world constraints get in the way).

In fact, even though Jeff and I seem to reach the same conclusion, I disagree with Jeff here. This might not be his only reason for discounting the importance of TDD, but at least this particular argument seems weak.

I'm not sure you could follow Robert's principles "mindlessly", so I don't see any problem with putting them out as design goals.

Several years ago, when I started coded heavily in Java, I read that you should only code to interfaces. It also said that behavior used by multiple implementations should be kept in abstract classes; in anticipation of that, it recommended extending abstract classes as a rule, so that specific behavior could be easily moved up if you wanted to generalize. And, actually, those rules make a lot of sense to me.

So, on my next assignment, I did exactly that. Every single class extended an abstract class. Every instantiation was to an interface. To some degree, mindlessly.

But, to some degree, thinking was always required. Do I need this method in the interface? Which part of the behavior goes in the abstract class and which in the implementation?

The real downside was the massive number of files. More files had to be maintained. Creating a class took more time, because I had to create 3 files every time I wanted one. Once that was done, however, the burden wasn't too great, and I did see some benefits later when I went back for modifications and extensions.

On subsequent projects, I created interfaces and abstract classes more selectively. This is closer to the agile idea of limiting complexity to what you know you will need.

With this practice, or pick a principle, say the Single Responsibility Principle, it's much easier to ignore it and just plow forward with the code.

I haven't yet run into a case where someone was absurdly over-applying solid development principles or practices; the incentives run the other way.

Coding epistemology

Jeff Atwood has a nice post about knowledge -- specifically, one way to learn something well.

I consider there to be the three levels of knowledge. They reflect my belief that knowledge needs to be shared to get the most value out of it.

1. knowledge you know. You read something, someone tells you, or you do it yourself, to the point where now you know it. But how do you know you know it?

2. knowledge you can explain to someone else. This requires you to organize the information. Many times, in the actual moment of explaining, you will suddenly realize a gap in your knowledge or a fallacy in your logic. I think this is akin to getting a call from your mom that she's coming over, and realizing only at that moment that your apartment is a mess; you're viewing yourself through someone else's eyes. And yes, the act has to involve explaining and a someone else, not just a mental exercise where you convince yourself that it could be done.

3. knowledge you can answer questions with. People react to your explanations in unpredictable ways; weird questions pop out. They try to apply it to things you'd never think of on your own, just as end users do things with your application that you'd never considered. These questions test the robustness and flexibility of your knowledge -- or, said another way, they test whether your understanding is at the implementation or the abstraction level.

You are more likely to succeed at numbers 2 and 3 if, as in Jeff's example, you wrote something yourself from scratch, as opposed to someone just explaining it to you once. Someone explains something to us, we can walk away with a false sense of security in our knowledge, which easily crumbles in the face of simple questions.

Implicit in number 3 is the possibility that someone's question may blow up your knowledge, perhaps due to false or missing assumptions.

For these reasons, I like to have a "training" on any major new functionality, either as a group, or one-to-one. To verify that the trainee (which could be me) is truly walking away with knowledge, that person will later have to explain it to the next person. And take questions.

Good person, wrong role?

A friend of mine recently lamented to me about the difficulties of hiring. "These guys come with beautiful resumes and rock solid recommendations, and then they come and are so disappointing." I'm sure anyone involved in hiring knows exactly what my friend is talking about.
To some degree, just chalk it up to the intrinstic difficulties of hiring. Even legendarily thorough and expensive processes employed at places like Google and Goldman, Sachs let in a fair share of low performers.

However, a book I read lately brought another cause to mind. David Sedaris's Holidays on Ice is, like all of his books, a collection of funny short stories. The stories are always personal vignettes involving his family or some trip he took. His new books always get to the top levels of the sales charts. I have read several, and they never fail to make me laugh.

Holidays on Ice is unique in that it also contains a few purely fictional stories.

His fictional stories stunk. I can't think of any real clunkers from among the dozens of his personal stories spanning multiple books, but I found every one of the fiction stories to be unamusing and uninteresting.

If you were evaluating his writing, and happened to read only his fiction, you'd walk away thinking, "Everyone talks so highly of this guy, but he can't write!"

I think this points to a simple truth: people can be hugely talented and accomplished, but it still requires getting them into the right role. The right role is not a title. The Vice-President role under John McCain would have been vastly different than it will be under Barack Obama. Same title, different role.

Getting someone into the right role can be very difficult. Sometimes there's a mismatch with the company culture. Some people are highly accomplished but not good team players. Or, vice versa, they are solid team players, but that makes them ineffective on a team of stars. A personal narrative writer assigned to fiction.

A good manager should be evaluating and searching for the right roles and responsibilities for each team member. This is the first step in empowering and motivating your employees. It is also never-ending. If that sounds like a lot of work, it is, which is why a rigorous hiring process can pay for itself.

But hiring will only get you so far, and then it's time to manage.