You get what you measure

Yesterday I watched Grand Designs on TV where a couple restored an old 1600-century house. They found that at least one window was shut with bricks and concrete (or the equivalent used at that time), but when they opened it up the whole room changed and the light and the view the window provided was beautiful.

The reason for shutting the window, as Kevin McCloud told, was that long ago there was a window-tax (!!) in England, hence a lot of windows where shut this way.

You get what you measure.

If you find an old Swedish mirror of some size, you will see that the mirror glass is often split in two or more pieces. When I first saw this I assumed it was because they didn’t know how to make larger pieces of mirror glass.

But, the real reason for this was, as in the case with the English windows, there once was a mirror tax in Sweden for mirrors larger than a certain size, hence mirror glass were cut in smaller pieces to avoid the tax.

You get what you measure.

In the average company, the same behavior is often seen where managers try to control their organizations by detailed measuring. In software development it is common to measure things like lines of code produced, number of people in a project, or produced documentation. This usually renders in… many lines of code, lots of people in the projects and piles of documentation. Wasteful and without customer value.

You get what you measure.

So, what should you measure?

In lean organizations, and promoted by lean software development, measure up is an important principle to avoid local optimizations. Local optimizations are bad because they consume resources that could be better used at other places, and they could create bottlenecks and lock-ups and it is likely to result in dysfunctional organizations.

Running Tested Feature (RTF) is one way of measuring up in a software developing organization. It lies above all detailed code metrics and it measures the whole development effort. The RTF index should be constantly increasing, otherwise there is a problem.

Net Promoter Score (NPS) is another way to measure up. Ask the user of a product “On a scale from 0-10, how much would you recommend this product to your friends and family?”. Promoters and detractors are detected this way. Promoters are also likely to be a good source of information for how to improve the product since they are more likely to have used the product extensively.

If you think RTF or net promoters is a good thing, start measuring those things instead of detailed, micro-managing metrics that will result in bad behavior and local optimizations.

You get what you measure. In a good way this time.

There are of course other ways of measuring up in a healthy way. If you have come across someting that you find interesting and useful, please make a comment.

5 Comments (+add yours?)

  1. johlrogge
    Feb 15, 2009 @ 16:57:46

    When traveling in Cairo i noticed that most houses in newly built parts had reinforcement rods sticking out of the roofs leaving them looking rather sloppy and “unfinished”. I asked our guide about this and there was indeed a reason for this: you don’t pay taxes for houses while they are being built. So by never finishing the house you can save a lot of money per year. You get what…

    You had some good suggestions for what to measure. I’m beginning to think though that no matter what you measure you will always fail unless there is a general wish to add value. Any measurements should be seen as indicative of wether you are on the right track, but keep your eyes on the road, not on the metrics.

    Reply

  2. danielbrolund
    Feb 15, 2009 @ 19:01:15

    That is true. With bad intentions no metric will work.

    I would say good metrics help good intentions.

    The people in old England and Sweden shutting windows and splitting mirrors, all they wanted was to comply with the laws of their countries while saving some money.

    Reply

  3. Johan
    Feb 19, 2009 @ 10:28:38

    Excellent meme – “You get what you measure”. So true in most cases…

    Reply

  4. Karolina Kuczko (@qczko)
    Aug 29, 2011 @ 21:12:52

    Nice article!

    Did you know that in Poland we had ‘chimney tax’ (pl. poddymne)? You had to pay for every chimney at home… and that’s how Polish folks invented houses without chimneys (with ‘inteligent’ tile)๐Ÿ˜‰.

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: