I remember reading project management literature in the early ’90s, and really enjoying the works of Tom Gilb. The thing that sticks out the most in my mind is Gilb’s Law:

Anything you need to quantify can be measured in some way that is superior to not measuring it at all. Programmers are fond of saying “you can’t really measure that,” and Gilb’s law says that’s just a cop-out—it might be hard to measure, and the measurements might not be 100% accurate, but anything is better than nothing!

It’s hard to argue with that (and I don’t really intend to). But like so many simple, absolute statements, it hides a lot of messy complexity. So much complexity that you (or your project) can fall into it and drown. There are three big traps lying behind Gilb’s law.

1. Difficulty equates to cost. People have to work to get those measurements, the procedures may hurt productivity, etc. And if the measurements are of low quality, do they really offset the cost? Gilb’s law says that measurement is possible, and it’s always better than not measuring—all else being equal. But the cost of the measurement may nullify its benefit.
2. Whether you call it Heisenberg’s Uncertainty Principle or the Hawthorne Effect, the result is the same: measurement nearly always has secondary effects. And they may or may not be desirable. Are the people on your team optimizing for the measurement? Is that going to help or hurt your real goal?
3. Finally: few people deal well with ambiguous numbers. No matter how many times you say that the numbers have a margin of error, or may reflect many other factors, most people look at measurements and behave as if they’re absolute.

Gilb’s law is a law, but when it hits the complexities of the real world, the result can be messy. Use it carefully.