Why ROI is Broken

The Corporate Executive Board has an interesting article on Business Week's website. They argue that ROI is being used incorrectly by many IT organizations. In particular, they argue that the problem is not the difficulty in accurately measuring the benefits of an IT project. Instead, they explain that, "The real reason to actively distrust ROI projections is the other side of the equation: Most IT shops are punting when it comes to capturing the true cost of the investment."

I would argue that there is a much more substantial reason to question ROI calculations by IT departments, even if they are tracking labor time and costs accurately. The fundamental challenge lies in the decision-making process leading up to the launch of a major IT project. What we often see is that the IT managers, the hardware and software vendors, and the consultants who will aid in the implementation all collaborate to build the ROI. What's the problem with that? Well, of course, all three parties are advocates for the project. They want it to be approved. Why would they ever come up with an ROI calculation that suggested that the project should not be done? In short, the advocates are the primarily analysts in many of these situations. We don't have an unbiased perspective. ROI is not a purely objective measure... it's only as good as the assumptions and information that you put into it. If advocates drive the inputs, the output of an ROI calculation is likely to be flawed/biased.