Measuring Up

Many companies still struggle to use metrics effectively. It may be that fresh thinking is what really counts.
Scott LeibsJune 1, 2007

When Red Sox ace Pedro Martinez gave up a double, a single, and another double in Game 7 of the 2003 American League Championship Series, he not only squandered a three-run lead (and ultimately the pennant), but also provided a poignant lesson in the value of metrics.

Martinez’s eighth inning collapse against the New York Yankees underscored what even a casual Red Sox fan knew: as Pedro got closer to the 100-pitch mark, his effectiveness declined markedly. Plotted on a line graph, the opposing team’s batting average soared like a shot over the Green Monster whenever Martinez approached a triple-digit pitch count.

But baseball managers don’t manage by line graphs. Grady Little left Martinez in, losing the game and, several weeks later, his job. The best the Sox could do was to enjoy watching the Yankees lose to the Florida Marlins in the World Series.

CFO Insights on Inflation, Workforce Challenges, and Future Plans 

CFO Insights on Inflation, Workforce Challenges, and Future Plans 

Download our 2022 survey report for a high-level view of finance team projections and strategies, directly from our executive readers.

Those events unfolded even as Michael Lewis’s book Moneyball, which described how a new breed of baseball visionaries used statistical analysis to assemble competitive teams on modest budgets, was proving to be not only a best-seller but also a business touchstone. If disciplined and innovative measures of player performance could level the literal playing field, what would happen if business managers thought more analytically about various aspects of their operations, from customer service to worker productivity to health-care costs to the fruits of innovation?

Some companies are doing just that, extending metrics into areas that would seem to defy measurement. How, for example, do you measure innovation, or the future capabilities of your workforce, or the optimum menu of health benefits? It’s not easy, and before serious progress can be made companies may need to overcome organizational inertia. But some experts say the time is ripe for companies to move away from a managerial “feel for the game” in favor of more-rigorous, data-driven decision making.

It’s hardly a new idea, of course; a passion for measurement lies at the very heart of American business history. Harvard Business School, which will celebrate its centennial next year, took as the model for its initial curriculum the works of Frederick Winslow Taylor, the efficiency guru whose careful quantification of manual labor gave birth to what came to be known as “scientific management.”

A century’s worth of MBAs would seem sufficient to propagate a by-the-numbers approach into every nook and cranny of the business world, but it hasn’t worked out that way. By most accounts, companies have done a respectable job of mastering financial metrics, but have largely taken a flier on measurements of operations or intangibles such as customer satisfaction or brand loyalty. Fifteen years ago the advent of the “balanced scorecard” sought to redress this imbalance by demonstrating how nonfinancial metrics could be captured and used to help managers “see their company more clearly — from many perspectives — and make wiser long-term decisions,” according to its creators, Robert Kaplan and David Norton. But despite the popularity of that approach at a strategic level, many consultants and academics say it left thorny questions unaddressed at more tactical levels.

“Metrics are a powerful communications tool,” says Michael Hammer, the reengineering pioneer who recently described the “seven deadly sins of performance measurement” in an article of that name in the Sloan Management Review. One value of metrics, he says, is to provide real-time assurance that long-term improvements are on track. “Even a small company that has embarked on any kind of major process improvement faces a long haul,” he says. “But with the right metrics in place, you can measure results right away, which becomes a powerful driver.”

That was the case at Wells Fargo, which relies on what it calls a “happy-to-grumpy” ratio to assess whether its efforts to develop a more “engaged” workforce are on track. “We don’t want to just measure results,” says CFO Howard Atkins, “we want to measure what drives our results, and that includes team-member engagement. That measure might not get cited in your general ledger, but it can be quantified in a statistically valid way, compared over time to certain goals, and correlated to business outcomes.”

Atkins says that groups within the bank that have higher employee-engagement scores also rank higher on productivity and customer satisfaction. James Harter, chief scientist for the Gallup Organization’s workplace-management and well-being practice, says that while it can be complicated to connect employee attitudes to financial performance, it can be done. Gallup administers a 12-question survey on behalf of its clients (including Wells Fargo) that assesses, on a one-to-five scale, how employees feel about everything from their role at work to their co-workers’ commitment to quality.

“We focus only on things that affect performance,” Harter says. “Otherwise, managers become overloaded with too much information.” Wells Fargo, in a sense, outsources the calculation of its happy-to-grumpy ratio, then incorporates that metric into a broader universe of measures that help it develop its workforce in a way that enhances corporate performance. Some critics argue that strong company performance may be what makes employees feel engaged, rather than vice versa. Harter agrees that there is some “reciprocal feedback” but says that employee engagement is more often predictive of financial performance than the reverse, in part because it predicts customer-service quality, employee turnover, and other outcomes that drive the bottom line.

Lowell Bryan, a McKinsey consultant and coauthor of Mobilizing Minds: Creating Wealth from Talent in the 21st Century Organization (McGraw-Hill, 2007), says that a decidedly unambiguous metric such as profit-per-employee (PPE) can be a useful way for companies to focus on what drives performance. “Metrics make the intangible more tangible,” he says. “In fact, you can’t game a metric such as PPE as easily as you can more-traditional measures of return-on-capital, because the conservative accounting around profit and the simplicity of an employee count” make the measure straightforward.

Making Connections

As companies extend metrics into various aspects of operations, they are often eager to discover linkages between them. What use is an improved customer-satisfaction score, after all, if there is no indication that it drove more sales? This attempt to relate one or more metrics to others is often labeled “business analytics” or “data-driven decision making,” although the lines between these and similar terms are blurring. Tom Davenport, co-author (with Jeanne G. Harris ) of Competing on Analytics (Harvard Business School Press, 2007), says that “like metrics, analytics is not new, but in both cases what is new is basing your strategy on them.”

Davenport points to Hilton Corp., which found that a 5 percent boost in customer-retention rates led to a 1.1 percent improvement in revenue in the following year. That, he says, is the emerging frontier: finding causal linkages between financial and nonfinancial metrics.

For example, retail giant Best Buy has been measuring employee engagement for a decade “and customer satisfaction in a systematic way for three years,” says Joe Kalkman, vice president of HR capabilities. “But we are just in the first year of understanding how these two metrics relate to one another.” Does having a more engaged workforce improve customer satisfaction, and does that in turn boost sales? If so, a company might frame its investments in employee engagement in a new light, confident that a dollar spent at one end of the chain will produce higher returns at the other end.

Best Buy has already had success in connecting improved employee-engagement scores to store performance: it found that for every 10th of a point it boosted the former, its stores saw a $100,000 increase in operating income. By looking for linkages between employee engagement and customer satisfaction, Best Buy hopes to, as Kalkman says, “fill out a systemic view of what makes our entire business model work.” Given the complexity of business today, he says, “single metrics are less and less effective.” A system of measures that can draw connections between various facets of operations and the company’s ultimate financial results is now what Best Buy and other companies are pursuing.

It’s also what balanced-scorecard creators Kaplan and Norton are pursuing: their next book, expected to be published next year, will look at the links between operational and strategic metrics. “It reflects the maturity and acceptance of the balanced scorecard,” Kaplan says, “and the increasing role that technology can play in delivering and analyzing data. The concept of tying metrics together to do more-sophisticated analysis is becoming huge.”

Run It Like a Business

At Pitney Bowes, the mailstream technology and services company, metrics played an important role in redesigning health benefits. The company has collected “de-identified” data on employee health claims, absenteeism, visits to on-site health clinics, and related measures for more than a decade, and several years ago decided that studying the patterns those various metrics provide might lead to insights about how to reshape its coverages to combat rising premiums.

Realizing that chronic diseases such as diabetes accounted for a huge percentage of health claims, the company redesigned its benefits so that preventive care is either free or available at very low cost. It even invented a sort of metrics system for employees, dubbed “Count Your Way to Health,” that provides targets ranging from 0 (that is, no smoking) to 100 (use a seat belt 100 percent of the time), with stops along the way for 1 (flossing every day) to 25 (maximum percentage of body mass attributable to fat) to 30 (minutes of exercise per day).

“Health benefits tend to be consensus-based, or driven by your competitive position,” says Jack Mahoney, director of strategic health initiatives at Pitney Bowes. “We thought, why not run it like an internal business, by using metrics?”

While success stories sound simple enough — gather data, analyze it, act on the revelations it provides — by most accounts companies botch metrics far more often than they get them right. “I’m continually astounded at how poor the use of metrics continues to be,” says Brent Wortman, a senior partner in the financial-management practice at Deloitte. He cites as common problems the inclination to roll numbers up from the bottom, so that the board and senior management are overwhelmed by irrelevant numbers; the lack of standards for linking operational and financial metrics; and the failure to put enough rigor behind hard-to-measure attributes, such as customer services, opting instead for either poorly conceived metrics or none at all.

Hammer agrees wholeheartedly. Among his “seven deadly sins” are vanity (emphasizing metrics that you know cast you in a good light), provincialism (allowing functional departments to measure themselves only on their own narrowly defined goals, often at the expense of the organization as a whole), and “inanity,” which he defines as a failure to appreciate the consequences of a given metric. As an example, he cites a fast-food chain that focused on reducing the amount of wasted food; to improve that metric, restaurant managers stopped the practice of cooking in advance of anticipated demand. Waste was reduced, but service was delayed and sales suffered.

“Companies have not brought to bear a rigorous, analytical mindset about what they measure, how, and why,” Hammer says. “Nor do they regularly review what metrics they track and discard those that are outmoded.” Now would be the time to redress that situation, Hammer says. “Why are metrics such a big deal in 2007? Because foreign competition, shareholders, customers, and other forces are collectively putting companies under more performance pressure than they’ve ever faced, and to be smart about performance you have to be smart about metrics.”

Scott Leibs is a deputy editor of CFO.

Innovation: Metrics Go Macro

One common rap against metrics is that they too often provide a look back rather than ahead. Another is that they are rarely applied to intangibles such as innovation. A third is that they are poorly conceived.

Can a federal effort to measure innovation overcome all those obstacles? The U.S. Department of Commerce believes it can. In April, a Commerce Department advisory committee formed earlier this year issued a public request for comment as it seeks to develop a set of metrics that will help the government assess the state of innovation within the U.S. economy. The goal is nothing less than to ensure that the United States remains competitive in the face of lower-cost competition from around the globe. That it should put the creation of suitable metrics at the heart of that effort says much about the value of metrics — and offers businesses useful lessons in how to think through the complexities of developing appropriate measures.

Don Siegel, professor and associate dean of the Anderson Graduate School of Management at the University of California, Riverside, and a member of the Commerce Department’s Measuring Innovation in the 21st Century Advisory Committee, says “the economy has changed and become much more knowledge-based, but we’re still stuck with an industrial mindset. People are still focused on old measures based on expenditures and simple measures of labor input.”

Siegel admits the job won’t be easy, a sentiment shared by Scott Anthony, president of Innosight LLC, a consulting firm that specializes in innovation. “No single metric gets it right,” Anthony says. “Take R&D spending: last year Ford was number one, but does anyone think of it as the most innovative company?”

“What makes it difficult,” Anthony says, “is that a system of metrics that focuses on an intangible will almost certainly include some that are squishy. And squishiness gets away from the very point of metrics.”

The advisory committee acknowledges the difficulty of the task it faces; its two-page request for comment (available at spells out a series of questions that it hopes to answer, and in so doing provides a handy primer for companies hoping to improve their own efforts to measure innovation or other intangible aspects of operations. — S.L.

4 Powerful Communication Strategies for Your Next Board Meeting