Not by the Numbers

While finance chiefs do seek tangible results from executive education programs, they don't like measuring their effects in terms of ROI.
Lisa YoonMay 31, 2005

In May 1998, CFO published “Sometimes a Great Notion,” an article on education for finance executives. When it came to return on investment, wrote Stephen Barr, “a number of companies contacted by CFO have had to rely on mostly anecdotal evidence that the educational investment is worth it.” Lacking a more precise method of measurement, companies usually used surveys or interviewed participants and their managers.

Six years later, companies haven’t much altered their methods of measuring the return on investment for executive education. What has changed is the attitude. Faith in the intrinsic value of education is common today, causing executives to require less justification for the investment. In 1998, executives like Terry Carlton, then finance director of decision support for Sprint Corp., could made statements like, “We’re firmly committed to training, but we’re not real confident that it’s making a difference.”

Today, they’re more confident. “People are our most valuable asset,” says Lance Newquist, sector vice president and CFO of Northrop Grumman Corp.’s integrated-systems division. “Anything we can do to invest in people is a good thing.”

At pharmaceuticals giant Johnson & Johnson, Jeff Cacciatore, head of the Finance Leadership Development Program, has a similar view. “We have an inherent belief in [education’s] value,” he says, “and that it will translate into business results.”

CFO George Brown of snack-food manufacturer Old London Foods says there’s a payoff in simply attending executive-education classes. “On some level it’s simply about exposure to other things,” he says, especially other learners. “It’s good to have the perspective of other managers, who might come from different industries or parts of the world.”

Executive education also provides a valuable “refresher” for an executive’s knowledge base, “so you don’t go stale,” says Brown. He himself takes part in courses, hunting for the latest in ideas, trends, and finance techniques — paying little mind to the potential effect his learning will have on profits.

For What It’s Worth

That attitude’s likely to come as a surprise to those who think CFOs would want hard numbers to back up executive-education spending proposals. To be sure, education isn’t cheap. Tuition for a single open-enrollment course at Harvard ranges from $3,500 to upward of $10,000 per person, for instance.

Then there are travel expenses and the cost of an employee’s time away from the job. Custom programs, depending on the school and the number of participants, can run into the hundreds of thousands.

What’s more, companies are spending more on education. Many schools, including Harvard and Columbia, report a steady rise in enrollment in recent years. Wouldn’t finance chiefs want to know if education is really worth that kind of money?

For the few that do want to try their hands at calculating answers to that question, a formula, widely recognized by ROI experts in the exec education field, is available:

Rather than a pure ROI analysis, however, most executives seem interested in qualitative results. Most of the 270 executives and human resources directors responding to a new study sponsored by the International University Consortium for Executive Education reported using non-numeric methods like interviews, 360-degree feedback, questionnaires, and self-assessment to weigh the worth of executive education. Only 10 percent said they derived the effectiveness of courses by measuring the financial value of resulting business changes.

No surprise, then, that when Johnson & Johnson’s Cacciatore’s asked if the company cares about evaluating executive education after completion, he says, “I don’t think we do.”

Dress Casual

At Old London Foods, there’s a relatively casual approach to exec ed. Brown says that it’s viewed mainly as an executive perk, with no set budget. Decisions on who gets to attend which programs are based on the company’s performance in a given year; when times are lean, the company eases up on educational spending.

While that approach is common among small companies like Old London, which recorded revenues of $50 million in 2004, other organizations do monitor the investment more closely. But even in such cases, CFOs are less interested in hard numbers. “They want to know why they should invest in executive education, what impact it will have, and where in the company it will have impact,” says Robert Yerex, who works with CFOs as head of research and analytics at Unicru, an HR consultancy.

A former CFO of small companies, Yerex explains that finance chiefs, weary of so many sales pitches, tend to become skeptical of ROI projections. “They’ll cut it in half,” he says. “You tell them 300 percent ROI, and they think 150 percent.”

ROI can also be limiting. “I’ve seen too many companies that focus on ROI, and they start thinking short-term,” says Steve Rehnberg, the CFO of Idaho Asphalt Co. Still, as with any other investment, he wants to know he’s getting a decent return from exec-ed spending. Rather than seeking a number, he looks for tangible results like improvements in management skills or the use of new analytical tools.

In the case of a business-performance-measurement program he attended at California Institute of Technology in 2001, he picked up the idea of using a “dashboard” of critical variables for business success, for instance. Rehnberg and his team now lay out all the company’s key variables, giving him a thorough picture of business performance. For example, one key variable at Idaho Asphalt is tons of asphalt shipped. The dashboard shows tons shipped from month-do-date versus year-to-date and offers up comparisons of those figures to company projections, enabling Rehnberg to anticipate problems. If the tons-shipped figure is lower than planned, for instance, he knows that accounts receivable will be lower; hence cash flow will be lower than projected.

Evidence vs. Proof

One reason that ROI is so rarely used is that it’s hard to distinguish what happens after an executive takes a course as mere evidence of a return or whether it’s absolute proof, as Karen Vander Linde, a PricewaterhouseCoopers’ HR principal, puts it. Does a boost in profits absolutely stem from an executive’s participation in a course? It’s often hard to single out executive education as a definite contributor to financial performance.

Even those who try to quantify educational returns for a living warn against the dogged pursuit of a dollar value as proof. Schon Beechler, the faculty leader of the Columbia Learning Impact Initiative, stresses that any number arrived at is only part of the picture. “To think ROI [alone] can measure long-term value is naïve,” she says.

In fact, math is only about a third of what goes on in the Columbia initiative, which analyzes the financial return of the four-week Columbia Senior Executive Program at the companies of the program’s alumni. The rest is investigation into intangible factors — which are more important in determining whether exec ed will pay off, Beechler contends

For example, Columbia researchers study participants’ personal attitudes and work environments. One finding is that alumni who had supportive bosses showed a significantly higher rate of success in business challenges after taking the program than those whose bosses were unsupportive, uninterested, or otherwise disengaged.

Indeed, the best executive education program won’t pay off back at work if the environment there doesn’t encourage development. For all that organizations spend on executive education programs, “a lot of companies are leaving money on the table from not addressing the support of participants,” says Beechler.

Northrop Grumman’s integrated systems unit addresses that problem by considering each executive individually. Unit heads assess executives and send them to educational programs based on their particular needs and potential. Newquist’s definition of return on such investments extends far beyond the individual level. “The return is that you have a strong talent pool,” he says.

In fact, there’s a point beyond which the pursuit of ROI becomes self-defeating; the point of executive education in the first place gets lost. “Measuring ROI is fine and noble” until the forced outcome becomes a number that goes out two decimal places,” says Unicru’s Yerex.