Vive le ROI

The quest to pin down ROI on technology is turning into an industry in itself -- and sending CFOs back to basics.
Adam LincolnOctober 1, 2002

Sometimes a finance chief has to cut his losses. Just ask Ross Hughes, CFO of Australia-based BankWest. In the early 1990s, executives at the bank, which has assets of more than US$12 billion, went shopping for a core banking system. When they couldn’t find an off-the-shelf package that met their requirements, they built their own. By the end of the decade, BankWest had developed a system it felt other banks might buy. Early signs were good: UK-based Halifax Bank of Scotland, which owns 56 percent of BankWest, snapped it up after a rigorous global tender process. Computing heavyweight IBM thought the system had prospects, too, and came on board in a joint venture to market the system in Asia. Hughes recalls: “We felt we’d created something exceptional.”

But the banks in Asia didn’t bite. So, earlier this year Hughes and his colleagues pulled the plug on plans for export success. The move also had implications for the company’s IT activity at home. In June, BankWest announced it would merge its in-house IT division with CBSIS, the subsidiary that drove development of the core banking system. The creation of a single IT unit would save the company “a few million dollars” by cutting head count and duplication of effort. But more importantly, Hughes says, it would focus IT resources on work that supported the bank’s own strategic agenda.

A retreat, certainly, but BankWest deserves kudos for attempting to transform IT from a cost center to a revenue generator. Besides, a data services and disaster recovery joint venture with the local subsidiary of U.S.-based Unisys is thriving, boasting clients from the government and mining sectors. But even if commercialization of their IT department is the last thing on most companies’ minds, BankWest’s soul-searching is not unique. The economic downturn, coupled with the dot-com burst, has prompted a backlash against IT, and turned the quest to pin down ROI into an industry in itself. Wherever they look for answers amid vendor talk of ROI calculators and consultants debate over the best measurement methodologies, companies want an answer to a universal question. As Hughes observes: “How do I know whether IT adds value to my organization?”

Drive Business Strategy and Growth

Drive Business Strategy and Growth

Learn how NetSuite Financial Management allows you to quickly and easily model what-if scenarios and generate reports.

Prima Donna

The relationship between IT and the rest of the business has always been tetchy. Even technology vendors admit that something was amiss. “Before the dot-com bust, technology was pretty much a prima donna,” observes Kuala Lumpur-based Rahul Patwardhan, president of the Asia Pacific and Europe operations of NIIT, the Indian outsourcing giant. “On one hand, this wasn’t good. But if one does not invest in innovation, then someone else will,” he says.

Sam Lo, group financial controller at Singapore-based ECnet, knows all about it. ECnet sells supply-chain solutions, an area full of promise — reflected in a client list that includes Matsushita, Philips, and Toshiba — but relegated to the back burner at many companies. As a result, Lo must rein in costs while ensuring the company doesn’t impair its ability to react when market conditions improve. As a technology purveyor, continued investment in a cutting-edge platform is key, while building internal operational efficiencies.

This means Lo must work doubly hard at his forecasts. He admits that it’s a challenge to make so many assumptions. “We look at the what-if scenarios, the implications if things don’t work out as we expected,” says Lo. “It’s very hard to calculate absolute returns, especially when the investment is for internal use. It’s easier to calculate the ROI on a platform offering services to customers,” he says. It doesn’t help that the goalposts are shifting. Three years ago ECnet bought financial systems from Microsoft Great Plains. While the implementation ran smoothly and the company can now produce group accounts in half the time, Lo reports that financial payback dates have been delayed, and plans to integrate this with front-end systems have been pushed back to 2003. That’s not the only bad news. “We won’t embark on the next round of upgrades until 2004,” he reports.

Like many of his peers, Lo doesn’t have much choice. Companies still recognize the need to innovate, but experts agree that there has been an undeniable refocusing of late. “Companies are taking a much more disciplined approach to IT-enabled investments — and having that responsibility at the top level,” says Marianne Broadbent, Australia-based group vice-president of U.S. research outfit Gartner Group. Gartner reckons 20 percent of global corporate IT budgets were wasted in 2001 on technology that failed to achieve its objective — a loss totaling US$500 billion. Culprit behavior included unnecessary customization of software, poor central control of software licensing, and the failure of E-business projects.

Indeed, E-business projects — more speculative by nature — led many astray. For a time, many companies didn’t seem concerned by the question of financial ROI — or at least by finding proof. Hard financial metrics flew out the window as companies became distracted by new priorities and metrics that would justify “must have” projects. In a survey by the Economist Intelligence Unit in 1999, “increase in customer satisfaction” was cited by 60 percent of respondents as a “very relevant” performance measure for E-business investments. This worthy but somewhat intangible aim beat more traditional concerns, such as reduced operating costs and increase in share price or business valuation.

That’s not to say soft measures don’t have a place. “These newfangled parameters, like sticky Web pages, were intended to help predict financial business impact of a project,” observes Patwardhan of NIIT. In other words, it was felt that assessing the stickiness of a site would lead to certain probabilities of people actually buying from the site, or advertising on the site. But, somewhere along the line, people forgot this and simply stayed with the soft parameters. “As long as one uses the complete logical model for computation and factors in the right probabilities and risk assessment, there is nothing wrong with using soft parameters,” Patwardhan says, but “when we forget that the purpose of using soft parameters is to describe the business model and how final financial numbers will get delivered, we have a problem.”

To Basket, to Basket

Of course, useful financial metrics do exist, and there’s really no reason not to use them. Hughes of BankWest is one CFO who believes in old-fashioned finance. “We stick to our cost of capital pretty rigorously,” he says. When assessing IT proposals, he applies net present value (NPV) and payback period, and has adopted a self-styled approach to economic value added (EVA). “EVA has been an evolutionary thing within BankWest,” he says. “While we haven’t got to the point of saying, ‘what is the fundamental EVA of a particular project?’, arguably, an NPV combined with a couple of other things, you’re sort of doing the same thing anyway.” Hughes also looks at earnings per share and absolute profits. “Even though a project might have an NPV that’s positive, it might be because five years out it delivers A$30 million (US$16 million), but next year it delivers a loss of A$10 million. So obviously there are P&L issues as well,” he says.

There’s a good reason why Hughes embraces this grab bag of measures. “The danger is you can choose not to do something just because it doesn’t meet a particular measure,” he says. “Don’t just look at payback period and live and die on that. I’m a strong believer in not using 20 measures, but I think a small basket of metrics is prudent. Your business case assessment might fail on one of them, but pass on four of them — and that’s okay,” he adds.

This is particularly important when strategic projects are at issue. Discounted cash flow (DCF) techniques such as NPV and internal rate of return (IRR) work well with traditional capital budgeting problems, such as replacement decisions or alternative production methods. Where they fall down is on strategic investments, such as evaluation of new product lines or investment in R&D. Difficulty estimating the discount rate, forecasting the project’s cash flows, estimating its impact on cash flows associated with other corporate assets, and estimating its effect on the company’s investment opportunities and strategy, can all undermine the integrity of a calculation. What’s more, DCF valuations assume the business will follow a predetermined or static plan, which isn’t always how things work out.

“We have the usual problems, arguments, debates over the weighted average cost of capital (WACC) calculation,” Hughes concedes. “Most accept that our WACC is as good as we can get it, but the issues we have relate to what discount rate should be used when we do an NPV, and for different projects,” he says. Hughes says that generally, the organization is consistent in its approach, but also examines what-if scenarios. “This enables us to say, ‘if we’re wrong here and it’s not 12 percent, and we applied 8 percent because of these three reasons, then let’s acknowledge that the NPV changes accordingly,’” he explains.

In other words, flexibility is key. An acceptable payback period is determined by the type of project, Hughes says, and depends, fundamentally, on how much of an asset the company thinks it has created. “If we think we’ve created a 10-year asset, then we can cope with a three-year payback. If we think we’ve only created a 12-month asset, then we mustn’t expect a three-year payback.” Recently, for example, BankWest installed enterprise resource planning (ERP) software from U.S.-based Oracle. “An ERP project doesn’t need to have payback within six months; it is going to be with us for a long time,” Hughes says.

Power Plays

ECnet’s Lo admits that at the time, he was nervous about choosing Great Plains Software to run his company’s financials. “The company was not very well known in Singapore at the time and we were looking for a company that would be around ‘forever’.” Then, in late 2000, Great Plains was acquired by Microsoft — a development Lo admits helped him to feel better about his decision. “It’s not just that one vendor could go bust. Many others are required to put a solution in place,” he says. The backing of Microsoft is a bonus, he says, because of that company’s central role in global technology platforms.

Indeed, Broadbent of Gartner says smart organizations are placing much greater emphasis on treating their IT-related investments as a portfolio. This, she explains, means “understanding that each time you make a decision about a particular investment, you don’t make it on its own. You have to look at your total portfolio, look at the level of risk to which you are exposed at the moment, look at the organizational stress that is there and ask: How much more change can the organization take?”

Having the right people overseeing the portfolio is also crucial. BankWest, says CFO Hughes, has seen a fundamental shift in the past few years. Now, he explains: “IT doesn’t generally come up with a project and say, ‘The business needs it.’ Typically, the business identifies a need, and takes responsibility for determining the right way to resolve that need.”

The Oracle ERP system is a case in point. Simply put, the business implications of ERP — such as change management, cultural issues, and accounting resources — were too big to be left to the IT experts. “IT becomes part of the feasibility and assessment team, and scoping team, but the project itself is not run by IT in any way,” Hughes says. Certainly, IT plays a critical part in the ultimate steering committee and project team, he says, but the ERP project manager, who was required to build the feasibility and business case and build up a number of metrics, hailed from the finance department, not IT.

Follow-through, says Hughes, is crucial. As well as a post-implementation review, a benefits realization study is conducted, which involves monitoring progress for an extended period of time. “Depending on the life of the whole thing, it might be done over six months, it might be done over a couple of years,” Hughes says, adding, “We try to keep the process fairly simple, so it’s not a burden to the business.”

Broadbent of Gartner says this is just as important as the initial investment decision-making process. Failure to instigate a “serious benefits realization or performance management process” is a major failure of companies, says Broadbent — causing trouble that can easily be avoided. There are, not surprisingly, plenty of “performance management” software suppliers that promise to help. She says such a process should not be afraid of pointing fingers. “I look to see: Who’s responsible for delivering the benefits? What is the person’s name? Is this tied to their performance and reward systems? Unless there is that kind of serious tracking, and then that is monitored on a regular basis, we find these things slip and slide, and people do not track appropriately at all.”

Vendor Management

Technology may seem to possess a hyped-up life force of its own, and it’s easy to take slingshots at IT vendors, but it’s a mistake for companies to feel maligned. “I really think it comes down to our own discipline — it’s up to us to manage suppliers adequately,” says BankWest’s Hughes. He believes companies can set the scene for ROI success by choosing the right supplier via a tight RFP or tender process, then building a contract with the right service level agreements (SLAs) for the installation period, as well as ongoing SLAs. “There’s a big difference between the salespeople in the software companies and the technical deliverers,” says Hughes. “The trick is to get into both of their heads and make sure you’ve got clarity from both,” he says.

One way vendors try to reassure clients that they see clearly is through so-called ROI calculators. These days, any vendor worth its salt attends sales pitches armed with a method for estimating likely outcomes from inputs tailored to the customer’s predicament. Meantime, a new breed of third party tools providers is emerging (see “ROI Consultants: Truth and Lies,” below).

Usefulness depends on the type of project. ROI calculators are most useful for what Patwardhan of NIIT calls “procurement-type ROI” — where a company has already decided on a type of project but must identify the most effective way to execute it, such as through in-house development, outsourcing, or even “offshoring” to overseas developers such as NIIT. Calculators fall short when it comes to the wider strategic benefits. “No cookie-cutter calculator can help here — each project will provide business returns in different ways,” says Patwardhan. “The devil is in the detail.”

Adam Lincoln is a technology writer based in the U.K.

ROI Consultants: Truth and Lies

If a vendor’s ROI calculator seems open to accusations of bias, a new breed of companies promise a more impartial approach to technology investment analysis. U.S.-based Alinean, for one, takes a “dashboard” approach that considers net tangible benefits, intangible benefits, and risk factors with software called ValueIT ProjectROI. Another US-based player is Nucleus Research, which provides pre- and post-deployment financial analysis and advisory services to help IT decision makers make the right moves.

“Unfortunately, we find many companies making decisions using methods that do not correctly evaluate benefits such as TCO or NPV,” says Ian Campbell, Nucleus’s vice president of research. “More seriously, there are numerous false ‘marketing metrics’ such as cumulative ROI (which dramatically overstates the ROI by using the sum of benefits over three years) and return on opportunity that consultants often promote but only serve to mislead,” he says.

Nucleus says that without opening a spreadsheet, CFOs can use some key criteria to assess a project and the likelihood of ROI:

  • Breadth: How many people will be helped by the application? The greater number of people, the greater the potential ROI.
  • Repeatability: How often will people use the application? The more often an application is used, the greater the ROI.
  • Cost: The more costly the task, the greater the benefit from automation or appropriate technology support.
  • Knowledge capture: The greater the potential to reuse the information in the system, the greater the potential ROI.

Costs and benefits can be either one-time or recurring, so be sure to include them appropriately. Nucleus suggests following these basic rules when gathering and including costs in the calculation:

  • Count everything that is directly associated with the project. For example: “I purchased a Web server for this project.”
  • Don’t count infrastructure items not associated with the project. For example: “I used the existing Web server.”
  • Do count infrastructure items that were driven by the project. For example: “The company purchased a Web server because of this project and two others like it”, means you should include one-third of the cost.

Personalized consultancy, of course, comes at a price. Question is, will it undermine the ROI? —A.L.

Project Portfolio Criteria: Stop the Squeaks

If the corporate world is unhappy with its IT scorecard, research from U.S.-based Meta Group reveals why. According to Meta, 70 percent of Global 2000 companies still use single-dimension criteria to select projects — generally cost/benefit analysis or some type of return calculation. Fewer than 10 percent apply several levels of criteria — risk, life cycle, return, planning horizon, and the like. “The remaining companies still rely on a first-come, first-served or squeaky-wheel method of allocating resources to projects,” laments Marnie Ross, a Meta Group analyst.

Still, in the next few years, investment in “project portfolios” will grow, slowly. This means taking the gamut of business complexities into account when evaluating IT projects, and hopefully cutting the chance of investing in low-value projects. Meta believes that by 2006, more than 30 percent of Global 2000 companies will use multidimensional project-portfolio decision criteria.

Ross says various factors should be weighted — depending on competition, regulatory restrictions, market position, and long-term business goals — to ensure the right mix of investments. Typical considerations include:

  • Term: A short-term project like application development might be required to take advantage of tax rebates available only for the current year. Long-term projects include application and infrastructure changes demanded by regulatory bodies.
  • Risk: Business risk tolerance is affected by factors such as corporate aggressiveness, cash flow, and ownership (public or private). Internal business risk factors (such as employee acceptance of new systems) should not be forgotten.
  • Duration: For resources to be tied up on a long-term project (more than two years, says Meta) other factors (regulatory, strategic benefit, high ROI) must exist.
  • Expense: Most companies take this into account, with existing spending authority structures, budget processes, and cost/benefit reviews. But as the sole criterion for decision making, notes Ross, it is “exceedingly incomplete”.
  • Scope: Unless intimate knowledge of customers is the main driver of business value, projects with a global scope usually deliver greater value than those focused on local results.
  • Posture: Tactical, line-of-business-driven “offensive” projects — those focused on seizing market share — and “defensive” projects — which reduce the risk of customer loss — can add value, provided other types of projects are in the portfolio.
  • Planning horizon: Typically, strategic projects are driven by corporate offices. The focus is on business sustainability and improved management capabilities — and ROI can be hard to pin down. Classic example: ERP.
  • Life cycle: Usually infrastructure-related, so-called asset life cycles span the business; they concern account skills, applications, processes, business markets, and customer segments. Over time, each element ages from “new and valuable” to “outdated liability.” By tracking these assets, project portfolio managers know when to retrain, reengineer, replace, release or retire systems — setting in train a whole new string of projects.