ROI: Mad to Measure

Calculating the return on E-business investments isn't easy, but that doesn't stop companies from trying.
Hilary Rosenberg and Russ BanhamSeptember 15, 2001

Measure not the work/Until the day’s out and the labour done/Then bring your gauges.

So said British poet Elizabeth Barrett Browning — who, it must be noted, did not have an MBA. Companies generally prefer it the other way around: They’d rather not lift a finger, or spend a penny, until a hefty dose of analysis proves that the effort will pay off.

But when it comes to spending on information technology, particularly on those technologies that support ebusiness, a less conforming attitude has prevailed: “Gauges? We don’t need no stinkin’ gauges.” So said a generation of corporate executives who confronted the Information Age with steely eyes, checkbooks at the ready. No wonder Alan Greenspan was moved to note that “the fact that the capital- spending boom is still going strong indicates that businesses continue to find a wide array of potential high-rate-of-return, productivity- enhancing investments. And I see nothing to suggest that these opportunities will peter out anytime soon.”

Drive Business Strategy and Growth

Drive Business Strategy and Growth

Learn how NetSuite Financial Management allows you to quickly and easily model what-if scenarios and generate reports.

That was in March of 2000, just as the Nasdaq index began its descent and the phrase “New Economy” became as dated as “Been there, done that.” In the first two quarters of this year business expenditures on equipment and software declined outright (according to Commerce Department figures) versus merely slowing in growth, as many surveys of CFOs and CIOs had seemed to indicate.

Current economic conditions have inspired renewed calls for a tough- love approach to technology investments. Many companies seem eager to move beyond what one consultant has called “the blank-check mentality”; as often as not, however, this gives way to a blank-stare mentality. Calculating technological payback has bedeviled companies for years, particularly when it comes to the sorts of infrastructure investments that are designed to provide a foundation for other, (perhaps) more quantifiable, investments.

Yet forward-thinking companies insist that technology bets can be placed with some rigor. Ebusiness does not require a leap of faith, they argue, so much as a willingness to evaluate return on investment in the broader context that its potential demands. Contributing editors Russ Banham and Hilary Rosenberg spent time with half a dozen companies that have brought some fresh thinking to this critical area. These enterprises compete in far different arenas, from professional sports to transportation to etailing, and they’ve tapped technologies ranging from wireless Web access to Internet advertising to email management. The technologies they’ve deployed, and the efforts they’ve made to assess the value they’ll receive in return, offer lessons for all companies interested in tapping the full potential of ebusiness.

Yellow Corp.

For freight company Yellow Corp. (, technology investments are a two-way street. The company drives new, lower-cost capabilities out to its customer base, and in return watches additional business roll in. That’s true for ebusiness initiatives as well as more traditional forms of IT spending. “Our cost/benefit matrix is applied to all [IT] projects and has to be measured,” says CFO Don Barger. He says that no projects are exempt from this two-pronged analysis, even an ambitious undertaking such as, a multifunction portal that has saved the company so much money that the prospect of its winning new business might seem almost beside the point.

And yet new business is very much on the minds of executives at Overland Park, Kansas—based Yellow, who know that faster cycle times improve the company’s capture rate: Make a customer happy on one route or shipment, and you’re almost certain to win additional business.

The company launched the portal at the start of last year, intent on providing customers with a single Web site that can satisfy a range of needs, including calculating rates, arranging pickups, tracking shipping status, filling out bills of lading, and communicating with customer service personnel on a range of problems and general queries. The company surveyed its customers beforehand and found them ready and eager to embrace Web interaction. A close look at the sites operated by UPS, FedEx, and others helped Yellow understand what features made the most sense.

Moving customer interaction to the Web has huge implications for the company, because it permits Yellow to offer an expanded range of services far more cheaply than if customers were to rely on phone contact with customer service reps. In fact, Yellow says that the portal has saved it $20 million in this regard. For many companies that might prompt cries of “Case closed!” on the ROI front, but Yellow doesn’t stop there. Its calculation also involves the increase in customer capture rate, which correlates to a rise in sales. While the company won’t release actual numbers, recent surveys found that customers doing business with Yellow via are 23 percent more likely to do repeat business than customers who interact in more traditional ways.

Time is money, of course, and as with all of Yellow’s technology projects, the portal has to deliver its minimum return in three years. “There’s so much change in the technology area that if you can’t get that payback, it probably won’t get a good return,” says Barger. Speed is also at the heart of most of the enhancements that have been made over the past 18 months. An application called Exact Express, for example, allows corporate personnel to give potential customers all pickup and delivery schedules and price options while on the phone or online with them. Sales made via this system were up 40 percent last year and 23 percent in the first quarter.

Yellow initially rolled the portal out to about 4,300 customers. Today it’s used by more than 10 times that many, and receives 18,000 hits per day. Fully half of the company’s customers use it, and about 60 percent of the firm’s revenue flows through it. —Hilary Rosenberg

Delta Air Lines

We need to justify things based on hard economic value,” says Vince Chirico, general manager of ebusiness for Delta Air Lines ( That no-nonsense approach would seem to be bad news for wireless applications, which at this stage seem more like a nice-to-have than a business necessity.

But for high-flying, hard-charging business travelers who spend much of their time rushing through airports with a cell phone in one hand and a carry-on bag in the other, wireless access to travel data has become more desirable than an aisle seat. Delta was quick to spot this need, and since last year the Atlanta-based company has been offering wireless Internet services that give customers quick access to a wide array of flight information; the airline intends to build on that foundation by adding several interactive features.

Today, customers can use a wireless device such as a personal digital assistant to retrieve flight information online, access various terminals, and download timetables to help them reschedule canceled flights. SkyMiles medallion members can use wireless devices to check in remotely for US flights. In the future, customers may be able to do much more, including select and change their seat assignments, reserve and pay for tickets, and request standby and upgrade services. Delta is now testing a virtual check-in system in six cities.

Delta prioritizes its Internet projects based almost solely on net present value, strategic value, and risk associated with each project. Project managers first figure a net present value for the initiative and then add a measure for strategic value representing customer service. Those two measures are weighted to arrive at a total value score. That total value is then plotted against risk, which equals the size of the initial investment in the project, whatever resources are required from within the airline to develop and implement it, and an assessment of possible technological hurdles, such as the rate at which the system becomes obsolete.

In the wireless area, Delta assesses ROI in terms of both productivity and cost savings. The productivity metric helps in calculating both net present value and strategic value. For example, the company estimates how many more revenue-producing calls (calls that book tickets) will be handled by reservations agents because, as a direct result of wireless self-service, they don’t need to take as many routine calls dealing with check-in, seat selection, or general inquiries.

Cost savings are the key component of the net present value calculation; savings can come from either materials or personnel. If more customers can access information and conduct transactions via the wireless Web, then Delta can serve a growing customer base without having to staff up significantly. And the airline expects to save on paper because last-minute changes to tickets made over the Web won’t require a new ticket to be printed at the gate. “The system helps us reduce future costs against existing resources,” thus raising ROI, Chirico says.

Delta expects a new revenue stream to further enhance ROI. In a venture with American, United, and Boeing, aircraft will be equipped with broadband Internet connectivity, allowing the airlines to sell in- flight wireless services starting next year. —HR

Portland Trail Blazers

When Microsoft co-founder Paul Allen bought the Portland Trail Blazers basketball team ( a few years ago, he decided that a winning team was only part of the total entertainment experience. He and the front-office staff looked for other ways to increase fan involvement. Last season, for example, “we had this wild notion of calling all our ticket holders and asking them to wear red [clothing] to the game, since red is our primary team color,” says Tony Cesarano, director of database and Internet marketing for Trail Blazers Inc. “We called it ‘The Lakers See Red’ night.”

Getting the word out to customers without tipping off the media was the challenge. Knowing better than most what technology can do, Allen had invested in a Web-based customer relationship management system in 1999, one that captured customers’ email addresses and specified their preferred way of being contacted. Within days, the Blazers got the word out about “See Red” night, and more than 90 percent of the fans caught the spirit.

The Blazers’s eCRM system, sold by Bellevue, Washington—based Onyx Software Inc. (, captures data on all interactions between the team and ticket buyers, from the big- time sponsors and corporate box holders who fete their clients to more regular folks who bring the kids for some hot dogs and hollering.

In the post—Michael Jordan era, basketball finds itself competing fiercely with action movies, boy bands, and even Nintendo, so knowing who the customers are and what they respond to has become more important than ever. Today, when season ticket holders call up the Blazers (or receive so-called wellness calls from service reps), they have a very personal experience, with questions like, “Did Tessa graduate yet? Did you get that Scottie Pippen poster? Will you be bringing the boys to Boy Scouts night?”

“Our customers expect more in this day and age,” says Jim Kotchik, senior vice president and CFO of the Trail Blazers. “They’re paying more for games and want more in return — a better entertainment experience and better service.” And with the newly built Rose Garden holding 20,000 fans versus fewer than 13,000 for the previous venue, the Trail Blazers had to find ways to put bodies in all those additional seats.

The CRM data is a veritable gold mine of potential sales leads, telling reps the types of events the ticket buyer enjoys (country western music, for example), how they like to be contacted (by email), and what they like to be contacted about (Friday night events). In addition, the system gives the company leverage with promoters representing artists who may be unsure whether a Portland stop makes sense. “By tapping into the CRM database, we can point out that there are 10,000 people interested in an Eric Clapton concert,” explains Cesarano.

The big test of the system came in 1999, a few months after it was installed. Allen wanted to bring a women’s basketball team franchise to Portland, but the Women’s National Basketball Association said he’d need to sell at least 5,500 season tickets before it would guarantee a franchise. The team decided that parents of school-age girls would be hot prospects, and the CRM system made it easy to identify and contact them.

Kotchik says the two primary ROI metrics used to determine the effectiveness of the eCRM strategy — productivity improvement and customer service — indicate it is paying off handsomely. “We can measure things like overall revenue, but that wouldn’t tell us if we’ve got more fans because of how we service them or because the team just pulled in two new superstars,” says Kotchik. “We needed to develop other metrics to analyze the impact.”

One metric is season tickets, which reduce a team’s dependency on more labor-intensive box-office sales. “The system helps us target and service season ticket holders, which offer a bigger bang for the buck,” says Cesarano. Since introducing the eCRM strategy, season ticket holders increased by 5.4 percent in the first year and 7.3 percent the following year, he notes.

“Our sales reps are able to focus their energies on the most likely targets for tickets to a particular event,” adds Cesarano. “And our service reps, through their wellness calls, enhance customer loyalty.” The number of such calls is up from zero before eCRM to nearly 7,000 this year.

The company measures the success rate of every individual sales campaign, tracking how much money went into the campaign and the results of who responded. “We also measure the number of customer complaints, as well as whether or not the customer later feels the problem has been resolved,” adds Cesarano. “And we measure the effectiveness of email notifications to customers about tickets to an event to determine how successful the strategy is.”

All measurements are proprietary, but Cesarano and Kotchik say they indicate conclusively that the strategy is a keeper. “We ask ourselves continually if we had to do this all over again, would we,” says Kotchik. “The answer is yes — even if it had cost twice as much.”

As the basketball season drew to a close in June, the Trail Blazers celebrated their 100th sold-out game. Scottie Pippen surely had something to do with that, but so did Tony Cesarano and Jim Kotchik. —Russ Banham


Mikasa ( learned the hard way how to be an etailer. In 1999, the Secaucus, New Jersey— based manufacturer of fine crystal stemware, dinnerware, and upscale gift products launched a Web site that it now says was simply a test balloon. “The site marketed only 400 of our 20,000 SKUs,” says Seth A. Rubin, director of ecommerce at Mikasa, which retails at upscale department stores and its own 165 shops.

Although the company spent little on the site, it made even less. Having failed to achieve a return on its investment, Mikasa shut it down after nine months.

The company remained keen on an Internet sales channel, but decided to take a completely different approach. Last year it relaunched, using a customized version of the WebSphere Commerce Suite from IBM ( and hosting the site at IBM partner Web Emporium (, a Phoenix-based application service provider.

Once they decide to outsource, many companies are only too happy to abandon any meaningful examination of ROI, but Mikasa CFO Brenda Flores keeps close tabs on several metrics. “We’re measuring ROI in terms of customer loyalty, revenue commitment, and cost-cutting,” she says.

According to Sandy Carter, worldwide vice president for WebSphere marketing and channel execution at IBM, Mikasa’s insistence on a close analysis of return on investment is part of a trend. “More and more prospective etailers are coming to us insisting upon clearly defined ROI metrics before undertaking a B2C or B2B project,” she says. “In today’s economic downturn, they’re finding they need to be able to justify the business results.”

So far, the metrics match expectations. “The new site enhances productivity,” says Rubin. “Previously, we would receive an online order and then have to manually place the order into our system here, and then we’d have to run the credit card through. Now it goes straight from the Web server to our back-end ERP system, and the credit card is automatically authorized. The only time there is ever any manual intervention is when there are exceptions [for example, if there’s a problem with the credit card].”

Carter says IBM works with customers to create and manage structures for measuring the effectiveness of an Internet channel, based on its own experience building its channel, through which it sells billions annually. “When we launched the Web site in 1993, we established three key metrics to measure effectiveness — cost savings, shorter timeframes to market, and improved customer service and loyalty,” she explains.

Big Blue continues to measure the effectiveness of the strategy. “For example, we recently learned through research we conducted with J.C. Penney that if a retailer only has a bricks-and- mortar presence, the average customer would spend about $100 a year,” says Carter. “But if the same retailer had a multiple-channel presence — retail, etail, and catalog — the customer spends nine times as much.”

However, that doesn’t mean every brick-and-click business will have the same experience, as Mikasa learned through its earlier etail experience. The previous B2C site wasn’t integrated into the company’s ERP system, it lacked a search function, and its product menu was more snack than dinner. The site also didn’t convey the company’s high-end image. “Mikasa is all about elegance,” says Carter. “We wanted to capture that in the look and feel of the site so that customers could not discern the difference between an online experience and shopping at one of their stores.”

Mikasa measures revenues, customer loyalty, and cost-cutting monthly. While its initial etail effort flopped, it does provide a useful benchmark. For example, the company can compare how quickly customers can search for a particular item on its new site compared with how long it took to find an item on its old site, which lacked a search engine. “If your online catalog is not easy to search, you decrease customer loyalty to the site,” explains Carter. “The number one reason people leave a Web site is poor navigation.”

Erik Redmond, Mikasa’s Internet project manager, notes another customer benefit. “A customer who is in one or our 165 stores may want to purchase a particular pattern that the store may not have,” he explains. “Since each of our stores now has Web access, they can scan the digital catalog for the item and, if they prefer, buy it then and there.”

Costs are much more easily measured because, as an ASP, Web Emporium charges a monthly fee to host the system. “We liked the idea of having a company whose core competency is hosting a Web site do that for us,” says Rubin. “The strategy also provides certainty in terms of cost.”

Expenses have been further pared through productivity improvements. “Mikasa used to ‘sneakernet’ orders in the past — basically take them by hand,” says Carter. “By having the computer do everything, they save on their processing costs, order costs, and people costs. They’re also saving money by having all 20,000 products online now, relieving them of the need to print and distribute catalogs.”

Mikasa measures monthly revenues from the site, broken down into several categories. For example, it scrutinizes the number of items bought per pattern and whether or not the item is part of a gift registry. It also measures the number of buyers who are registered members. Mikasa then compares the results with similar metrics from its store purchases, looking for the patterns that spell success. — RB

KPMG Consulting

Ed Courtney knows a little something about information overload. As a managing director at Big Five consultancy KPMG Consulting (, Courtney knows that project managers often receive as many as 250 emails a day. As bad as that sounds, consider that the email system at KPMG holds only 200 emails per account, forcing the managers to delete dozens of emails just to be sure they see everything that’s coming in.

Email isn’t sexy, but it remains the killer app for office productivity. Technology research firm Gartner estimates that email messages account for 75 percent of the knowledge exchanged within and between companies. For firms in a host of services, from consulting to insurance to law to advertising, email is often the primary means by which project data is accumulated, shared, and stored.

“At any given time I have about a dozen projects going on, and I’m getting emails from managers and other employees who are on four or five of these projects, but not the same ones,” says Courtney. “Because we’re project focused here, and everyone is working on multiple activities, and they’re all mobile and they all use email for updates, I’m swamped with emails that I have to read.”

Last year, the top brass at the Tysons Corner, Virginia—based consultancy decided that something had to change. KPMG set out to not only manage its email better, but to turn that better management into a competitive advantage. Today, the 18-month effort has boosted productivity by 15 percent, and the company believes that ultimately a 40 percent boost is possible.

Combining software from Intraspect Software ( with its own knowledge management technology, KPMG now enables its employees to process their mail far more quickly than before. So quickly, in fact, that the firm says it has been able to pare consulting fees and thus win additional business. More business and lower fixed costs will drop directly to the company’s bottom line, Courtney says.

KPMG didn’t just need bigger in-boxes, it needed a system that could differentiate urgent from not-so-urgent messages, and could collate and store email pertaining to different projects, including relevant charts, photos, budgets, and other attachments. And it needed workflow capabilities so that if there were different versions of a document being emailed back and forth, everyone could discern which version was the most current.

The system also functions as a useful gatekeeper. “KPMG wanted a way for managers to be notified of something only when they needed to be notified,” says Robert Schoettle, vice president of marketing at Intraspect. The new system eliminates emails that “don’t contain value,” says Schoettle, while email that is deemed relevant includes “threads” connecting it to related messages, “so readers can discern the back-and-forth evolution of the issue or item being discussed.”

Most important, Courtney says, is that the system virtually eliminates the need for on-site status-and-delivery meetings. “We used to fly five or six dozen of our employees who were working on a project to some central location to discuss it with the client,” he notes. “Then we’d fly them home on weekends. In the meantime we’d rent office space for everyone to gather. This is not exactly inexpensive.”

Now those same employees gather and collaborate on the Web in the virtual workspace created for that project. They participate in discussions with project leaders, post and receive documents, add their thoughts to status reports, and so on. “The productivity improvements are enormous,” asserts Courtney.

How does he know this, other than using just plain common sense? “We’re measuring ROI two ways—how long it takes us to develop proposals for clients and how long it takes us to produce different deliverables,” he explains. “Each involves multiple people across multiple geographies. Since we know how long it took us to do these the old way, we can gauge the impact on productivity.”

There are soft benefits as well. “A new employee who is relatively inexperienced can ramp up rapidly by entering the virtual workspace and examining everything there is to know about an ongoing project,” says Courtney. “And we’re decreasing frustration levels by helping employees easily find what or whom they’re looking for. We’ve basically reinvented the way we collaborate.” —RB

Saab Cars U.S.A.

When Saab Cars U.S.A. ( was trying to determine which Internet users were ripe for the plucking, it found that the key was fermented grapes. Hoping to entice affluent shoppers into showrooms, Saab did some demographic research and found that its target market had a strong affinity for travel. “They’re also pretty adventurous, the kind of people who love to do outrageous things,” says Colin Price, interactive marketing manager at Norcross, Georgia—based Saab Cars U.S.A.

So Saab crafted an ad campaign that used the lure of a free vacation to California’s wine country, from Monterey to Napa Valley. A convertible Saab would serve as the primary means of transport, with a hot-air-balloon ride and some sea kayaking thrown in. “Basically, we made the sweepstakes virtually impossible for people in our target market to pass up,” says Price.

But how best to communicate the contest to that market of well- educated, highly paid 25- to 54-year-olds? Saab’s technology partner, Boston-based Digitas (, did its own research and discovered that two Web sites catered to this demographic —, a wine site, and, an upscale travel site. “We designed a banner ad that blended in with the look and feel of these sites, so viewers would click through thinking it was actually part of the site,” says Digitas chairman and CEO David Kenny.

Viewers who took the bait saw a flash movie in which a convertible Saab 9-5 cruises up the Pacific Coast Highway across a bridge and into Big Sur. The clip ends with a shot of a hot-air balloon ascending the deep blue sky. And then the hook: To participate, entrants would have to test-drive a Saab at their local dealership.

That is where the rubber meets the road for Saab, because the company has found that its best chance of competing with the likes of better-known Euro-sedan kings such as BMW and Mercedes is to get people behind the wheel of one of its models. The tag line “People who test- drive a Saab usually buy a Saab” isn’t just wishful thinking, but is borne out by market research (although Saab won’t quantify “usually”).

Therefore, any calculation of ROI for the Web advertising effort had to be designed around its success as a vehicle for getting people into showrooms and out on the road. Based on that metric, the effort was a stunning success. “The number of people who took test-drives based on the contest exceeded projected estimates by 300 percent,” says Ken Adams, CFO at Saab Cars U.S.A.

“Previously, we had established a benchmark in terms of the number of sales we wanted to generate through the ad campaign, then estimated the number of test-drive sign-ups consistent with that,” adds Adams. “Basically, we had an objective and worked backward from that.”

The winner of the contest, announced late last year, was a woman from Cincinnati. “The best part was that after the trip, she came home and bought a Saab,” says Price.

Saab won’t give actual figures for the number of entrants who ultimately bought a Saab (although the winner and others did), but Adams says that the company was keenly interested in quantifying the success of its marketing effort. “We did a monthly comparison of the database of names produced by the Internet sweepstakes against the databases of other advertising campaigns, including direct mail, dealership strategies, television advertising, and so on,” he says. “What we found out was that the Internet strategy was highly profitable given the investment, and sales exceeded our estimates. Most important, we reaffirmed that the test-drive is the critical part of the strategy.”

Digitas, which designed and built the technology around the advertising campaign, was also eager to measure results. “Saab wanted a high-ROI campaign that would be measurable across several metrics,” Kenny explains. “We measured demographic detail on the number of people who clicked through to enter the contest, then clicked through to the local dealership to set up a time and date for the test-drive, and then actually completed it.” Designing the contest page so that visitors could click through and communicate with their local dealer proved a major advantage, and tapped an aspect of Web advertising that other media would be hard-pressed to replicate.

On a cost-per-test-drive basis the advertising campaign proved so successful that Saab and Digitas recently launched another Internet sweepstakes, this time with the banner ads built around jazz music and featured on the New York Times and Washington Post Web sites. Says Price, “We discovered that our target market likes a little jazz with their wine, so we’re offering a weeklong trip to the Montreux Jazz Festival in Switzerland.” There are rumored to be some respectable wineries not too far west of there. An easy drive, in fact. — RB

4 Powerful Communication Strategies for Your Next Board Meeting