Because it’s nearly impossible to draw precise boundary lines around technology categories, companies that sell software are often free to ride the wave of whatever three-letter acronym happens to be hot at the moment. As a result, a panoply of software products continues to crowd into the CRM (customer relationship management) market, which is projected to reach $3 billion this year. Many of these products are only indirectly related to CRM’s ostensible goal of getting customers to buy more and stay loyal, and many hail from no-longer-trendy trendy niches, and now seek shelter under the CRM umbrella. Differentiating among all those huddled bodies can be a buyer’s nightmare.

A recent series of surveys focusing on various CRM products by research firm Gartner, for example, included software to manage partner relationships (PRM) and employee bonuses, two technology categories that once stood proudly on their own. Meanwhile, companies that make CRM software are expanding their offerings in all directions. SalesForce.com, for example, made its name with sales-force automation but now boasts that its new online application development tool, sforce, will let information-technology staffs build complementary non-CRM software, including project, document, and asset-management systems.

Complicating buying decisions even further is the fact that many companies overbought in the past. Gartner found that 42 percent of CRM software purchased ended up as unused “shelfware,” compared with the standard 20 percent rate across other types of software, often thanks to the lure of deep volume discounts.

“Rather than looking at CRM technology as a single thing,” says Gartner research vice president Beth Eisenfeld, “the question enterprises should be asking is, ‘Which of the many CRM applications should we be implementing?'” Eisenfeld headed up Gartner’s recent research into the benefits of 14 different types of CRM products, asking companies to rank the tools by their usefulness toward four goals: cost savings, efficiency, effectiveness, and competitive advantage.

But those rankings provide few easy answers about what to buy. A sales-force automation tool, for example, probably won’t help lower costs or provide a competitive advantage, Eisenfeld points out, “but it’s like the foundation of a house — you need it in order to open and shut doors.” Meanwhile, E-service technologies like E-mail routing can save money, but may actually harm the business if customers bolt when they can’t get a live person to help them.

Analysts say that the effectiveness of specific categories of CRM depends on a company’s business model. “If you sell primarily through a distribution channel, then PRM [partner relationship management] becomes important, while if you sell via the Web, E-commerce tools will be more important,” says Denis Pombriant, vice president of CRM research at Aberdeen Group. “But I wouldn’t say that sales tools work better for the sales team than marketing tools do for marketing staff. It really just depends on what you need.”

Pombriant recently tested user satisfaction with three broad categories of CRM tools: sales, marketing, and customer service. CRM customers considered productivity the earliest benefit of sales tools, cost control the top boon from customer-service tools, and revenue generation the main benefit for marketing tools. Interestingly, though, the mean scores for each fell within a tight band, ranging from 2.4 for marketing to 2.7 for sales on a scale of 1 to 5.

Pombriant interprets the results to mean, generally, that CRM buyers are becoming more discerning. “We’ve certainly been through the early-adopter phase in the industry, and I think companies are now really understanding what their needs are,” he says.

Of course, getting a fix on exactly how much benefit accrues from any product is tricky, since few buyers are actually measuring returns in dollar figures. “When I ask our clients if they are achieving ROI, at least 80 percent say yes,” says Eisenfeld. “Then I dive down and get all these wishy-washy answers.”

More than half of the 653 companies Gartner surveyed in late 2002 claimed to have gotten a return on their CRM investments, but Eisenfeld also takes those claims with a grain of salt; most companies focused exclusively on benefits such as head-count reductions or improved order-processing times but didn’t quantify those in monetary terms and then subtract the costs from the benefits to calculate ROI. “Although respondents in our survey claim satisfactory ROI, we estimate approximately 13 percent of them are actually able to cite the ROI they claim,” she says, “whereas the rest can only anecdotally claim benefits.”

Why are companies so reluctant to track ROI on their CRM investments?

Some say that the amounts spent on targeted projects just aren’t worth the time spent on post-purchase calculations. Morningstar Inc., which sells stock research, software, and consulting services to mutual funds and big companies with 401(k) plans, spent about $200,000 on Siebel Systems Inc.’s sales-force automation system 18 months ago to collect and centralize its sales data. “Our costs were so small, I don’t even need to track results to know we’re getting returns,” says Morningstar COO Tao Huang, who led the project.

The product has helped the company take a global view of its relationships with multinational customers, he says, allowing it to win additional accounts within those institutions and cross-sell them on other products. Those benefits, along with the others that have accrued from having more accurate, up-to-date sales data in one place, he says, have increased the company’s revenues for the unit using CRM by more than 100 percent since implementation.

An emerging area of CRM dubbed customer analytics may further defy ROI analysis. “The analytical pieces are far lower in costs than the operational ones [like contact-management systems],” says Tony LoFrumento, executive director for CRM at Morgan Stanley. “But if you use such software correctly, it affects every decision senior management makes, becoming almost an allocation methodology for how much time and money to spend on each customer.”

At Morgan Stanley, LoFrumento uses CRM tools from SAS Institute Inc. to calculate each customer’s profitability and assign service levels accordingly, since “no company today can afford to give red-carpet treatment to everyone.” He says the tools have already paid for themselves, based on the service pricing changes the company has implemented and the success of its more-targeted cross-selling efforts.

His next step, though, is to figure out customers’ future profitability, based on their age, assets, and even investing style. Determining such so-called lifetime values of its customers will help Morgan Stanley determine how much to spend on marketing and retention programs. That puts returns on the technology investment out past the tenures of most executives, and is heavily contingent upon what doesn’t happen (customers leaving), two factors that are not particularly conducive to traditional measurements.

ROI is similarly elusive when the software in question supports forecasts. Office-supplies giant Staples Inc. adopted SAS’s analytics software about seven years ago to handle weekly sales forecasts (which now come within one percent of actual sales, on average), and now uses the tool to make decisions about where to locate new stores, how to time marketing efforts, and even which products to place most prominently in stores.

“It’s hard to quantify all the good things that come from a good recast,” says Alan Gordon, director of sales forecasting for Staples. The company saw a 13 percent increase in first-quarter sales and is predicting a pro-forma 20 percent growth in earnings this year, but tracking those revenues back to the CRM tool fell by the wayside long ago, Gordon says.

Is it wise to be so cavalier about such spending? Gartner’s Eisenfeld says no. “There is no question in my mind that people are getting benefits with CRM,” she says. “But the question is, is the benefit worth the cost?” Eisenfeld believes companies should translate the efficiency and effectiveness gains they expect to get from CRM into potential revenues, and then continue to track returns over the life of a technology. With most annual CRM-maintenance fees ranging from 15 to 23 percent of the initial cost of the software, it may pay to switch.

Aberdeen’s Pombriant, however, sees little danger in the lack of rigor. “Who has the time and resources to constantly come back and measure?” he says. “What the marketplace is telling me is that they look at the benefits and impute ROI.”

In fact, in Aberdeen’s recent independent survey of 325 Siebel customers, analysts found that the difference in reported ROI was not statistically significant between those companies that measured ROI and those that estimated it. But that study came on the heels of a report from Nucleus Research, of Wellesley, Massachusetts, that found that more than half of the Siebel customers it surveyed (a very small base of 23 companies) believed they had not made back their investments after an average of two years.

One problem, of course, is that while CRM was initially sold on its revenue-boosting capacities, the weak economy made such gains difficult to come by even if the software adds value. As CRM continues to morph, its ability to drive better decisions and reduce costs now gets more play. That doesn’t ease ROI calculations, but it may help customers have more cordial relationships with the vendors that have sold them on the promise of CRM.