Back in the 1980s, American Airlines (AA) was partnered with British Airways (BA), and AA’s marketing head wanted to know how many of the airline’s gold card members, its most profitable customers, were flying BA rather than AA. Larry Tieman, then a managing director in IT at AA, dove into the massive amounts of data AA had collected on its customers and reported back that, basically, all of AA’s gold card holders were flying BA. The reason seemed obvious: BA’s service simply was better. So AA launched a huge quality-improvement initiative. It upgraded its food service and changed its flight schedules to increase on-time rates. It invested heavily to compete on quality on all fronts.
The results, Tieman recalls, were disastrous.
“None of it mattered,” says Tieman, who between 2000 and 2010 was a senior technology vice president at FedEx. “The quality of food meant nothing. What was sticky was frequent flyer loyalty programs.
“With Big Data,” Tieman says, “you may be spot-on about a problem, but the solution doesn’t magically appear out of the data.”
Big Getting Bigger
According to a recent Gartner report, “enterprise data . . . is expected to grow by 650% in the next five years.” That’s Big. A June IDC study found that the world’s data is doubling every two years, and that businesses will “manage 50 times more data, and files will grow 75 times more in the next decade.” That’s Big Data.
Where’s all this data coming from? It’s coming from ubiquitous sensing devices such as RFID (radio frequency identification) readers, wireless networks (all those smart phones), social networks (the “like” buttons on Facebook; Twitter feeds; mobile location services such as Foursquare), point-of-sale systems, and on and on.
“CFOs have a gut sense that there’s money out there in all that data,” says Forrester Research principal analyst Brian Hopkins. “The challenge is how to turn that data into new opportunities.” The good news, Hopkins says, is that new technologies are making it more economical to make sense of Big Data which, in fact, has been around for a long, long time. The caveat is that those technologies will not provide those opportunities. That’s still up to the people who make business decisions. Relying on data alone could lead a company down the path AA took: investing heavily in the wrong things.
“There’s no magic; no presto box from IBM that you can throw petabytes into and get answers. It just doesn’t exist,” Hopkins concludes.
The Three ‘V’s of Big Data
The sheer volume of all this data is just one of the three “V’s” that technology must address. There’s also the variety problem: according to Gartner, 80% of that 650% growth in data will be unstructured, meaning it will exist in formats your current computing systems can’t use or access, as it’s buried in call center conversations, e-mails, notes, and Word files. In other words, you can’t capture it on a spreadsheet. Then there’s the velocity problem: the data is coming at you hard and fast; it changes quickly and therefore requires speedy analysis and decision making.
Traditional IT dealt with the volume problem through data warehousing provided by large software firms such as Oracle (which was expensive). You could then query the database though programs that extracted the data, transformed it into a usable format (addressing variety), and loaded it (a process known as ETL) for analysis by still other tools. All that took time, clearly failed to address the velocity problem and, as a side effect, increased costs as well as the business’s impatience with IT.
A recent McKinsey report on Big Data suggests that “organizations will have to deploy new technologies (e.g., storage, computing, and analytical software)” in order to “capture value from Big Data.” Those new technologies are evolving right now. The most famous Big Data tool is probably the open-source database management tool Hadoop (named after a toy elephant belonging to its creator’s child), which breaks down huge data sets into small segments that can be run by applications more quickly and cheaply in virtualized (cloud) database environments. And once the data is accessible, it can be run through analytic tools designed to handle Big Data, provided by Appistry, Cloudera, LexisNexis, or Teradata, among others.
The Value Proposition
Not surprisingly, the financial-services sector is a big adopter of Big Data, using it for predictive modeling, according to KPMG principal Thomas P. Keegan. “Financial services compete on analytics,” Keegan says. “For example, they’ve all got bonds on sovereign debt. If a company can see that Ireland is right behind Greece, they’ve saved money.
“Netflix,” continues Keegan, “uses crowdsourcing and predictive analytics to come up with better recommendations for what movies we might want. Heritage Healthcare is trying to identify people who come back to the emergency room every six months to get them preventative health care, thereby lowering costs as well as improving patient care.”
The potential of Big Data seems limitless.
“At FedEx,” says Larry Tieman, “we used to collect data only around the shipment, pickup, and delivery. Now we pick up every bar code scan on a package hundreds of times. We take all that data and our local stations analyze it for route efficiency. Using that data, we might move a plane from one gate in Memphis to another to get it to New York faster.
“Ten years ago, we couldn’t tell salesmen how much a customer was shipping until a week after the package was shipped. We’d sign up a customer, and he’d never ship. Now, if he signed a contract and he’s not shipping, we start pinging his BlackBerry. ‘You guys were supposed to ship the first of the month. What’s wrong? Can I help?’”
It all sounds great, but . . . .
The Human Factor
Big Data, says Forrester’s Hopkins, “is not about the CFO writing a check to the CIO, closing his eyes, and hoping he gets some value back.
“Business needs to take ownership of the data. Don’t put your data scientists in IT. Put them in marketing; put them in fraud or actuarial. And then, when you have a sense of how much revenue is out there, use that to justify appropriate investment.”
Or, as Tieman says, “Tools are available to anybody and you can buy what you can afford to, but what you do with it is a people-based activity, a skill base you have to mature. And it doesn’t come quickly.”
It’s also true that there’s a limit to the amount of information people can absorb. As Jonathan Byrnes, author of the book Island of Profit in a Sea of Red Ink, wrote in a recent Harvard Business Review article, with large volumes of information comes the risk of “flooding your managers and sales reps with so much information they will not be able to act effectively. They will lose the critical understanding of the smallest number of realistic actions that will produce the most powerful set of results.”
And it’s not just managers and sales reps at risk of information overload, or analysis paralysis. It’s CFOs. So it’s essential to know what you want to get from Big Data before losing yourself in it.
“Nailing down an ROI for Big Data is very difficult,” says Tieman. “The benefits are never clear-cut. So the CFO hates it, and depending upon how cranky he is that day, he can be mean or really, really mean about it. So you can’t go into Big Data as an IT project.
“It’s a business transformation project.”