Crowd Control

Motorola uses a prediction-market variant to determine which among thousands of employee-submitted ideas merit a further look.
David McCannFebruary 10, 2010

In 2003 Motorola rolled out a system through which employees could propose ideas for products or anything else that might boost the company’s value. By one measure it was an unqualified success: it produced 10,000 ideas over the next four years.

But the volume of submissions to the system, called Think Tank, overwhelmed the review boards that were set up to vet the ideas. A backlog swelled, and missed opportunities abounded. In one case, a competitor brought out a product with features that had been suggested by a Motorola employee years earlier, according to Rami Levy, a technologist with the company’s mobile devices business. “There was no effective way to pick out the right ideas, no matter how many people we put on the boards,” he says.

One idea that did attract attention was to solve the idea-overload problem by launching a variant of a prediction market, in which participants bet, like stock traders, on the likelihood of certain events happening. The basic idea behind prediction markets is that the “wisdom of crowds” tends to enhance forecasting accuracy.

4 Powerful Communication Strategies for Your Next Board Meeting

4 Powerful Communication Strategies for Your Next Board Meeting

This whitepaper outlines four powerful strategies to amplify board meeting conversations during a time of economic volatility. 

The trading system Motorola subsequently deployed in late 2007 is more accurately described as a decision market, because its purpose is to help the company decide what ideas merit high-priority treatment. Other large companies, including General Electric, Qualcomm, and BestBuy, have likewise employed the concept in recent years. Levy presented a case study of Motorola’s implementation and results at last week’s 2010 Corporate Performance Management Conference in New York, hosted by CFO.

Called TIX, or Think Tank Idea Exchange, the system relies in part on a voting mechanism Motorola had adopted in 2005 when trying to get control over the proliferating ideas. Employees simply signify with a “yes” vote the ideas they believe should be pursued. Ideas that receive at least five votes are eligible for TIX.

All employees may participate in the decision market, and in fact Levy says most ideas come from lower-level workers. Participation is far-flung, with traders coming from 120 sites worldwide and ideas from 80 sites. Most ideas relate to new products or enhancements, but anything is fair game; even ideas for acquisitions and other enterprise strategies have been submitted. There is a low barrier to entry, with no requirement for a business case or plan to accompany an idea. On the other hand, those who propose ideas can’t hide from scrutiny: anonymous submissions are not allowed.

Each week for the first two months of a three-month cycle, the company loads new ideas into the system, where they remain for 30 days. During that time, employees, who are staked to 100,000 imaginary dollars, can trade in the ideas as if they were stocks. An initial offering price is set for each stock, and then, as with real securities, the price rises or falls depending on trading activity. Players can even short stocks that represent what they think are bad ideas.

At the end of a stock’s market period, Motorola deems it a “winner” or “loser” based on a matrix of criteria: the stock’s volume-weighted average price (VWAP) over the last 15 days of trading, the number of shares outstanding, how many people traded the stock, and the average daily trading volume. Winners (there have been about 120 to date) are given top priority with the review boards.

In designing TIX, Motorola worked with other companies that had previously launched decision markets. (As with other relatively nascent fields, such as open-source technologies, support communities have sprung up to share lessons learned.) Potential issues were identified and addressed beforehand. “You can’t just put this market out there and throw the switch on it,” says Levy. “Like any market, there are problems — and there’s no SEC here.”

For example, VWAP was chosen as a metric instead of closing price. That’s because if players know when the trading period of a low-priced stock is about to end, they may jump out of it in favor of something else that seems to have a better chance of winning. In turn, that would skew the market away from the company’s goal for people to invest, and stay invested, in ideas they really believe in, Levy says.

Levy calls another potential snafu “the millionaire problem”: that is, successful investors could use their bankrolls to disproportionately affect the market. To mitigate that, the company keeps initial offering prices low; resets the market every three months, with all-fresh stocks and everyone starting over with $100,000; and limits the amount of money an investor can put into a single stock.

Then there is the “pessimists win” problem. Allowing investors to short stocks provides an appropriate penalty for bad ideas, but since players know that most stocks — about 85% — are going to be losers, they could run up their bankrolls by simply shorting everything. So extra dollars are awarded for holding stocks, and for holding those that win.

All of this raises a question: What motivates Motorola’s employees to participate? So far, they’ve had to be content with getting recognition for winning ideas or doing well in the market, knowing they are enhancing the company’s value. But the current average of 20 traders per day is well below the peaks seen shortly after TIX was launched and during the real stock market’s turmoil in late 2008. So, says Levy, the company is looking into providing some tangible rewards.

But even the current level of trading strongly supports the company’s goals. There is now a well-filtered list of ideas for the review boards to address, Levy says. A quarter of the reviewed ideas are now pursued, up from 10% before TIX. And the average number of days for a decision to be reached on a new idea has plummeted from 180 to less than 80.

Those improvements have propped up the bottom line, according to Levy. He says the combined revenue from product-based ideas and cost savings from internal innovations is “conservatively” 5 to 10 times TIX administration costs, which largely involve two to three dedicated employees. The cost to purchase and implement prediction-market software — called Foresight Server, from Consensus Point — was “under $100,000,” he says.

But additional, softer benefits were key goals for the program, too. These have been realized through collaboration forums that allow employees to see and comment on others’ ideas, which are thus improved by the crowd’s input. The forums facilitate people from disparate regions and company organizations forming relationships, working together on ideas, and avoiding duplication of effort, Levy says. Motorola actually introduced the forums in 2005 along with the voting mechanism, but participation spiked after TIX was introduced and continues to rise.

The bottom line, says Levy: “TIX has proved to be an excellent conduit for enabling collaborative innovation and creating new value for Motorola in a fun and enjoyable way that encourages participation at a minimal cost.”