On the Internet, the shortest distance between two points is a well-conceived search query. In fact, these days most “surfing” entails a hurried perusal of the first few sites returned by a search engine as you quickly try to find exactly what you’re looking for. Search is the glue that holds the Web together, and while the media hype surrounding Google’s initial public offering might make you think that the technology is fully baked and capitalized upon, in fact things are just beginning to get interesting.
A multi-billion-dollar industry has grown up around search technology. Competition is fierce as household-name companies battle for supremacy on the high end even as a slew of smart, scrappy start-ups vie to become the next Google or, failing that, innovate their way to niche success. The technology is changing and penetrating corporate life in ways that might surprise you. It’s transforming the way companies — including yours, most likely — advertise and market their brands, products, and services. If that’s not enough, search also promises to change the way companies create, store, and share information, both on individual PCs and over corporate networks.
What seems like a simple utility is actually a multifaceted IT issue worthy of inclusion in corporate strategy. Below you’ll find five solid hits that will get you up to speed on the world of search.
They’re Feeling Lucky (And Smart)
From its humble beginnings 14 years ago, when a McGill University student named Alan Emtage invented the first search tool, Internet search has grown into a bona fide industry: market watchers peg the industry’s revenue at about $4 billion. Safa Rashtchy of securities firm Piper Jaffray & Co. expects search-related revenue worldwide to top $11 billion by 2008.
Not one penny of that comes directly from the millions of searches that you and everyone else on the Web conduct each day. Basically, Web search is the content that delivers an audience to advertisers in various and increasingly sophisticated ways.
Most search today is offered in one of two ways: either through stripped-down, search-only sites such as Google that use what’s called algorithmic search (complex formulas to determine the “best” results for any given keywords) or through content sites or portals, including Microsoft’s MSN and Yahoo, many of which also use algorithmic techniques and/or indexing of sites by category.
Google, of course, has proven such a success that its name is a verb. Currently generating almost $1 billion in revenue, with net income of $106 million, the company employs more than 1,900 people (known as Googlers), including two massage therapists and a former neurosurgeon (now an operations manager). Used by 46 million people a month, its engine searches some 6 billion items, including nearly 4.3 billion Web pages, 880 million images, 845 million Usenet messages, and 1,000 mail-order catalogs.
But Google is hardly resting on its laurels, and its rash of recent innovations gives some indication of how much work remains to be done in shortening the distance between searchers and what they search for. New services include Froogle, a comparison-shopping search engine; Search by Number, which can be used to track packages, search for products and patents by UPC codes and patent numbers, or find area codes; and Google Local, which finds stores and services in a given zip code. The company is also rumored to be developing software that will search information stored on PCs.
Turf Guys
Google’s success is being challenged by other large companies, mainly Yahoo and Microsoft, and to a lesser extent Ask Jeeves, AOL, and Amazon. And then there are the smaller companies.
Unlike Google, neither Microsoft nor Yahoo is primarily a search company. Microsoft, of course, is the leading maker of PC software, with a virtual lock on the PC operating-system and office-applications markets, while Yahoo operates the leading Web portal, attracting nearly 275 million visitors a month.
To catch up and compete with industry-leading Google, Microsoft and Yahoo have relied on a combination of acquisitions and in-house development. Yahoo’s buying spree includes Inktomi (one of the original search pioneers), AltaVista (another early search engine), Overture (a leader in commercial search services), and Kelkoo (a European comparison-shopping engine). More pointedly, earlier this year Yahoo severed its ties with Google, which had supplied the actual search-engine component on Yahoo’s site, and instead rolled out its own search algorithm.
Microsoft, meanwhile, has signaled its seriousness by creating a vice president of search position and developing a line of search-related products and services incorporating its own algorithmic search engine. Among those now being tested are automated services to capture news (tentatively to be called MSN Newsbot), Web logs (tentatively MSN Blogbot), and natural-language queries and answers (tentatively MSN Answerbot). “Consumers say that 50 percent of their search results are still not relevant,” says Karen Redetzki, an MSN product manager, “so we see a huge opportunity.”
Amazon.com, though not primarily a search company, has built its powerful E-commerce business around proprietary, and highly sophisticated, search functionality that helps shoppers find what they want fast. Its latest feature, called Search Inside the Book, lets visitors use a search engine to peer inside the pages of more than 100,000 books. Amazon is also working to compete in two new areas: licensing and comparison shopping. In late 2003 it announced A9, a separately run subsidiary that will develop E-commerce search tools for use by both Amazon and licensees; the company launched its beta site in April. And a year ago, it formed Amazon Services to license its search technology to other E-commerce sites. Customers include Target, Toys “R” Us, and the National Basketball Association’s online store.
But Exhibit A in the case that search is still very much an evolving market may well be Ask Jeeves. The company was founded in 1996, nearly disappeared during the dot-com bust, and was revived in 2001 thanks to, oddly enough, the acquisition of other search-related companies and the sale of part of its business to, you guessed it, a search company (of sorts, see below). With 28 million visitors monthly, Ask Jeeves is not putting the scare in Google just yet. But its CEO, Steve Berkowitz, sees an opportunity to compete via technological differentiation. Unlike other search sites, Ask Jeeves allows visitors to type in natural-language queries — for example, “What’s the weather in San Francisco?” — in addition to keyword queries. Citing surveys that show the average Web visitor uses 2.5 search engines a month, Berkowitz says, “Our opportunity is to become the only viable alternative to the others.”
Thinking Outside the Search Box
Search continues to be an engine of growth, and start-ups are plentiful. Many have real customers, real sales, and real working capital. Nearly all are bursting with clever ideas, products, and services.
Among these new-generation search vendors, none entertain fantasies of competing with Google, MSN, or Yahoo. Instead, they’re focusing on how to make search work better and how to work better with search.
One innovator, Kanisa Inc., helps companies’ customers find useful information within corporate databases. Kanisa actually predates Google by a year, and its original search software product is still used by customers, including some 25,000 Microsoft service agents. The company has been rejuvenated in part by last year’s acquisition of search technology from Ask Jeeves. Today, customers including Ford, Nike, ADP, and Nestle Purina PetCare incorporate Kanisa’s technology into their Websites.
Purina, the pet food and supplies company’s U.S. operation, powers its “Ask Purina” feature with Kanisa’s natural-language query and search technology on all its customer-facing sites. It also offers the tool on the Pets.Yahoo.com site through a partnership with Yahoo. Visitors can type in full questions — such as “What’s the best food for a two-month-old kitten?” — and the results will comprise a series of relevant links and summaries. While that’s great for customers, the real benefit from Purina’s perspective is that such self-service search tools reduce phone traffic to its call center. “If you look at the number of queries you get times the cost to handle a phone call, the ROI is huge,” says Brad Worth, a brand manager in Purina’s interactive group.
Other search companies hope to rise high by diving deep. BrightPlanet Corp., for example, says Google’s universe of more than 4 billion Web pages, while impressive, only scratches the surface. What company founders call the “deep Web” contains, they say, much, much more. In fact, BrightPlanet COO Duncan Witte says just the top 60 deep Websites (including the Library of Congress Online Catalog, eBay.com, the National Climatic Data Center site, the SEC’s Edgar, MP3.com, and Amazon.com) collectively possess some 84 billion documents — many of them in formats other than the simple HTML pages that today’s search engines traffic in. BrightPlanet’s products, which are jointly marketed by IBM and other consultants, promise to help information analysts plumb these overlooked sources.
A long list of other search innovators are exploring other aspects. Groxis converts lists of search results into graphical maps. Black Tulip Systems Corp. has licensed technology from NASA that searches through an enterprisewide network. Mamma.com offers “metasearch” software that essentially searches using multiple search engines, then consolidates the results into a single report.
What many of these companies have going for them, in addition to cool names, is strong intellectual capital, often involving patents. Major investors are another advantage; search-ad specialist Quigo Technologies, for example, recently landed an investment led by Bob Davis, a former Lycos CEO. Industry expertise doesn’t hurt, either: version 2.0 of the Grokker software offered by Groxis was developed with help from John Seely Brown, a former director of Xerox PARC, the famed Palo Alto Research Center.
Search Is Transforming Marketing and Advertising
For advertising and marketing, Internet search is doing nothing less than changing the rules. By letting companies place ads on relevant search-results pages, search companies are transforming advertising from intrusive to inclusive. Sitting through a TV ad for a mountain-crushing SUV while watching “The Daily Show with Jon Stewart” is intrusive, but reading a Web ad for that very same vehicle, this time because you’ve searched for “mountain-crushing SUV,” is less so. “This is the only medium in which advertisers and users have the same reason for being there,” says Berkowitz of Ask Jeeves. That may be optimistic — some of us just want to check the box scores without being assailed by ads — but it is true that search engines can identify active shoppers in a way that TV can’t.
According to a survey sponsored by the Interactive Advertising Bureau and conducted by PricewaterhouseCoopers, that has translated into a big business today: keyword-related ads account for 35 percent of all online advertising (the rest is mostly banners, sponsorships, and classifieds). Total online ad spending in 2003 hit $7.3 billion, the IAB says, putting search ads’ share at about $2.5 billion.
One satisfied user of Google’s AdWords program is Harrisdirect LLC, a Bank of Montreal unit that provides investor services. The company bids against competing advertisers for some 1,000 keywords, says Eric Frenchman, Harrisdirect’s marketing director, vying for top billing in “sponsored links” lists that Google generates for those search words. It works like this: Harrisdirect gives Google a list of search terms it wants to bid for, along with the maximum bid it is willing to make for each term. For example, it might tell Google that it wants to display an ad on results pages for the search term “invest” and that it’s willing to bid as much as $100. Google would then look at the bids from all companies that want to advertise on the word and rank them according to their final bid, with the highest bidder at the top of the list. But that’s only half the equation: Google also factors in an ad’s actual click-through rate, the percentage of times users who draw the ad on a search-results page actually click on it. “It’s a complicated process, one that requires us to analyze which search terms are most relevant,” says Frenchman. “We use real-time reporting to track people as they click on a keyword to a ‘landing page’ and then to an online application.”
Search Is Changing How Your Company Works
Search technology is not only changing the Web and online advertising, it’s also changing the way companies store, organize, and share information. There are two new frontiers for search: the PC on your desktop, and the servers and networks within your organization.
On the PC front, Microsoft is reportedly developing search-related functions as part of Longhorn, the next major release of its Windows operating system. Industry watchers say Longhorn will enhance the current Windows file structure with a searchable database. If they’re right — and Microsoft CEO Steve Ballmer recently told online advertisers that his company had erred by not putting more effort into search technology — then users of Windows PCs will have an entirely new way of storing, searching, and retrieving files. One expected feature is a search box on the Windows desktop that will reportedly activate a minisearch engine that will prioritize data on the PC according to the context of a document or file.
Part of the challenge is designing a search engine that can explore different kinds of files, not just those of a single application such as Word or Outlook. One small company, Filehand LLC, has created a utility that helps users search their PC for information in PDF, Word, PowerPoint, Excel, WordPerfect, HTML, text, and even MP3 files, with other formats soon to be added.
A related challenge is to be able to search across all the computers within a company, peering into the structured data found in databases and the unstructured data found in spreadsheets, reports, presentations, E-mail, and the like. Verity Inc., a 16-year veteran of the computer industry, has refocused much of its business on what it calls intellectual capital management. In the past, explains Andy Feit, Verity’s senior vice president of marketing, to get an answer to a relatively simple question — “What is the history of our relationship with Acme Corp.?” for example — a user might have to search 5, 10, or even more separate applications and databases. Much of the information may not have even been published on any company site, but instead stored on a private PC or department server. But with Verity’s Federator product, which searches an enterprise network much as Google does the Web, the question could be answered with a single search.
Raytheon Co. found it needed to first hire Verity to analyze its entire data network, essentially telling Raytheon what it had. The company now uses Verity K2 software to provide enterprise search services and Verity’s Data Discovery Program to organize intranet data. The effort is part of the larger Information & Knowledge Management Plan that Raytheon is rolling out this year to its 78,000 employees worldwide. Says Keith Cromack, Raytheon’s director of information services: “Unless you can manage the content better, label it, and give it some context, finding it won’t do you much good.”
Given the hodgepodge of computer systems at use in most companies, however — not to mention an unrelenting information glut — many would settle for simply finding it. Search engines are likely to remain virtually synonymous with Internet use for many years to come.
Peter Krass is president of Petros Consulting and a freelance writer and editor in Brooklyn, New York. He was formerly a senior editor at Inc. magazine, editor-in-chief at Planet IT, and features editor at InformationWeek.
Thinking Small May Pay Off
A potentially huge and largely untapped market is search-related ads from small businesses. There are literally millions of small companies in the United States, and Internet analysts say few of them use such ads. A surprisingly large number don’t even have Websites, relying instead on yellow-pages listings, word of mouth, and local advertising, a business estimated at $18 billion a year. New search features such as personalization and localization could change that — and create a huge new business. Google, Yahoo, and others are hard at work in this space, as are specialized companies. Four-year-old Quigo Technologies has developed artificial-intelligence software that “reverse matches” searches to better understand how people actually find what they’re looking for, helping companies develop better advertising strategies.
Safa Rashtchy of Piper Jaffray & Co. estimates that the number of Internet searches being conducted is rising by 10 percent to 20 percent a year. He believes more than 200,000 companies worldwide have tried online ads, in large part because the cost per sales lead is so much lower online than it is in the “real” world: 45 cents per lead for search-related ads, compared with 55 cents for E-mail, $1.18 for yellow-pages listings, $2 for banner ads, and $9.94 for direct mail. “I would suggest that probably no more than 20 percent of Fortune 500 companies are using search in this way, so there is room for potential growth,” he adds.
Last month Yahoo said it would pursue local advertisers in two ways: by targeting key retailers and other big companies, and through “locator pages” that will direct searchers to small businesses. —P.K.