The Trilogy

• The first question - What is an alternative search engine?
• The central question - How are the Alt Search Engines doing?
• The last question - Can Google be defeated?

The First Question - What is an Alternative Search Engine?

Before we can talk about an alternative search engine, it's necessary to address the question of what a search engine is. As simple as that seems, we at AltSearchEngines.com have discovered that there is no consensus on a definition. That being said, everyone knows what Google is, and that Google is a search engine. How so? Because it crawls the Internet, uses that information to build an index, and then retrieves information from that index when a user asks a question, or query.

Less clear is the situation where the index is not a result of fetching information from the Web, but rather is a closed system, a database of some kind. In these situations there is no crawler, just the query and the resulting matches (if any). For example, one "search engine" is Abbreviations.com.

Abbreviations.com is not sending out a bot to crawl the WWW looking for and picking up new abbreviations and their meanings, its index has been compiled; it is a database. When I ask it what "WYSIWYG" means, does it search for an answer and return it for me? Yes. Is it a "search engine?" Not in the fullest sense of the term, perhaps, but in effect, yes.

Having made that distinction, we must now put it aside. When I use the phrase "Alternative Search Engine" at AltSearchEngines.com, I specifically mean an alternative to Google. What then? Are we saying that Google is the only major search engine out there? Yes. Perhaps not statistically, or globally, but practically speaking it is.

Finally we arrive at our destination, or rather, definition. An "Alternative Search Engine," when used in my essays means that the search engine, broadly defined, exhibits at least one feature that is superior to a similar search on Google. "Alternative" in this case does not mean "replacement." Another way to look at it is this: if Google lets you down, there is almost assuredly an alternative search engine that would delight you.

This has important implications. More than anything else, it means that we do not consider any one alternative search engine to be a match for Google. In the incessant drumbeat of the mainstream media, this refers to the elusive "Google Killer." A search engine that on its own will reduce Google's market share as it takes the search world by storm, welcoming millions of former Googlers who have now jumped ship to this new champion. The unthinkable becomes thinkable - a better search engine than Google! Hasta la vista, Google.

Let me be clear, the defining characteristic of the Alternative Search Engines is that they do one facet of a search better than Google, just one. For example, if you wanted to search for the specific salary of a specific position at a specific company, you should go to GlassDoor.com because it is designed to do just that. I doubt that Google could match it.

Many searchers have begun to notice that Google does not perform equally well for all types of inquiries. In fact, for some it does quite poorly, returning page after page of links without providing any value or satisfying the user. Those who keep plugging away like a gambler at a slot machine often complain of "search fatigue." That is, they just give up on using Google for finding what they need.

In my next essay we are going to pick things up right here. How many of the Alternative Search Engines are there? What exactly do they do? And how come you've never heard of them?

By: John Battelle

Of all the jobs I've had in the past twenty odd years, I'm pretty sure the one that pleased my parents the most was my brief stint as a reporter at the Los Angeles Times, my family's local paper of record. My mother in particular seemed quite proud to see her son's work land on her breakfast table each morning (well, not exactly, I worked at a local edition in Ventura, and my mother lived in Pasadena. Every every once in a while, one of my pieces did make the national edition...).

In any case, when I delivered the news I was leaving the Times to help start a magazine focused on technology, Mom wasn't entirely convinced. "The newspaper," she declared, "is our social glue. It's what keeps us all on the same page. Technology is going to destroy that, everyone will end up reading whatever suits their fancy."

That was back in 1992 (the magazine was Wired), and given what's happened with the newspaper industry in the past 16 or so years, it's hard to argue with either my decision to leave the Times, or my mother's assertion that our culture was on the brink of losing an important component of its "social glue."

Fast forward to two nights ago, when I was talking with my own son about technology. Earlier in the day, he had emailed me from his room downstairs, asking me - for the fourth time - for the link to his Boy Scout troop web page. Mildly irritated, I turned to my browser's toolbar and entered "CA Troop 43" or some such, but the results did not give me the page I wanted. In fact, they gave me a bunch of other Troops - Troop 9, Troop 777 - that happened to have the numbers 4 or 3 on their site for other reasons.

I recalled that my son's troop used an old community application that was difficult for search engines to find, putting it in what search experts call "the deep web" - websites that are easily accessible if you know the exact URL, but near impossible to find using Google.

If I had enabled Google Desktop Search, I could have found that scout website - desktop search scans my entire hard drive and integrates it with my web search, and it turns out, I had the URL in an email folder on my local computer. But desktop search, like Web History, kind of creeps me out. I remembered that I had the URL in an email, searched my mail, and found it that way. Instead of sending it - yet again - to my son, I thought I'd talk to him at dinner and perhaps teach him the value of bookmarks.

This story is getting a bit convoluted, but stay with me. What happened next was interesting. At dinner I gently chided my son for lazy information gathering habits. "I'm not your personal Google," I told him. "Why couldn't you find the URL yourself? Did you even try?"

"Actually dad, I did, and it gave me all sorts of wrong answers - Troop 43 in Texas, and Iowa, but nothing for mine!"

"What did you search for, son?"

"Troop 43, of course!"

"Well that's your first problem. There's a Troop 43 in nearly every state, sometimes in every county, and the one here in Marin is really hard to find because no one links to it. Also, it's on a community domain, one that probably protects its content from search engine crawlers. You should have narrowed your search - like I did, but to be honest, even that wouldn't have helped. You have to search elsewhere - did you think to search your mail?"

As my son contemplated the idea that Google might not be omniscient, something struck me. Faced with the same question - "what is Troop 43's website address" - my son and I both did the exact same thing - we asked Google. I asked the question a little bit differently than did my son, but we both got poor results. Yet due to years of conditioning, we instinctively assumed Google would give us the One Right Answer. That assumption - in particular my son's, who has never used a computer without a browser and Google services - united us.

In ten short years, Google has become our social glue - we all presume that two people, asking roughly the same question, will get pretty much the same answer, and that answer will be correct. For most of the past decade, that was a pretty fair assumption. Google has become a universal search resource, reliable, accurate, and ... consistent.

But for a variety of reasons, that assumption is no longer true. The ongoing goal of all search providers has been to personalize search - to tailor answers to the individual who is doing the searching. Search no longer takes one signal - your query - and finds results against the entire web. Instead it takes many signals - your search history, your geographic location, things you've clicked on in the past, files on your hard drive (if you allow it), and many others - and processes those signals against probable sub sets of data that have a higher chance of providing *you* the best answer. And that answer, increasingly, will be quite different from someone else's, even if that other person asks exactly the same question.

Along the way, I think, something has been lost. It's the same thing my mother lamented as she watched my generation abandon the newspaper - common ground, common spaces - a common set of facts around which we as humans can gather, debate, and connect. And therein lies an opportunity, I sense, to create a new kind of search that is in fact *not* personalized, but rather socialized - shared and common to all.