Monday, September 8

Solving Search: A Game of Guess the Magic Words

Maybe sometimes when you are searching for information you feel like Popeye trying to open the magic cave door to rescue Olive Oyl from the forty thieves:
I wonder what words he used when he opened this door? Open sissy, open cecil, no that can't be it! whoop, it's giving way, it's giving way...
- Popeye watch the full episode via the IA (or jump to the second half on YouTube)
Unfortunately, Popeye's super strength won't help you with your search tasks. Or perhaps you feel like the bumbling fool Hasan attempting to prevent Daffy from stealing the master's treasure:

Using today's search engines is playing the game of Guess the Magic Words. Guess the right words, and Open Sesame! Guess poorly and you bang your head against the cave door for minutes or hours. How good are you?

A current problem for search engines today is this: If you type a long query and give the search engine more information about your information need you are likely to get worse results than if you entered only a few brief keywords.

Barney Pell from PowerSet describes the current language of search engines as keywordese. If you enter too few of these keywords your query is likely too vague; too many keywords and relevant documents are mistakenly filtered out. And too often we don't know the 'magic words' to find the desired information.

Until search engines utilize the information in long queries without being overwhelmed by the 'noise' search will remain broken.

Marissa Mayer said in a recent LA Times Article on Google's 10 year anniversary:
I think there will be a continued focus on innovation, particularly in search. Search is an unsolved problem. We have a good 90 to 95% of the solution, but there is a lot to go in the remaining 10%.
Search isn't 90% solved. It's not easy to quantify because search is constantly evolving. Regardless, beating the Guess the Magic Words level in the game of search is still a long way off.


  1. jeff..hi! i like your blogs..i cant imagine that such ah very busy person like's to cook...keep up the good work and god bless...MJ philippines

  2. Ya, not really:

    We’re all familiar with 80-20 problems, where the last 20% of the solution is 80% of the work. Search is a 90-10 problem. Today, we have a 90% solution: I could answer all of my unanswered Saturday questions, not ideally or easily, but I could get it done with today’s search tool. (If you’re curious, the answers are below.) However, that remaining 10% of the problem really represents 90% (in fact, more than 90%) of the work.

    There's no way we have a 90% solution, after all we are still in the 1500 or 1600s of search. Oi!