Science for SEO: Grand challenges for IR

There's an article in the LA Times where Marissa Mayer from Google is interviewed about the last 10 years of Google. She states in the interview:

"Search is an unsolved problem. We have a good 90 to 95% of the solution, but there is a lot to go in the remaining 10%."

She's right, but there's a lot to do still. SIGIR published a paper entitled "Some(what) Grand Challenges for Information Retrieval", written by Nicholas Belkin (Rutgers Uni). Some of the challenges he identifies are (and I quote):

1) "the ability to characterize and differentiate among information-related goals, tasks and intentions in some principled manners that go beyond straightforward listings, that will apply across a wide variety of contexts, and from which design principles for IR systems can be inferred".

2)" we need to develop methods for inferring information-related goals, tasks and intentions from implicit sources of evidence, such as previous or concurrent behaviors".

3) "going from characterization and identification of goals, tasks and intentions, to IR techniques which actually respond effectively to them, is a challenge that has been barely noted, much less addressed, to date. One reason is, of course, that we so far don’t have the necessary characterizations, but another is that putting together the research expertise in the study of information behavior and in the development of IR techniques is a challenge in and of itself."

4) We need to identify what "context" is

5) To understand how emotions affect information search.

6) Personalisation is far from perfect.

7) Integrating IR in the search environment.

8) Better evaluation is needed.

People like Liadh Kelly (Dublin City Uni) have been looking at improving retrieval in a more human way:

"Existing retrieval techniques are good at retrieving from non-personal spaces, such as the World Wide Web. However they are not sufficient for retrieval of items from these new unstructured spaces which contain items that are personal to the individual, and of which the user has personal memories and with which has had previous interaction. We believe that there are new and exciting possibilities for retrieval from personal archives."

That kind of work is interesting because it starts to push through into area of IR we just didn't look at previously.

It's very interesting to look at what Bruce Croft (UMASS) said we wanted from IR in 1995. His 10 issues are relevance and feedback, IE, multimedia retrieval, Effective retrieval, Rooting and filtering, Interfaces and browsing, "Magic", Indexing and retrieval, Distributed IR and integrated solutions. The "Magic" issue concerns the vocabulary mismatch, we have come a long way in this area. In fact we have come a long way since 1995, definitely, however the fundamental issues still remain, we don't yet have a system that works perfectly. Google has addressed all of Croft's issues though.

So yes, 10% of the search problem left...arguably a bit more. Multilingual information retrieval is still quite a challenge, a big challenge.