Lack of user focus
- none of the papers at ECIR, etc… have the word user in the title
- This an example of the tenor of mainstream IR research. Important people (i.e. Karen Sparck Jones) indicated that the operating conditions of real people with real systems have been neglected.
- IR is really about helping people find information. We need to consider motivations, intentions, background information, current task at hand, etc…
- While some progress has been made, the TREC (Cranfield) evaluation methodology, this setting largely ignores users and interactive retrieval
- New ranking models and techniques are limited to incremental improvements
- Improvements in TREC-style evaluation performance rarely leads to improvement in interactive systems
- Goals, tasks and intentions. These affect judgments of usefulness and the way they want to interact with information objects.
- The ability to differentiate between these in principled ways that can be applied across many contexts and this should lead to design principles.
- Ways of inferring context better. Most work in this field has been explicit definition of tasks.
- Systems that effectively respond to different needs
The problem is to address the interactive model of IR. We need models that incorporate the user into the model as a central process, not just as a feedback mechanism. We may have to give up strict and formal for real and useful models of IR.