Friday, April 17

TREC 2009 Web Track details

The guidelines for the TREC 2009 Web Track are available.

There are two tasks:

1) Adhoc -- 50 short 'title only' topics
2) Diversity -- same topics as Adhoc, but:
The goal of the diversity task is to return a ranked list of pages that together provide complete coverage for a query, while avoiding excessive redundancy in the result list. For this task, the probability of relevance of a document is conditioned on the documents that appear before it in the result list.
Posted by at 10:40 AM

1 comments:

  1. I must say, the web track is starting to look quite interesting, particularly the diversity task. I'm very curious to see how everything pans out.

    I'm also curious about how the organizers are interpreting the users' goals for the diversity task. I can think of several different scenarios:
    (1) A single user is interested in a diverse set of information about the topic, that may not be fully answered by a single document.
    (2) A variety of users issuing the same query have different underlying information needs (or intents).

    I don't know if either scenario would necessitate in different evaluations or ranking techniques, but they strike me as very different processes from the users' point of view.
    ReplyDelete

Subscribe to: Post Comments (Atom)