Friday, July 16

The Impact of TREC and its Future Directions

This week NIST released a report on the Economic Impact Assessment of TREC.

In section 6 they report the results of a survey where stakeholders were asked to rate the importance of the different TREC tracks. The most important tracks were Adhoc Track with 77% rating it very important and the Web Track with 74%. Other highly rated tracks were the TrecVid and Q&A tracks. In contrast, the Spam Track (mainly email) and Speech track were ranked at the bottom.

Here are few highlights from the conclusion. In monetary terms:
As described in Section 6, $16 million of discounted investments have made by NIST and others in TREC have resulted in $81 million in discounted extrapolated benefits or a net present value of $65 million.
The report continues to say that the same rate of benefit may not continue because much of the growth happened in the late 90s with the rapid expansion of the PC and the use of the Web.

The other main section I found interesting was the possible future directions:
Several trends in survey responses emerged; 37 respondents indicated that TREC should expand into new tracks, 20 said TREC should develop new evaluation methods, and 17 said TREC should develop new data sets. Common suggestions were the following:
  • Focus on more user behavior data (e.g., social data, Twitter, geographically based) to improve on the Interactive track.
  • Continue to look at multimedia search techniques (e.g., pictures, video).
  • Expand into more focused search areas (e.g., chemistry, drug design, evidence-based medicine).
More broadly, several respondents suggested that TREC should work with industry to increase their participation in the TREC workshops, as well as to solicit data that they might allow the TREC audience to use, thus increasing the usefulness of TREC results...

Wednesday, July 14

Google Funds Digital Classics Research on Google Books

Today Google announced that is providing almost 1 million dollars to researchers working on projects in the digital humanities.

The first award winners are listed in the blog entry. In particular I want to highlight Gregory Crane's award on classics. They previously worked to release a collection of high quality scans for 500 important classics works which are available for download.

Also, congratulations to David Mimno from UMass with David Blei for , The Open Encyclopedia of Classical Sites.