Comments on Jeff's Search Engine Caffè: Open Source Search Engines, Retrieval Tools and Li...

On the MG4J subject you might mention Mímir -- htt...

2011-10-21T05:55:33.266-04:00

On the MG4J subject you might mention Mímir -- http://gate.ac.uk/family/mimir.html -- which layers annotation structure and semantic search (via an RDF repository) on top of MG4J.

A nice list -- thanks!

Jeff,

Thanks for putting this site together...

2008-05-23T11:38:00.000-04:00

Jeff,

Thanks for putting this site together - very informative.

I wanted to get your thoughts on one mof my requirements:

I am trying to build a search application that would require the following features of a search engine:

1. ability to handle federated searches (collating search results, ranking, etc)

2. The federated searches have to support Web-services to access data from multiple sources and in some cases be able to go directly against a database.

I am looking for a pluggable java-based open source solution. Would you have any thoughts on this?

Thanks...

TopX would be another engine from the research wor...

2007-04-06T15:57:00.000-04:00

TopX would be another engine from the research world that you could add...

It is now available at:

http://topx.sourceforge.net/

Do you know of existing projects that are dedicate...

2007-03-30T20:42:00.000-04:00

Do you know of existing projects that are dedicated to crawl only certain document type: PDF, DOC and PPT?

I'm interested in building a web site that would allow the community of researchers, lawyers, etc. directly search these document types.

I know Google has a filetype: filter, but I'd like to do more around the document: a la Digg, bookmarking them, discussing them, etc.

Hi Jeff,

Thanks for posting this list. Two ...

2007-03-30T12:32:00.000-04:00

Hi Jeff,

Thanks for posting this list. Two quick comments...

1. I'd also mention that Lucene is used by Solr, a cool enterprise search server (used by CNET, Krugle and others).

2. The maximum usable size of a single Lucene index has many free variables. For a typical Nutch-generated index, I think the upper bounds of 10-20M documents (assuming standard hardware) is about right. Document size, number of fields, complexity of the query, and amount of RAM are among the factors that can make this number go up or down.

Lucene does support merging results from multiple indexes, and adjusting for IDF skew in the process. The main problem here (IMO) with effectively using this support is that the operational support (e.g. code/scripts for managing federated searchers) doesn't really exist.

Hi Otis,

I talked to some of the Amazon dev...

2007-03-29T11:17:00.000-04:00

Hi Otis,

I talked to some of the Amazon developers at SIGIR 2006. They were giving out free t-shirts at their booth :-).

Oh, I didn't know Amazon's search inside the book ...

2007-03-29T08:57:00.000-04:00

Oh, I didn't know Amazon's search inside the book uses Lucene. Where did you see they use Lucene for that?
Simpy uses Lucene, too, of course.