I am Fan and I am new to Apache Lucene Open Relevance Project. Recently I
have been working on developing a web tool for the project with a number of
collaborators and we are thinking of the system requirements now. The
requirements are listed as following:
1. The system is designed to used for evaluating the searching relevancy of
queries or search engines.
2. The system can be used on various commercial or open-source search
engines, such as Google, Bing, Lucene, Solr, etc.
3. The system should be able to accept and store test collections. The test
collections include a collection of queries and qrels.
4. The system should be able to run test collections and evaluate queries
or search engines with various evaluators. The system should provide with
certain standard evaluators, while user should be able to run customized
5. The system should be able to keep track of timestamps, queries, document
set, schema(including data model, analyzer, tokenizer, filter, etc.),
scoring model, evaluation metrics, searching results and relevancy score.
Any ideas? Thanks!