"By eliminating anything that does not directly contribute to the core competency, Cranfield loses realism but gains substantial experimental power". I think I never really appreciated the paradigm's resilience. An interesting read by Ellen Voorhees: https
386 followers
643 followers
The first evolved in NIST TREC into the Cranfield paradigm and helps humans find useful information. This motivation is central in tasks like NaturalQuestions, MS Marco, and TREC QA. These questions are asked by humans who don't know the answer. https://t.
1,521 followers
RT @tetsuyasakai: Sakai, T.: How to Run an Evaluation Task: with a Primary Focus on Ad Hoc Information Retrieval, Information Retrieval Eva…
2,671 followers
Sakai, T.: How to Run an Evaluation Task: with a Primary Focus on Ad Hoc Information Retrieval, Information Retrieval Evaluation in a Changing World: Lessons Learned from 20 Years of CLEF, pp.71-102, 2019. https://t.co/EJZPQtVv5Z https://t.co/z5CymGk2J5