Overview of the 2019 open-source IR replicability challenge (OSIRRC 2019)

Ryan Clancy, Nicola Ferro, Claudia Hauff, Jimmy Lin, Tetsuya Sakai, Ze Zhong Wu

研究成果: Conference article査読

5 被引用数 (Scopus)


The Open-Source IR Replicability Challenge (OSIRRC 2019), organized as a workshop at SIGIR 2019, aims to improve the replicability of ad hoc retrieval experiments in information retrieval by gathering a community of researchers to jointly develop a common Docker specification and build Docker images that encapsulate a diversity of systems and retrieval models. We articulate the goals of this workshop and describe the "jig" that encodes the Docker specification. In total, 13 teams from around the world submitted 17 images, most of which were designed to produce retrieval runs for the TREC 2004 Robust Track test collection. This exercise demonstrates the feasibility of orchestrating large, community-based replication experiments with Docker technology. We envision OSIRRC becoming an ongoing community-wide effort to ensure experimental replicability and sustained progress on standard test collections.

ジャーナルCEUR Workshop Proceedings
出版ステータスPublished - 2019
イベント2019 Open-Source IR Replicability Challenge, OSIRRC 2019 - Paris, France
継続期間: 2019 7月 25 → …

ASJC Scopus subject areas

  • コンピュータサイエンス一般


「Overview of the 2019 open-source IR replicability challenge (OSIRRC 2019)」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。