Treble CLEF Evaluation package and language resources CLEF
Treble. CLEF Evaluation package and language resources CLEF Evaluation Packages Nicolas Moreau / Khalid Choukri - ELDA
Treble. CLEF and LRs CLEF evaluation campaigns produced large volumes of valuable data A major objective of Treble. CLEF: make CLEF collections publicly available ELDA responsible for: Inventory of CLEF resources IPR issues Production of CLEF Evaluation Packages
Evaluation Packages CLEF test suites consist of: Multilingual document collections Sets of topics Sets of relevance assessments Scoring tools Participant results and related publications Step-by-step documentation on how to perform a system evaluation Reusable test suites for system benchmarking
CLEF Packages Already distributed (ELRA/ELDA catalogue): CLEF 2000 -2003 (Ad. Hoc on news + domain specific) Produced or being produced: CLEF Ad. Hoc/News 2004 -2008 => Preliminary version CLEF Ad. Hoc/TEL 2008 CLEF QA 2003 -2008 + CLEF QAST 2007 -2008 CLEF Domain specific 2004 -2008 Geo. CLEF 2005 -2008 Still in discussion: Image. CLEF, Video. CLEF Web. CLEF, CLEF-IP, and others. . .
Distribution and Pricing Policy Distribution ELRA/ELDA catalogue: http: //catalog. elra. info/ Restricted to evaluation use Pricing policy Low prices for academics Cover the management of data (catalogue management, DVD shipping, etc. )
- Slides: 5