  The second Information Retrieval Education through EXperimentation track
(EIREX 2011) was run at the University Carlos III of Madrid, during the 2011
spring semester. EIREX 2011 is the second in a series of experiments designed
to foster new Information Retrieval (IR) education methodologies and resources,
with the specific goal of teaching undergraduate IR courses from an
experimental perspective. For an introduction to the motivation behind the
EIREX experiments, see the first sections of [Urbano et al., 2011a]. For
information on other editions of EIREX and related data, see the website at
http://ir.kr.inf.uc3m.es/eirex/. The EIREX series have the following goals: a)
to help students get a view of the Information Retrieval process as they would
find it in a real-world scenario, either industrial or academic; b) to make
students realize the importance of laboratory experiments in Computer Science
and have them initiated in their execution and analysis; c) to create a public
repository of resources to teach Information Retrieval courses; d) to seek the
collaboration and active participation of other Universities in this endeavor.
This overview paper summarizes the results of the EIREX 2011 track, focusing on
the creation of the test collection and the analysis to assess its reliability.
