  Problem Roulette (PR) is a web-based study service at the University of
Michigan that offers random-within-topic access to a large library of past exam
problems in introductory physics courses. Built on public-private cloud
computing infrastructure, PR served nearly 1000 students during Fall 2012 term,
delivering more than 60,000 problem pages. The service complements that of
commercial publishing houses by offering problems authored by local professors.
We describe the service architecture, including reporting and analytical
capabilities, and present an initial evaluation of the impact of its use. Among
roughly 500 students studying electromagnetism, we find that the 229 students
who worked fifty or more problems over the term outperformed their complement
by 0.40 grade points (on a 4.0 scale). This improvement partly reflects a
selection bias that better students used the service more frequently. Adjusting
for this selection bias by using overall grade point average (GPA) as a
classifier, we find a grade point improvement of 0.22 for regular PR users,
significantly above the random noise level of 0.04. Simply put, students who
worked one or more additional problem per weekday earned nearly a
quarter-letter average grade improvement irrespective of GPA. Student comments
emphasize the importance of randomness in helping them synthesize concepts. The
PR source code is publicly available on the bitbucket repository.
