  We describe an investigation of the use of probabilistic models and
cost-benefit analyses to guide resource-intensive procedures used by a
Web-based question answering system. We first provide an overview of research
on question-answering systems. Then, we present details on AskMSR, a prototype
web-based question answering system. We discuss Bayesian analyses of the
quality of answers generated by the system and show how we can endow the system
with the ability to make decisions about the number of queries issued to a
search engine, given the cost of queries and the expected value of query
results in refining an ultimate answer. Finally, we review the results of a set
of experiments.
