  Investing efficiently in future research to improve policy decisions is an
important goal. Expected Value of Sample Information (EVSI) can be used to
select the specific design and sample size of a proposed study by assessing the
benefit of a range of different studies. Estimating EVSI with the standard
nested Monte Carlo algorithm has a notoriously high computational burden,
especially when using a complex decision model or when optimizing over study
sample sizes and designs. Therefore, a number of more efficient EVSI
approximation methods have been developed. However, these approximation methods
have not been compared and therefore their relative advantages and
disadvantages are not clear. A consortium of EVSI researchers, including the
developers of several approximation methods, compared four EVSI methods using
three previously published health economic models. The examples were chosen to
represent a range of real-world contexts, including situations with multiple
study outcomes, missing data, and data from an observational rather than a
randomized study. The computational speed and accuracy of each method were
compared, and the relative advantages and implementation challenges of the
methods were highlighted. In each example, the approximation methods took
minutes or hours to achieve reasonably accurate EVSI estimates, whereas the
traditional Monte Carlo method took weeks. Specific methods are particularly
suited to problems where we wish to compare multiple proposed sample sizes,
when the proposed sample size is large, or when the health economic model is
computationally expensive. All the evaluated methods gave estimates similar to
those given by traditional Monte Carlo, suggesting that EVSI can now be
efficiently computed with confidence in realistic examples.
