  When evaluating the clinical performance of a medical imaging device, a
multi-reader multi-case (MRMC) analysis is usually applied to account for both
case and reader variability. For a clinical task that equates to a quantitative
measurement, an agreement analysis such as a limits of agreement (LOA) method
can be used to compare different measurement methods. In this work, we
introduce four types of comparisons; these types differ depending on whether
the measurements are within or between readers and within or between
modalities. A three-way mixed effect ANOVA model is applied to estimate the
variances of individual differences, which is an essential step for estimating
LOA. To verify the estimates of LOA, we propose a hierarchical model to
simulate quantitative MRMC data. Two simulation studies were conducted to
validate both the simulation and the LOA variance estimates. From the
simulation results, we can conclude that our estimate of variance is unbiased,
and the uncertainty of the estimation drops as the number of readers and cases
increases and rises as the value of true variance increases.
