A Probabilistic Model for the MRMC Method, Part 2: Validation and Applications
A Probabilistic Model for the MRMC Method, Part 2: Validation and Applications
We have previously described a probabilistic model for the multiple-reader, multiple-case paradigm for receiver operating characteristic analysis. When the figure of merit is the Wilcoxon statistic, this model returns a seven-term expansion for the variance of this statistic as a function of the numbers of cases and readers. This probabilistic model also provides expressions for the coefficients in the seven-term expansion in terms of expectations over the internal noise, readers, and cases. Finally, this probabilistic model sets bounds on both the overall variance of the Wilcoxon statistic and the individual coefficients.In this article, we will first validate the probabilistic model by comparing variances determined by direct computation of the expansion coefficients to empirical estimates of the variance using independent sampling. Validation of the probabilistic model will enable us to use the direct estimates of the expansion coefficients as a gold standard to compare other coefficient-estimation techniques. Next, we develop a coefficient-estimation technique that employs bootstrapping to estimate the Wilcoxon statistic variance for different numbers of readers and cases. We then employ constrained, least-squares fitting techniques to estimate the expansion coefficients. The constraints used in this fitting are derived directly from the probabilistic model.Using two different simulation studies, we show that the novel (and practical) bootstrapping/fitting technique returns estimates of the coefficients that are consistent with the gold standard.The results presented also serve to validate the seven-term expansion for the variance of the Wilcoxon statistic.
- University of Arizona United States
Diagnostic Imaging, Observer Variation, Analysis of Variance, Models, Statistical, ROC Curve, Computer Simulation, Mathematical Computing, Statistics, Nonparametric
Diagnostic Imaging, Observer Variation, Analysis of Variance, Models, Statistical, ROC Curve, Computer Simulation, Mathematical Computing, Statistics, Nonparametric
4 Research products, page 1 of 1
- 2018IsAmongTopNSimilarDocuments
- 2013IsAmongTopNSimilarDocuments
- 2016IsAmongTopNSimilarDocuments
citations This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).9 popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.Average influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).Top 10% impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.Top 10%
