Publication: Publication Date: 1 December 2022
Confidence Interval Comparisons For Probability of Detection On Hit/Miss Data
Probability of detection (POD) studies for evaluating the capabilities of an inspection system for Air Force aircraft structural components commonly use a Logistic Regression model with a Wald 95% confidence interval. However, hit/miss POD data is distributed as a Binomial, and the sample sizes are commonly too small for Wald’s identically and independently normality distributed assumption to be true. This paper uses a large set of simulated representative hit/miss data to compare and contrast the performance of the four confidence intervals methods: Standard Wald, Modified Wald, Profile Likelihood Ratio, and Profile Modified Likelihood Ratio. Performance is measured in terms of bias and existence of a90/95 with respect to data distribution, sample size, overlap, and evenness. This paper provides guidance and methodology on new POD methods that more reliably and accurately estimate a90/95.
DOI: https://doi.org/10.32548/2022.me-04273
References
Agresti, A. 2002. Categorical Data Analysis. 2nd ed. Hoboken, New Jersey: Wiley. https://doi:10.1002/0471249688.
Annis, Charles P.E. 2014. “Influence of sample characteristics on probability of detection curves.” AIP Conference Proceedings 1581, 2039-2046. https://doi.org/10.1063/1.4865074
Annis, C., L. Gandossi. 2012. ENIQ TGR Technical Document - Influence of Sample Size and Other Factors on Hit/Miss Probability of Detection Curves (ENIQ report N. 47). EUR 25200 EN. Luxembourg (Luxembourg): Publications Office of the European Union. JRC68677
Annis, C., J. C. Aldrin, and H. A. Sabbagh. 2015. “Profile likelihood: What to do when maximum probability of detection never gets to one.” Materials Evaluation, 73 (1): 96–99.
Annis, C., L. Gandossi, and O. Martin. 2013. “Optimal sample size for probability of detection curves.” Nuclear Engineering and Design, 262 (September 2013): 98-105. JRC74951
Barndorff-Nielsen, O. E. 1986. “Inference on Full or Partial Parameters Based on the Standardized Signed Log Likelihood Ratio.” Biometrika 73 (2): 307–22. https://doi.org/10.2307/2336207.
Barndorff-Nielsen, O., and D.R. Cox. 1979. “Edgeworth and Saddle-Point Approximations with Statistical Applications.” Journal of the Royal Statistical Society: Series B (Methodological), 41: 279-299. https://doi.org/10.1111/j.2517-6161.1979.tb01085.x
Bates, O., and D. Watts. 1988. Nonlinear Regression Analysis and Its Applications. Hoboken, New Jersey: Wiley. doi:10.1002/9780470316757.
Brazzale, A.R., A.C. Davison, and N. Reid. 2007. Applied Asymptotics: Case Studies in Small-Sample Statistics. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge, UK: Cambridge University Press.
Brazzale, Alessandra R., and Anthony C. Davison. 2008. “Accurate Parametric Inference for Small Samples.” Statist. Sci. 23 (4): 465–484. https://doi.org/10.1214/08-STS273
Casella, G., and R. Berger. 2001. Statistical Inference. 2nd ed. Boston, Massachusetts: Cengage Learning.
Gandossi, L. and C. Annis. 2010. ENIQ TGR Technical Document – Probability of Detection Curves: Statistical Best-Practices (ENIQ report N. 41). EUR 24429 EN. Luxembourg (Luxembourg): Publications Office of the European Union. JRC56672.
Gerhard, D. 2016. “Simultaneous small sample inference for linear
combinations of generalized linear model parameters.” Communications in Statistics – Simulation and Computation 45 (8): 2678–90. https://doi:10.1080/03610918.2014.895836.
Hothorn, T., F. Bretz, and P. Westfall. 2008. Simultaneous Inference in General Parametric Models. Technical Report. Department of Statistics, University of Munich.
Mameli, V. and A.R. Brazzale. 2016. “Modern likelihood inference for the maximum/minimum of a bivariate normal vector.” Journal of Statistical Computation and Simulation, 86 (10): 1869-1890, https://doi.org/10.1080/00949655.2015.1089872
US DOD (Department of Defense). 2010. MIL-HDBK-1823A: Nondestructive Evaluation System Reliability Assessment, Department of Defense Handbook.
Venables, W.N. and B.D. Ripley. 2002. Modern Applied Statistics with S. 4th edition. Statistics and Computing. New York, New York: Springer. https://doi.org/10.1007/978-0-387-21706-2
Metrics
| Usage | Shares |
|---|---|
Total Views 121 Page Views |
Total Shares 0 Tweets |
121 0 PDF Downloads |
0 0 Facebook Shares |
| Total Usage | |
| 121 |