Article Article
Confidence Interval Comparisons For Probability of Detection On Hit/Miss Data

Probability of detection (POD) studies for evaluating the capabilities of an inspection system for Air Force aircraft structural components commonly use a Logistic Regression model with a Wald 95% confidence interval. However, hit/miss POD data is distributed as a Binomial, and the sample sizes are commonly too small for Wald’s identically and independently normality distributed assumption to be true. This paper uses a large set of simulated representative hit/miss data to compare and contrast the performance of the four confidence intervals methods: Standard Wald, Modified Wald, Profile Likelihood Ratio, and Profile Modified Likelihood Ratio. Performance is measured in terms of bias and existence of a90/95 with respect to data distribution, sample size, overlap, and evenness. This paper provides guidance and methodology on new POD methods that more reliably and accurately estimate a90/95.



Agresti, A. 2002. Categorical Data Analysis. 2nd ed. Hoboken, New Jersey: Wiley. https://doi:10.1002/0471249688.

Annis, Charles P.E. 2014. “Influence of sample characteristics on probability of detection curves.” AIP Conference Proceedings 1581, 2039-2046.

Annis, C., L. Gandossi. 2012. ENIQ TGR Technical Document - Influence of Sample Size and Other Factors on Hit/Miss Probability of Detection Curves (ENIQ report N. 47). EUR 25200 EN. Luxembourg (Luxembourg): Publications Office of the European Union. JRC68677

Annis, C., J. C. Aldrin, and H. A. Sabbagh. 2015. “Profile likelihood: What to do when maximum probability of detection never gets to one.” Materials Evaluation, 73 (1): 96–99.

Annis, C., L. Gandossi, and O. Martin. 2013. “Optimal sample size for probability of detection curves.” Nuclear Engineering and Design, 262 (September 2013): 98-105. JRC74951

Barndorff-Nielsen, O. E. 1986. “Inference on Full or Partial Parameters Based on the Standardized Signed Log Likelihood Ratio.” Biometrika 73 (2): 307–22.

Barndorff-Nielsen, O., and D.R. Cox. 1979. “Edgeworth and Saddle-Point Approximations with Statistical Applications.” Journal of the Royal Statistical Society: Series B (Methodological), 41: 279-299.

Bates, O., and D. Watts. 1988. Nonlinear Regression Analysis and Its Applications. Hoboken, New Jersey: Wiley. doi:10.1002/9780470316757.

Brazzale, A.R., A.C. Davison, and N. Reid. 2007. Applied Asymptotics: Case Studies in Small-Sample Statistics. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge, UK: Cambridge University Press. 

Brazzale, Alessandra R., and Anthony C. Davison. 2008. “Accurate Parametric Inference for Small Samples.” Statist. Sci. 23 (4): 465–484.

Casella, G., and R. Berger. 2001. Statistical Inference. 2nd ed. Boston, Massachusetts: Cengage Learning.

Gandossi, L. and C. Annis. 2010. ENIQ TGR Technical Document – Probability of Detection Curves: Statistical Best-Practices (ENIQ report N. 41). EUR 24429 EN. Luxembourg (Luxembourg): Publications Office of the European Union. JRC56672.

Gerhard, D. 2016. “Simultaneous small sample inference for linear 

combinations of generalized linear model parameters.” Communications in Statistics – Simulation and Computation 45 (8): 2678–90. https://doi:10.1080/03610918.2014.895836.

Hothorn, T., F. Bretz, and P. Westfall. 2008. Simultaneous Inference in General Parametric Models. Technical Report. Department of Statistics, University of Munich.

Mameli, V. and A.R. Brazzale. 2016. “Modern likelihood inference for the maximum/minimum of a bivariate normal vector.” Journal of Statistical Computation and Simulation, 86 (10): 1869-1890,

US DOD (Department of Defense). 2010. MIL-HDBK-1823A: Nondestructive Evaluation System Reliability Assessment, Department of Defense Handbook.

Venables, W.N. and B.D. Ripley. 2002. Modern Applied Statistics with S. 4th edition. Statistics and Computing. New York, New York: Springer.

Usage Shares
Total Views
44 Page Views
Total Shares
0 Tweets
0 PDF Downloads
0 Facebook Shares
Total Usage