Publication: Publication Date: 1 December 2022
Modern Design and Analysis For Hit/Miss Probability of Detection Studies Using Profile Likelihood Ratio Confidence Intervals
To characterize the capability of an inspection system, indications from the system must be collected over a range of defect sizes. For flaw indications, insufficient sample size, overlap, or evenness between hit and miss indications may cause the probability of detection (POD) estimations to not exist or have high bias. Extensive simulations of representative Lognormal, Weibull, and Uniformly distributed data at varying levels of overlap, evenness, and sample size were fit using four modeling techniques: logistic regression, Firth’s Bias Adjusted Likelihood, the Lasso, and a ranked set sampling method from nonparametric statistics. Profile likelihood ratio confidence intervals were used instead of the standard Wald method to calculate a90/95. The probability of existence and the percent bias of the estimates provide recommendations for the ideal levels of overlap, evenness, modeling technique, and sample size requirements when designing a hit/miss POD study.
DOI: https://doi.org/10.32548/2022.me-04272
References
Agresti, A. 2002. Categorical Data Analysis. 2nd ed. Hoboken, New Jersey: Wiley. https://doi.org/10.1002/0471249688.
Annis, C. 2014. “Influence of sample characteristics on probability of detection curves.” 40th Annual Review of Progress in Quantitative Nondestructive Evaluation 1581:2039–46.
Annis, C., J.C. Aldrin, H.A. Sabbagh. 2015. “What is Missing in Nondestructive Testing Capability Evaluation?” Materials Evaluation 73 (1): 38–42.
Berens, A. 1989. “NDE Reliability Data Analysis.” in ASM Handbook, Volume 17: Nondestructive Evaluation of Materials. 9th ed. Materials Park, OH: ASM International: 689–701.
Berens, A., 2000. Probability of Detection (POD) Analysis for the Advanced Retirement for Cause (RFC)/Engine Structural Integrity Program (ENSIP) Nondestructive Evaluation (NDE) System Development, Technical Report AFRL-ML-WP-TR-2001-4010. Technical Report. Air Force Research Laboratory.
Casella, G., and R. Berger. 2001. Statistical Inference. 2nd ed. Boston, Massachusetts: Cengage Learning.
Chen, H., E.A. Stasny, and D.A. Wolfe. 2005. “Ranked set sampling for efficient estimation of a population proportion.” Statistics in Medicine 24 (21): 3319–29. https://doi.org/10.1002/sim.2158.
Harding, C., and G. Hugo. 2003. “Experimental determination of the probability of detection hit/miss data for small data sets.” Review of Progress in Quantitative Nondestructive Evaluation 22:1823–44.
Hastie, T., R. Tibshirani, and J. Friedman. 2016. The Elements of Statistical Machine Learning. 2nd ed. New York: Springer.
Henry, Christine, Christine Schubert Kabban, Matthew Cherry, and Ryan Mooers. 2019. “Ranked set sampling applied to hit-miss probability of detection data: A case study,” AIP Conference Proceedings 2102, 090002. https://doi.org/10.1063/1.5099820
Hollander, M., D.A. Wolfe, and E. Chicken. 2014. Nonparametric Statistical Methods. 3rd ed., New York: John Wiley and Sons Inc.
James, G., D. Witten, T. Hastie, and R. Tibshirani. 2013. An Introduction to Statistical Learning with Applications in R. New York: Springer. https://doi.org/10.1007/978-1-4614-7138-7.
Knopp, J., and L. Zeng. 2013. “Statistical analysis of hit/miss data.” Materials Evaluation 71 (3): 323–29.
Knott, C.E., and C. Schubert Kabban. 2022. “Confidence Interval Comparisons for Probability of Detection on Hit/Miss Data,” Materials Evaluation 80 (12): 50-65. https://doi.org/10.32548/2022.me-04273.
US DOD (Department of Defense). 2010. MIL-HDBK-1823A: Nondestructive Evaluation System Reliability Assessment, U.S. Department of Defense Handbook.
US DOD (Department of Defense). 2016. MIL-STD-1530D: Department of Defense Standard Practice: Aircraft Structural Integrity Program, Airplane Requirements.
NTIAC. 1997. Nondestructive Evaluation (NDE) Capabilities Data Book. 3rd ed. Nondestructive Testing Information Analysis Center. Austin, TX: Texas Research Institute Austin Inc.
Olin, B., and W. Meeker. 1996. “Applications of statistical methods to nondestructive evaluation.” Technometrics 38 (2): 95–112. https://doi.org/10.1080/00401706.1996.10484451.
Santos, K.C.P., and E.B. Barrios. 2017. “Improving predictive accuracy of logistic regression model using ranked set samples.” Communications in Statistics - Simulation and Computation 46 (1): 78–90. https://doi.org/10.1080/03610918.2014.955113.
Singh, R. 2000. Three Decades of NDI Reliability Assessment. Technical Report Karta-3510-99-01. AF NDI Office.
Metrics
| Usage | Shares |
|---|---|
Total Views 77 Page Views |
Total Shares 0 Tweets |
77 0 PDF Downloads |
0 0 Facebook Shares |
| Total Usage | |
| 77 |