Article Article
Intelligence Augmentation and Human-Machine Interface Best Practices for NDT 4.0 Reliability

NDT 4.0 is a vision for the next generation of nondestructive inspection systems following the expected fourth industrial revolution based on connected cyber-physical systems. While an increasing use of automation and algorithms in nondestructive testing (NDT) is expected over time, NDT inspectors will still play a critical role in ensuring NDT 4.0 reliability. As a counterpoint to recent advances in artificial intelligence algorithms, intelligence augmentation (IA) refers to the effective use of information technology to enhance human intelligence. While attempting to replicate the human mind has encountered many obstacles, IA has a much longer history of practical success. This paper introduces a series of best practices for NDT IA to support NDT 4.0 initiatives. Algorithms clearly have a great potential to help alleviate the burden of “big data” in NDT; however, it is important that inspectors are involved in necessary secondary indication review and the detection of rare event indications not addressed well by typical algorithms. Examples of transitioning algorithms for NDT applications will be presented, emphasizing the successful interfacing of inspector and software for optimal data review and decision making.



Aldrin, J.C., J.D. Achenbach, G. Andrew, C. P’an, B. Grills, R.T. Mullis, F.W. Spencer, and M. Golis, 2001, “Case Study for the Implementation of an Automated Ultrasonic Technique to Detect Fatigue Cracks in Aircraft Weep Holes,” Materials Evaluation, Vol. 59, No. 11, pp. 1313–1319.

Aldrin, J.C., C.V. Kropas-Hughes, J. Knopp, J. Mandeville, D. Judd, and E. Lindgren, 2006, “Advanced Echo-Dynamic Measures for the Characterisation of Multiple Ultrasonic Signals in Aircraft Structures,” Insight, Vol. 48, No. 3, pp. 144–148.

Aldrin, J.C., D.S. Forsyth, and J.T. Welter, 2016a, “Design and Demonstration of Automated Data Analysis Algorithms for Ultrasonic Inspection of Complex Composite Panels with Bonds,” 42nd Annual Review of Progress in Quantitative Nondestructive Evaluation, AIP Conference Proceedings, Vol. 1706, No. 1, p. 020006.

Aldrin, J.C., C. Annis, H.A. Sabbagh, and E.A. Lindgren, 2016b, “Best Practices for Evaluating the Capability of Nondestructive Evaluation (NDE) and Structural Health Monitoring (SHM) Techniques for Damage Characterization,” 42nd Annual Review of Progress in QNDE, Incorporating the 6th European-American Workshop on Reliability of NDE, AIP Conference Proceedings, Vol. 1706, p. 200002.

Aldrin, J.C., E.K. Oneida, E.B. Shell, H.A. Sabbagh, E. Sabbagh, R.K. Murphy, S. Mazdiyasni, E.A. Lindgren, and R.D. Mooers, 2017, “Model-Based Probe State Estimation and Crack Inverse Methods Addressing Eddy Current Probe Variability,” 43rd Annual Review of Progress in QNDE, AIP Conference Proceedings, Vol. 1806, No. 1, p. 110013.

Aldrin, J.C., and E.A. Lindgren, 2018, “The Need and Approach for Characterization - US Air Force Perspectives on Materials State Awareness,” 44th Annual Review of Progress in QNDE, AIP Conference Proceedings, Vol. 1949, No. 1, p. 020004.

Aldrin, J.C., E.A. Lindgren, and D. Forsyth, 2019, “Intelligence Augmentation in Nondestructive Evaluation,” 45th Annual Review of Progress in QNDE, AIP Conference Proceedings, Vol. 2012, No. 1, p. 020028.

Avatar Partners, 2017, “Vuforia Model Targets Application in Aircraft Maintenance,” available at

ASTM, 2015, ASTM E3023-15, Standard Practice for Probability of Detection Analysis for â Versus a Data, ASTM International, West Conshohocken, PA.

Bainbridge, L., 1987, “Ironies of Automation,” in New Technology and Human Error, J. Rasmussen, K. Duncan, and J. Leplat (eds.), John Wiley & Sons, Chichester, UK, pp. 271–283.

Bato, M.R., A. Hor, A. Rautureau, and C. Bes, 2017, “Implementation of a Robust Methodology to Obtain the Probability of Detection (POD) Curves in NDT: Integration of Human and Ergonomic Factors,” Les Journées COFREND 2017, 30 June–1 July, Strasbourg, France.

Bertović, M., 2016a, “Human Factors in Non-Destructive Testing (NDT): Risks and Challenges of Mechanised NDT,” Dissertation, Bundesanstalt für Materialforschung und -prüfung (BAM), Berlin, Germany.

Bertović, M., 2016b, “A Human Factors Perspective on the Use of Automated Aids in the Evaluation of NDT Data,” 42nd Annual Review of Progress in Quantitative Nondestructive Evaluation, AIP Conference Proceedings, Vol. 1706, No. 1, p. 020003.

Case, N., 2018, “How to Become a Centaur,” Journal of Design and Science, doi: 10.21428.61b2215c.

Cowen, T., 2013, Average Is Over: Powering America Beyond the Age of the Great Stagnation, Penguin Group, New York, NY.

Dudenhoeffer, D.D., D.E. Holcomb, B.P. Hallbert, R.T. Wood, L.J. Bond, D.W. Miller, J.M. O’Hara, E.L. Quinn, H.E. Garcia, S.A. Arndt, and J. Naser, 2007, “Technology Roadmap on Instrumentation, Control, and Human-Machine Interface to Support DOE Advanced Nuclear Energy Programs,” Report No. INL/EXT-06-11862, Idaho National Laboratory, Idaho Falls, ID.

Forsyth, D., J.C. Aldrin, and C.W. Magnuson, 2018, “Turning Nondestructive Testing Data into Useful Information,” Aircraft Airworthiness & Sustainment Conference, 23–26 April, Jacksonville, FL.

Fuchs, P., T. Kröger, T. Dierig, and C.S. Garbe, 2019, “Generating Meaningful Synthetic Ground Truth for Pore Detection in Cast Aluminum Parts,” Proceedings of Conference on Industrial Computed Tomography (iCT2019), Padua, Italy.

Fukushima, K., and S. Miyake, 1982, “Neocognitron: A Self-Organizing Neural Network Model for a Mechanism of Visual Pattern Recognition,” Competition and Cooperation in Neural Nets, pp. 267–285, Springer-Verlag, Berlin/Heidelberg, Germany.

Gerbert, P., 2018, “AI and the ‘Augmentation’ Fallacy,” MIT Sloan Management Review, available at

Hao, K., 2019, “When Algorithms Mess Up, the Nearest Human Gets the Blame,” MIT Technology Review, available at

Hinton, G.E., S. Osindero, and Y.-W. Teh, 2006, “A Fast Learning Algorithm for Deep Belief Nets,” Neural Computation, Vol. 18, No. 7, pp. 1527–1554.

Jahanzaib, I., and J. Jasperneite, 2013, “Scalability of OPC-UA Down to the Chip Level Enables ‘Internet of Things’,” in Proceedings of 11th IEEE International Conference on Industrial Informatics (INDIN), Bochum, Germany, pp. 500–505.

Jordon, H., 2018, “AFRL Viewing Aircraft Inspections through the Lens of Technology,” available at

Kobryn, P., E. Tuegel, J. Zweber, and R. Kolonay, 2017, “Digital Thread and Twin for Systems Engineering: EMD to Disposal,” 55th AIAA Aerospace Sciences Meeting, 9–13 January, Grapevine, TX.

LeCun, Y., Y. Bengio, and G. Hinton, 2015, “Deep Learning,” Nature, Vol. 521, No. 7553, pp. 436–444.

Lewis-Kraus, G., 2016, “The Great A.I. Awakening,” The New York Times Magazine, available at

Lindgren, E.A., 2017, “Opportunities for Nondestructive Evaluation: Quantitative Characterization,” Materials Evaluation, Vol. 75, No. 7, p. 862–869.

Lindgren, E.A., J.R. Mandeville, M.J. Concordia, T.J. MacInnis, J.J. Abel, J.C. Aldrin, F. Spencer, D.B. Fritz, P. Christiansen, R.T. Mullis, and R. Waldbusser, 2005, “Probability of Detection Results and Deployment of the Inspection of the Vertical Leg of the C-130 Center Wing Beam/Spar Cap,” 8th Joint DoD/FAA/NASA Conference on Aging Aircraft, 31 January–3 February, Palm Springs, CA.

Link, R., and N. Riess, 2018, “NDT 4.0 – Significance and Implications to NDT – Automated Magnetic Particle Testing as an Example,” 12th European Conference on Non-Destructive Testing (ECNDT 2018), 11–15 June, Gothenburg, Sweden.

Meier, J., I. Tsalicoglou, and R. Mennicke, 2017, “The Future of NDT with Wireless Sensors, AI and IoT,” 15th Asia Pacific Conference for Non-Destructive Testing, 13–17 November, Singapore, Singapore.

Meyendorf, N.G., L.J. Bond, J. Curtis-Beard, S. Heilmann, S. Pal, R. Schallert, H. Scholz, and C. Wunderlich, 2017a, “NDE 4.0—NDE for the 21st Century—the Internet of Things and Cyber-Physical Systems Will Revolutionize NDE,” 15th Asia Pacific Conference for Non-Destructive Testing, 13–17 November, Singapore, Singapore.

Meyendorf, N.G., R. Schallert, S. Pal, and L.J. Bond, 2017b, “Using Remote NDE, Including External Experts in the Inspection Process, to Enhance Reliability and Address Today’s NDE Challenges,” 7th European-American Workshop on Reliability of NDE, 4–7 September, Potsdam, Germany.

Müller, C., M. Bertovic, D. Kanzler, T. Heckel, and R. Holstein, 2014, “Assessment of the Reliability of NDE: A Novel Insight on Influencing Factors on POD and Human Factors in an Organizational Context,” Proceedings of the 11th European Conference on Non-Destructive Testing (ECNDT 2014), Prague, Czech Republic.

Müller, C., M. Golis, and T. Taylor, 2000, “Basic Ideas of the American-European Workshops 1997 in Berlin and 1999 in Boulder,” Proceedings of the 15th World Conference on Non-Destructive Testing (WCNDT), Rome, Italy, pp. 1–7.

Olden, J.D., and D.A. Jackson, 2002, “Illuminating the ‘Black Box’: A Randomization Approach for Understanding Variable Contributions in Artificial Neural Networks,” Ecological Modelling, Vol. 154, No. 1–2, pp. 135–150.

Oliver, N., T. Calvard, and K. Potočnik, 2017, “The Tragic Crash of Flight AF447 Shows the Unlikely but Catastrophic Consequences of Automation,” Harvard Business Review, available at

Rastogi, A., 2017, “Artificial Intelligence–Human Augmentation Is What’s Here and Now,” Medium, available at

Rice, S., and S. Winter, 2019, “The Boeing 737 Max Crisis Won’t Stop the March of Airline Automation,” Quartz, available at

Rummel, W.D., 2010, “Nondestructive Inspection Reliability—History, Status and Future Path,” Proceedings of the 18th World Conference on Nondestructive Testing, Durban, South Africa, pp. 16–20.

Sarter, N.B., D.D. Woods, and D.R. Billings, 1997, “Automation Surprises,” Handbook of Human Factors & Ergonomics, 2nd ed., G. Salvendy (ed.), Wiley, New York, NY, pp. 1926–1943.

Scharre, P., 2016, “Centaur Warfighting: The False Choice of Humans Vs. Automation,” Temple International and Comparative Law Journal, Vol. 30, pp. 151–165.

Sharp, T. D., J.M. Kesler, and U.M. Liggett, 2009, “Mining Inspection Data of Parts with Complex Shapes,” Proceedings of ASNT Annual Conference, American Society for Nondestructive Testing, Columbus, Ohio.

Singh, R., 2019, “The Next Revolution in Nondestructive Testing and Evaluation: What and How?,” Materials Evaluation, Vol. 77, No. 1, pp. 45–50.

Skagestad, P., 1993, “Thinking with Machines: Intelligence Augmentation, Evolutionary Epistemology, and Semiotic,” Journal of Social and Evolutionary Systems, Vol. 16, No. 2, pp. 157–180.

Stapleton, A., 2017, “DARPA’s Latest Take on AI Illuminates the Course of Intelligent Assistance,” available at

Taleb, N.N., 2007, “Black Swans and the Domains of Statistics,” The American Statistician, Vol. 61, No. 3, pp. 198–200.

Taleb, N.N., 2012, Antifragile: Things That Gain from Disorder, Random House, New York, New York.

Tharp, D., 2017, “What Cyborg Chess Can Teach Us about the Future of Financial Planning,” posted 24 July, available at

US DOD, 2009, MIL-HDBK-1823A, “Nondestructive Evaluation System Reliability Assessment,” US Department of Defense.

Vrana, J., K. Kadau, C. Amann, and D.U. Schnittstellen, 2018, “Non-Destructive Testing of Forgings on the Way to Industry 4.0,” Proceedings of ASNT Annual Conference, American Society for Nondestructive Testing, Columbus, OH.

Wilson, J.H., and P.R. Daugherty, 2018, “Collaborative Intelligence: Humans and AI Are Joining Forces,” Harvard Business Review, available at

Usage Shares
Total Views
85 Page Views
Total Shares
0 Tweets
0 PDF Downloads
0 Facebook Shares
Total Usage