Pulmonologists collaborate with explainable artificial intelligence for improved diagnostic interpretation of pulmonary function tests.

Read the official abstract here.

Rationale: Few studies have investigated the collaborative potential between artificial intelligence (AI) and a pulmonologist for diagnosing disease. We hypothesized that a collaboration between a pulmonologist and an AI with explanations (explainable AI, XAI)) will be superior in diagnostic interpretation of pulmonary function tests (PFTs) than an individual pulmonologist without support.

Methods: The study was conducted in two phases, a single-centre P1 and a multi-centre P2. Each phase utilized two different sets of 24 PFT reports with gold-standard diagnoses. Each PFT was interpreted without (control) and then with XAI’s suggestions (intervention). Pulmonologists provided a preferential and up to 4 differential diagnoses. Primary endpoint compared accuracy of preferential and differential diagnosis between groips. Secondary endpoints were number of differential diagnoses, diagnostic confidence and inter-rater agreement. The XAI model utilized an earlier AI-based PFT interpretation model, for which Shapley values expressed reasoning for predictions.

Results: In P1 (N=16 pulmonologists), mean preferential and differential diagnostic accuracy significantly increased by 10.4% and 9.4%, respectively between groups (p<0.001). Improvements were highly significant (p<0.0001) in P2 (5.4% and 8.7% respectively, N=62). In both phases, number of differential diagnoses did not reduce, but diagnostic confidence and inter-rater agreement significantly increased in intervention. Pulmonologists updated their decisions in half of the cases, and consistently improved their baseline performance if AI provided correct predictions.

Conclusion: A collaboration between pulmonologist and XAI is superior at interpreting PFTs when compared to individual pulmonologists.

Authors: Nilakash Das, Sofie Happaerts, Iwein Gyselinck, Eric Derom, Mustapha Abdo, Guy Brussselle, FelipBurgos, Marco Contoli, AnhTuan Dinh-Xuan, Frits M.E. Franssen, Sherif Gonem, Neil Greening, Christel Haenebalcke, William D-C.Man, Jorge Moisés, Rudi Peche, VitaliiPoberezhets, Jennifer K Quint, Michael C. Steiner, Eef Vanderelst, Marko Topalovic, and Wim Janssens