Breakthrough in Quantum Computing: Two Studies Demonstrate Significant Advancements

Reply to: False conflict and false confirmation errors are crucial components of AI accuracy in medical decision making - Nature Communications

The purpose of our publication "Dermatologist- explainable AI enhances trust and confidence in diagnosing melanoma" was to build and evaluate an explainable AI (XAI) model to distinguish between melanomas and nevi1. The model was intended as a prototype of an assistance system for clinicians and thus designed to be able to explain its decisions in a dermatologist-understandable way. In addition to the development of the XAI, we conducted a reader study with clinicians to quantify their interaction with our XAI with regard to the influence on the clinicians' diagnostic accuracy, their confidence in their own diagnoses and their trust in the assistance systemlike. The reader study was conducted in different phases, so that the clinicians diagnosed the same lesions at different points in time with at least two weeks in between with different levels of AI support to increase comparability. Rosenbacke, Melhus and Stuckler focused in their Matters Arising on three error types that occur in the field of human-AI-interactions and on sub-group investigations based on physicians' performance2; an important topic in human-AI interaction tasks overall3,4. We fully agree with Rosenbacke and colleagues that the investigation of these three errors and the performance-based sub-group analysis are highly relevant points for the introduction of (X)AI into clinical practice. They mention three types of errors that can occur: (i) false confirmation error -- when the physician and the AI agree but both are wrong; (ii) false conflict error -- when the physician is correct, AI is incorrect, and the physician changes diagnosis (which is a particularly difficult case from an ethical perspective); and (iii) true conflict error -- when the physician is incorrect but AI is correct, and the physician overrides the correct AI diagnosis. From our point of view, those three errors are very important to investigate, but not able to provide a complete picture. Thus, we propose introducing one additional error type and four additional scenarios leading to correctly diagnosed cases. The errors Rosenbacke and colleagues did not mention are the (iv) true confirmation errors -- when both the physician and the AI diagnosis are correct, but the physician subsequently switches to an incorrect diagnosis. This occurred in 3.9% of cases in our dataset and might be caused by unrealistic explanations. Furthermore, we argue that we must also consider the four correctly diagnosed cases to paint a complete picture. These are: (a) correct true confirmation cases -- when both the AI and the physician are correct and the physician does not change their diagnosis; (b) correct true conflict cases -- when the AI is correct and the physician is wrong, but the physician accepts the AI decision; (c) correct false confirmation cases -- when both the AI and the physician's initial diagnoses are wrong, but the physician changes their diagnosis when receiving an incorrect AI suggestion; and (d) correct false conflict cases -- when the AI is wrong, the clinician is correct and the clinician overrides the incorrect AI decision. Especially when investigating the individual errors for different subgroups, it is necessary to take all eight cases into account. All the mentioned scenarios are summarized in Table 1, Subtable A) which also contains the scenario identifiers (i-iv and a-d). It should be noted that the AI's and the clinicians' correctness are independent, since the clinicians delivered their initial diagnosis without AI advice. To conduct the sub-group analysis, we defined the 25%-quantile in physicians' accuracy as the threshold for the worst performers and the AI's accuracy (80.4%) as the threshold for the best performers respectively, as suggested by Rosenbacke and colleagues. We report the absolute numbers for all 8 cases in Table 1 for the whole available dataset (Subtable B), the best performers (Subtable C) and the worst performers (Subtable D). It should be noted that the relative performance of a clinician is not trivially discernible in a clinical setting. However, it might be correlated with years of experience or the weekly load of lesions seen by the clinician.

False conflict and false confirmation errors are crucial components of AI accuracy in medical decision making - Nature Communications

Recent studies are beginning to delve deeper into how physicians respond to conflicts with AI. The most common and discussed error occurs when physicians tend to override a correct AI diagnosis in cases of true conflict error. Previous studies found that this arises from distrust in the AI's "black box" logic. In cases of false conflict errors, however, the physicians tended to express doubt and over-rely upon AI, especially when uncertain about their initial diagnosis. When explanations are added to the AI diagnoses (as XAI), it tends to mitigate true conflict errors but exacerbate false conflict errors. This phenomenon whereby even mere exposure to explanations can induce overreliance on AI has been documented in several studies. Finally, false confirmation is perhaps the most pernicious; it reinforces trust in AI, while perpetuating clinical errors. These false confirmation errors remind us of the confirmation bias highlighted by Ghassemi and colleagues. This issue is likely present in the study conducted by Chanda and colleagues, though it was not explicitly addressed.