Effects of Uncertainty Visualization on Attention, Arousal and Auditor Judgment

We conduct two experiments to examine the effects of visualizing the measurement uncertainty (uncertainty) in artificial intelligence (AI) recommendations on individuals’ attention and cognitive arousal, and auditors’ incorporation of uncertainty information into their judgments. The first experiment employs psychophysiological measurement techniques to better understand how different methods of presenting uncertainty information affect assimilation into decision-making processes. Results of this experiment indicate that participants attend more to the uncertainty inherent in AI recommendations when uncertainty is incorporated directly into the visualization, relative to the current practice of visualizations that are devoid of graphically embedded uncertainty information but rather supplemented with textual descriptions of margins of error. Pupillometry and eye tracking analyses indicate that participants exhibit greater attention to uncertainty information, fixate more on the bounds of uncertainty, and spend more time examining uncertainty information when uncertainty is depicted in the visualization . In addition, participants who view visualizations depicting uncertainty are more likely to make decisions consistent with the uncertainty inherent in recommendations from AI. The second experiment examines the effects of visualizations that depict uncertainty information versus those that do not on practicing Big 4 auditors, and finds that experienced auditors are more likely to appropriately use uncertainty information when it is visualized. Taken together, the experiments reveal how decision makers and auditors incorporate visualizations of uncertainty information into their judgments, and results indicate that a novel uncertainty display has significant potential to improve auditor judgment and audit quality.