Do saliency maps find the defect? A ground-truth evaluation of explainable AI for safety-critical magnetic tile inspection

Abstract

Saliency maps are widely used to justify the decisions of deep-learning inspection systems, on the assumption that the highlighted region is the defect that caused the decision. This study tests that assumption on real industrial images. A convolutional neural network was trained to separate defective from defect-free tiles in the public Magnetic Tile Surface Defect dataset (1,344 images, five defect types, pixel-level defect masks), using five stratified train–test splits. Six saliency methods (Gradient, Gradient × Input, SmoothGrad, Integrated Gradients, Grad-CAM and Occlusion), a model-free Sobel edge map and a random map were evaluated on 382 held-out defective images for localisation against the masks, deletion-based faithfulness, cascading weight randomisation and agreement across independently trained models. The detector reached a test AUROC of 0.870 ± 0.032 and a balanced accuracy of 0.769 ± 0.061. Integrated Gradients, Gradient and Grad-CAM placed their maximum on the defect in about 60% of images, against 31% for the edge map and 15% for the random map, and Grad-CAM gave the best pixel AUROC (0.786 vs. 0.729 for edges, p = 7 × 10−9). Localisation was poor for large, diffuse defects (fray, uneven surface: AUROC 0.47–0.72). Deleting the 5% most salient pixels reversed 18–35% of correct detections, compared with 1–2% for random deletion of the same number of pixels. However, removing the annotated defect itself by inpainting left 72% of detections unchanged, the same edit made only 10% of correctly classified defect-free tiles look defective, and blurring a whole defect-free image made 95% of them look defective. The maps also pointed at the defect about as often when the model missed it as when it detected it. Gradient-based maps lost their structure after weight randomisation, whereas Integrated Gradients retained a rank correlation of 0.44 with the original map even when every layer was randomised, and agreement between three identically trained models ranged from 0.21 (Gradient) to 0.71 (Grad-CAM). Saliency maps that look plausible on a defect therefore do not show that the decision was based on that defect. For safety-critical inspection, saliency should be validated with mask-based localisation, defect-removal tests, sanity checks and stability analysis before it is used as evidence.