A Comparative Study on Out of Scope Detection for Chest X-ray Images

Abstract

Image classification models in actual applications may receive input outside the intended data distribution. For crucial applications such as clinical decision-making, it is critical that a model can recognize and describe such out-of-distribution (OOD) inputs. The objective of this study is to investigate the efficacy of several approaches for OOD identification in medical images. We examine three classes of OOD detection methods (Classification models, Confidence-based models, and Generative models) on the data of X-ray images. We found that simple classification methods and HealthyGAN perform the best overall. However, HealthyGAN cannot generalize to unseen scenarios, while classification models still retain some performance advantage. We also investigate the type of images that might be harder to detect as out of scope. We found that image crop-outs, while being easily identifiable by humans, are more challenging for the models to detect.