When a doctor examines an unfamiliar medical image, they often ask, ‘Have I seen something like this before?’ This intelligent system embeds that concept into AI-assisted analysis of upper gastrointestinal endoscopy images. It begins in a standard way: a trained classification model predicts an initial lesion category for the image. However, this prediction isn’t final immediately. Next, a retrieval model searches the training dataset for images with similar features. The system then compares the lesion categories of these retrieved cases with the initial prediction. Based on this comparison, and under certain conditions, the retrieved examples can influence the final lesion classification. If the final category matches a predefined lesion class, a trained segmentation model is used to pinpoint the lesion in the image. This process involves multiple stages: classification, retrieval, and segmentation. What makes this approach notably interesting is its use of stored examples, training data that can guide decisions not just during training but also in analyzing new cases. While it doesn’t replace the expertise of clinicians or eliminate the need for clinical judgment, it provides an alternative to relying solely on a single model output. Overall, this approach demonstrates how AI’s first prediction can serve as just one form of evidence, with the system re-evaluating its guess based on similar past cases before finalizing the lesion classification and identifying the area of concern.
An AI system that compares its initial diagnosis with similar cases
Patent number: CN 113744203 B
Inventor(s): Z. Zheng, S. Tang, Y. Liang, X. Yu, H. Yu, and Y. Xu
Citation: Z. Zheng, S. Tang, Y. Liang, X. Yu, H. Yu, and Y. Xu, “基于多任务辅助的上消化道病变区域确定方法及装置 [Multi-task-assisted method and apparatus for determining upper gastrointestinal lesion regions],” China Patent CN 113744203 B, Mar. 25, 2025.
