SeATrans: Learning Segmentation-Assisted Diagnosis Model via Transformer

Clinically, the accurate annotation of lesions/tissues can significantly facilitate disease diagnosis. For example, segmentation optic disc/cup (OD/OC) on fundus image would glaucoma diagnosis, skin lesions dermoscopic images is helpful to melanoma etc. With advancement deep learning techniques, a wide range methods proved also automated diagnosis models. However, existing are limited in sense that they only capture static regional correlations images. Inspired by global and dynamic nature Vision Transformer, this paper, we propose Segmentation-Assisted Transformer (SeATrans) transfer knowledge network. Specifically, first an asymmetric multi-scale interaction strategy correlate each single low-level feature with features. Then, effective called SeA-block adopted vitalize via correlated To model segmentation-diagnosis interaction, embeds based information encoder, then transfers embedding back space decoder. Experimental results demonstrate SeATrans surpasses state-of-the-art (SOTA) segmentation-assisted several tasks.