Learning Mutual Modulation for Self-supervised Cross-Modal Super-Resolution

AbstractSelf-supervised cross-modal super-resolution (SR) can overcome the difficulty of acquiring paired training data, but is challenging because only low-resolution (LR) source and high-resolution (HR) guide images from different modalities are available. Existing methods utilize pseudo or weak supervision in LR space thus deliver results that blurry not faithful to modality. To address this issue, we present a mutual modulation SR (MMSR) model, which tackles task by strategy, including source-to-guide guide-to-source modulation. In these modulations, develop cross-domain adaptive filters fully exploit spatial dependency help induce emulate resolution mimic modality characteristics source. Moreover, adopt cycle consistency constraint train MMSR self-supervised manner. Experiments on various tasks demonstrate state-of-the-art performance our MMSR.KeywordsMutual modulationSelf-supervised super-resolutionCross-modalMulti-modalRemote sensing