Registration for Correlative Microscopy Using Image Analogies

Abstract. Correlative microscopy is a methodology combining the functionality of light microscopy with the high resolution of electron microscopy and other microscopy technologies for the same biological specimen. In this paper, we propose an image registration method for correlative microscopy, which is challenging due to the distinct appearance of biological structures when imaged with di↵erent modalities. Our method is based on image analogies and allows to transform images of a given modality into the appearance-space of another modality. Hence, the registration between two di↵erent types of microscopy images can be transformed to a mono-modality image registration. We use a sparse representation model to obtain image analogies. The method makes use of representative corresponding image training patches of two di↵erent imaging modalities to learn a dictionary capturing appearance relations. We test our approach on backscattered electron (BSE) Scanning Electron Microscopy (SEM)/confocal and Transmission Electron Microscopy (TEM)/confocal images and show improvements over direct registration using a mutual-information similarity measure to account for di↵erences in image appearance.