Multimodal Registration of Fish and Nanosims Images Using Convolutional Neural Network Models

Nanoscale secondary ion mass spectrometry (nanoSIMS) and fluorescence in situ hybridization (FISH) microscopy provide high-resolution, multimodal image representations of the identity cell activity respectively targeted microbial communities microbiological research. Despite its importance to microbiologists, registration FISH nanoSIMS images is challenging given morphological distortion background noise both images. In this study, we use convolutional neural networks (CNNs) for multiscale feature extraction, shape context computation minimum transformation cost matching thin-plate spline (TPS) model Registration accuracy was quantitatively assessed against manually registered images, at both, pixel structural levels using standard metrics. Although all six tested CNN models performed well, ResNet18 observed outperform VGG16, VGG19, GoogLeNet ShuffleNet ResNet101 based on most This study demonstrates utility CNNs with significant morphology distortion. We also show aggregate shape, preserved by binarization, be a robust registering microbiology-related