A deep learning framework for automated molecular recognitions in scanning probe microscopy images.

Researchers

Journal

Modalities

Models

Abstract

Computer vision as a subcategory of deep learning tackles complex vision tasks by dealing with data of images and videos. Molecular images with exceptionally high resolution have been routinely achieved thanks to the development of surface-sensitive techniques of scanning probe microscopy (SPM) advancing our understanding on materials. However, extracting useful information from SPM image data requires careful analysis which heavily relies on constant human supervision. In this work, we address the challenge of molecule detection, classification and instance segmentation in binary molecular nanostructures with the computer version. We develop a deep learning framework using an advanced computer vision algorithm, Mask R-CNN, which can automatedly perform SPM image analysis with high efficiency and accuracy. To demonstrate its usefulness and high sensitivity in molecular recognition, we employ the framework to determine two triangular-shaped molecules of similar STM appearance. Our framework could accurately differentiate two molecules and label the positions of each molecule in STM images. We foresee that the application of computer vision in SPM images will become an indispensable part in the field, accelerating data mining and the discovery of new materials.© 2022 Wiley-VCH GmbH.