  Microscopy images are powerful tools and widely used in the majority of
research areas, such as biology, chemistry, physics and materials fields by
various microscopies (scanning electron microscope (SEM), atomic force
microscope (AFM) and the optical microscope, et al.). However, most of the
microscopy images are colorless due to the unique imaging mechanism. Though
investigating on some popular solutions proposed recently about colorizing
images, we notice the process of those methods are usually tedious,
complicated, and time-consuming. In this paper, inspired by the achievement of
machine learning algorithms on different science fields, we introduce two
artificial neural networks for gray microscopy image colorization: An
end-to-end convolutional neural network (CNN) with a pre-trained model for
feature extraction and a pixel-to-pixel neural style transfer convolutional
neural network (NST-CNN), which can colorize gray microscopy images with
semantic information learned from a user-provided colorful image at inference
time. The results demonstrate that our algorithm not only can colorize the
microscopy images under complex circumstances precisely but also make the color
naturally according to the training of a massive number of nature images with
proper hue and saturation.
