  Training of semantic segmentation models for material analysis requires
micrographs and their corresponding masks. It is quite unlikely that perfect
masks will be drawn, especially at the edges of objects, and sometimes the
amount of data that can be obtained is small, since only a few samples are
available. These aspects make it very problematic to train a robust model. We
demonstrate a workflow for the improvement of semantic segmentation models of
micrographs through the generation of synthetic microstructural images in
conjunction with masks. The workflow only requires joining a few micrographs
with their respective masks to create the input for a Vector
Quantised-Variational AutoEncoder model that includes an embedding space, which
is trained such that a generative model (PixelCNN) learns the distribution of
each input, transformed into discrete codes, and can be used to sample new
codes. The latter will eventually be decoded by VQ-VAE to generate images
alongside corresponding masks for semantic segmentation. To evaluate the
synthetic data, we have trained U-Net models with different amounts of these
synthetic data in conjunction with real data. These models were then evaluated
using non-synthetic images only. Additionally, we introduce a customized metric
derived from the mean Intersection over Union (mIoU). The proposed metric
prevents a few falsely predicted pixels from greatly reducing the value of the
mIoU. We have achieved a reduction in sample preparation and acquisition times,
as well as the efforts, needed for image processing and labeling tasks, are
less when it comes to training semantic segmentation model. The approach could
be generalized to various types of image data such that it serves as a
user-friendly solution for training models with a small number of real images.
