  The locations of different mRNA molecules can be revealed by multiplexed in
situ RNA detection. By assigning detected mRNA molecules to individual cells,
it is possible to identify many different cell types in parallel. This in turn
enables investigation of the spatial cellular architecture in tissue, which is
crucial for furthering our understanding of biological processes and diseases.
However, cell typing typically depends on the segmentation of cell nuclei,
which is often done based on images of a DNA stain, such as DAPI. Limiting cell
definition to a nuclear stain makes it fundamentally difficult to determine
accurate cell borders, and thereby also difficult to assign mRNA molecules to
the correct cell. As such, we have developed a computational tool that segments
cells solely based on the local composition of mRNA molecules. First, a small
neural network is trained to compute attractive and repulsive edges between
pairs of mRNA molecules. The signed graph is then partitioned by a mutex
watershed into components corresponding to different cells. We evaluated our
method on two publicly available datasets and compared it against the current
state-of-the-art and older baselines. We conclude that combining neural
networks with combinatorial optimization is a promising approach for cell
segmentation of in situ transcriptomics data.
