  In neutral hydrogen (HI) galaxy survey, a significant challenge is to
identify and extract the HI galaxy signal from observational data contaminated
by radio frequency interference (RFI). For a drift-scan survey, or more
generally a survey of a spatially continuous region, in the time-ordered
spectral data, the HI galaxies and RFI all appear as regions which extend an
area in the time-frequency waterfall plot, so the extraction of the HI galaxies
and RFI from such data can be regarded as an image segmentation problem, and
machine learning methods can be applied to solve such problems. In this study,
we develop a method to effectively detect and extract signals of HI galaxies
based on a Mask R-CNN network combined with the PointRend method. By simulating
FAST-observed galaxy signals and potential RFI impacts, we created a realistic
data set for the training and testing of our neural network. We compared five
different architectures and selected the best-performing one. This architecture
successfully performs instance segmentation of HI galaxy signals in the
RFI-contaminated time-ordered data (TOD), achieving a precision of 98.64% and a
recall of 93.59%.
