  Accurate segmentation of organs from abdominal CT scans is essential for
clinical applications such as diagnosis, treatment planning, and patient
monitoring. To handle challenges of heterogeneity in organ shapes, sizes, and
complex anatomical relationships, we propose a \textbf{\textit{\ac{MDNet}}}, an
encoder-decoder network that uses the pre-trained \textit{MiT-B2} as the
encoder and multiple different decoder networks. Each decoder network is
connected to a different part of the encoder via a multi-scale feature
enhancement dilated block. With each decoder, we increase the depth of the
network iteratively and refine segmentation masks, enriching feature maps by
integrating previous decoders' feature maps. To refine the feature map further,
we also utilize the predicted masks from the previous decoder to the current
decoder to provide spatial attention across foreground and background regions.
MDNet effectively refines the segmentation mask with a high dice similarity
coefficient (DSC) of 0.9013 and 0.9169 on the Liver Tumor segmentation (LiTS)
and MSD Spleen datasets. Additionally, it reduces Hausdorff distance (HD) to
3.79 for the LiTS dataset and 2.26 for the spleen segmentation dataset,
underscoring the precision of MDNet in capturing the complex contours.
Moreover, \textit{\ac{MDNet}} is more interpretable and robust compared to the
other baseline models.
