  In recent years Deep Learning has brought about a breakthrough in Medical
Image Segmentation. U-Net is the most prominent deep network in this regard,
which has been the most popular architecture in the medical imaging community.
Despite outstanding overall performance in segmenting multimodal medical
images, from extensive experimentations on challenging datasets, we found out
that the classical U-Net architecture seems to be lacking in certain aspects.
Therefore, we propose some modifications to improve upon the already
state-of-the-art U-Net model. Hence, following the modifications we develop a
novel architecture MultiResUNet as the potential successor to the successful
U-Net architecture. We have compared our proposed architecture MultiResUNet
with the classical U-Net on a vast repertoire of multimodal medical images.
Albeit slight improvements in the cases of ideal images, a remarkable gain in
performance has been attained for challenging images. We have evaluated our
model on five different datasets, each with their own unique challenges, and
have obtained a relative improvement in performance of 10.15%, 5.07%, 2.63%,
1.41%, and 0.62% respectively.
