  In recent years, continuous latent space (CLS) and discrete latent space
(DLS) deep learning models have been proposed for medical image analysis for
improved performance. However, these models encounter distinct challenges. CLS
models capture intricate details but often lack interpretability in terms of
structural representation and robustness due to their emphasis on low-level
features. Conversely, DLS models offer interpretability, robustness, and the
ability to capture coarse-grained information thanks to their structured latent
space. However, DLS models have limited efficacy in capturing fine-grained
details. To address the limitations of both DLS and CLS models, we propose
SynergyNet, a novel bottleneck architecture designed to enhance existing
encoder-decoder segmentation frameworks. SynergyNet seamlessly integrates
discrete and continuous representations to harness complementary information
and successfully preserves both fine and coarse-grained details in the learned
representations. Our extensive experiment on multi-organ segmentation and
cardiac datasets demonstrates that SynergyNet outperforms other state of the
art methods, including TransUNet: dice scores improving by 2.16%, and Hausdorff
scores improving by 11.13%, respectively. When evaluating skin lesion and brain
tumor segmentation datasets, we observe a remarkable improvement of 1.71% in
Intersection-over Union scores for skin lesion segmentation and of 8.58% for
brain tumor segmentation. Our innovative approach paves the way for enhancing
the overall performance and capabilities of deep learning models in the
critical domain of medical image analysis.
