  Since the emergence of convolutional neural networks (CNNs), and later vision
transformers (ViTs), the common paradigm for model development has always been
using a set of identical block types with varying parameters/hyper-parameters.
To leverage the benefits of different architectural designs (e.g. CNNs and
ViTs), we propose to alternate structurally different types of blocks to
generate a new architecture, mimicking how Lego blocks can be assembled
together. Using two CNN-based and one SwinViT-based blocks, we investigate
three variations to the so-called LegoNet that applies the new concept of block
alternation for the segmentation task in medical imaging. We also study a new
clinical problem which has not been investigated before, namely the right
internal mammary artery (RIMA) and perivascular space segmentation from
computed tomography angiography (CTA) which has demonstrated a prognostic value
to major cardiovascular outcomes. We compare the model performance against
popular CNN and ViT architectures using two large datasets (e.g. achieving
0.749 dice similarity coefficient (DSC) on the larger dataset). We evaluate the
performance of the model on three external testing cohorts as well, where an
expert clinician made corrections to the model segmented results (DSC>0.90 for
the three cohorts). To assess our proposed model for suitability in clinical
use, we perform intra- and inter-observer variability analysis. Finally, we
investigate a joint self-supervised learning approach to assess its impact on
model performance. The code and the pretrained model weights will be available
upon acceptance.
