  In medical image analysis, automated segmentation of multi-component
anatomical structures, which often have a spectrum of potential anomalies and
pathologies, is a challenging task. In this work, we develop a multi-step
approach using U-Net-based neural networks to initially detect anomalies (bone
marrow lesions, bone cysts) in the distal femur, proximal tibia and patella
from 3D magnetic resonance (MR) images of the knee in individuals with varying
grades of osteoarthritis. Subsequently, the extracted data are used for
downstream tasks involving semantic segmentation of individual bone and
cartilage volumes as well as bone anomalies. For anomaly detection, the
U-Net-based models were developed to reconstruct the bone profiles of the femur
and tibia in images via inpainting so anomalous bone regions could be replaced
with close to normal appearances. The reconstruction error was used to detect
bone anomalies. A second anomaly-aware network, which was compared to
anomaly-na\"ive segmentation networks, was used to provide a final automated
segmentation of the femoral, tibial and patellar bones and cartilages from the
knee MR images containing a spectrum of bone anomalies. The anomaly-aware
segmentation approach provided up to 58% reduction in Hausdorff distances for
bone segmentations compared to the results from the anomaly-na\"ive
segmentation networks. In addition, the anomaly-aware networks were able to
detect bone lesions in the MR images with greater sensitivity and specificity
(area under the receiver operating characteristic curve [AUC] up to 0.896)
compared to the anomaly-na\"ive segmentation networks (AUC up to 0.874).
