  The segmentation of the mitral valve annulus and leaflets specifies a crucial
first step to establish a machine learning pipeline that can support physicians
in performing multiple tasks, e.g.\ diagnosis of mitral valve diseases,
surgical planning, and intraoperative procedures. Current methods for mitral
valve segmentation on 2D echocardiography videos require extensive interaction
with annotators and perform poorly on low-quality and noisy videos. We propose
an automated and unsupervised method for the mitral valve segmentation based on
a low dimensional embedding of the echocardiography videos using neural network
collaborative filtering. The method is evaluated in a collection of
echocardiography videos of patients with a variety of mitral valve diseases,
and additionally on an independent test cohort. It outperforms state-of-the-art
\emph{unsupervised} and \emph{supervised} methods on low-quality videos or in
the case of sparse annotation.
