  This paper introduces 3D-Speaker-Toolkit, an open source toolkit for
multi-modal speaker verification and diarization. It is designed for the needs
of academic researchers and industrial practitioners. The 3D-Speaker-Toolkit
adeptly leverages the combined strengths of acoustic, semantic, and visual
data, seamlessly fusing these modalities to offer robust speaker recognition
capabilities. The acoustic module extracts speaker embeddings from acoustic
features, employing both fully-supervised and self-supervised learning
approaches. The semantic module leverages advanced language models to apprehend
the substance and context of spoken language, thereby augmenting the system's
proficiency in distinguishing speakers through linguistic patterns. Finally,
the visual module applies image processing technologies to scrutinize facial
features, which bolsters the precision of speaker diarization in multi-speaker
environments. Collectively, these modules empower the 3D-Speaker-Toolkit to
attain elevated levels of accuracy and dependability in executing
speaker-related tasks, establishing a new benchmark in multi-modal speaker
analysis. The 3D-Speaker project also includes a handful of open-sourced
state-of-the-art models and a large dataset containing over 10,000 speakers.
The toolkit is publicly available at
https://github.com/alibaba-damo-academy/3D-Speaker.
