  In expressive and controllable Text-to-Speech (TTS), explicit prosodic
features significantly improve the naturalness and controllability of
synthesised speech. However, manual prosody annotation is labor-intensive and
inconsistent. To address this issue, a two-stage automatic annotation pipeline
is novelly proposed in this paper. In the first stage, we use contrastive
pretraining of Speech-Silence and Word-Punctuation (SSWP) pairs to enhance
prosodic information in latent representations. In the second stage, we build a
multi-modal prosody annotator, comprising pretrained encoders, a text-speech
fusing scheme, and a sequence classifier. Experiments on English prosodic
boundaries demonstrate that our method achieves state-of-the-art (SOTA)
performance with 0.72 and 0.93 f1 score for Prosodic Word and Prosodic Phrase
boundary respectively, while bearing remarkable robustness to data scarcity.
