  We present Speakerly, a new real-time voice-based writing assistance system
that helps users with text composition across various use cases such as emails,
instant messages, and notes. The user can interact with the system through
instructions or dictation, and the system generates a well-formatted and
coherent document. We describe the system architecture and detail how we
address the various challenges while building and deploying such a system at
scale. More specifically, our system uses a combination of small, task-specific
models as well as pre-trained language models for fast and effective text
composition while supporting a variety of input modes for better usability.
