  We introduce TorchOpera, a compound AI system for enhancing the safety and
quality of prompts and responses for Large Language Models. TorchOpera ensures
that all user prompts are safe, contextually grounded, and effectively
processed, while enhancing LLM responses to be relevant and high quality.
TorchOpera utilizes the vector database for contextual grounding, rule-based
wrappers for flexible modifications, and specialized mechanisms for detecting
and adjusting unsafe or incorrect content. We also provide a view of the
compound AI system to reduce the computational cost. Extensive experiments show
that TorchOpera ensures the safety, reliability, and applicability of LLMs in
real-world settings while maintaining the efficiency of LLM responses.
