  LLMs have significantly advanced the e-commerce industry by powering
applications such as personalized recommendations and customer service.
However, most current efforts focus solely on monolithic LLMs and fall short in
addressing the complexity and scale of real-world e-commerce scenarios. In this
work, we present JungleGPT, the first compound AI system tailored for
real-world e-commerce applications. We outline the system's design and the
techniques used to optimize its performance for practical use cases, which have
proven to reduce inference costs to less than 1% of what they would be with a
powerful, monolithic LLM.
