  How do you scale a machine learning product at a startup? In particular, how
do you serve a greater volume, velocity, and variety of queries
cost-effectively? We break down costs into variable costs-the cost of serving
the model and performant-and fixed costs-the cost of developing and training
new models. We propose a framework for conceptualizing these costs, breaking
them into finer categories, and limn ways to reduce costs. Lastly, since in our
experience, the most expensive fixed cost of a machine learning system is the
cost of identifying the root causes of failures and driving continuous
improvement, we present a way to conceptualize the issues and share our
methodology for the same.
