  Industry practitioners always face the problem of choosing the appropriate
model for deployment under different considerations, such as to maximize a
metric that is crucial for production, or to reduce the total cost given
financial concerns. In this work, we focus on the text classification task and
present a quantitative analysis for this challenge. Using classification
accuracy as the main metric, we evaluate the classifiers' performances for a
variety of models, including large language models, along with their associated
costs, including the annotation cost, training (fine-tuning) cost, and
inference cost. We then discuss the model choices for situations like having a
large number of samples needed for inference. We hope our work will help people
better understand the cost/quality trade-offs for the text classification task.
