  Large Language Models (LLMs) have sparked significant interest in their
generative capabilities, leading to the development of various commercial
applications. The high cost of using the models drives application builders to
maximize the value of generation under a limited inference budget. This paper
presents a study of optimizing inference hyperparameters such as the number of
responses, temperature and max tokens, which significantly affects the
utility/cost of text generation. We design a framework named EcoOptiGen which
leverages economical hyperparameter optimization and cost-based pruning.
Experiments with the GPT-3.5/GPT-4 models on a variety of tasks verify its
effectiveness. EcoOptiGen is implemented in the `autogen' package of the FLAML
library: \url{https://aka.ms/autogen}.
