  API call generation is the cornerstone of large language models' tool-using
ability that provides access to the larger world. However, existing supervised
and in-context learning approaches suffer from high training costs, poor data
efficiency, and generated API calls that can be unfaithful to the API
documentation and the user's request. To address these limitations, we propose
an output-side optimization approach called FANTASE. Two of the unique
contributions of FANTASE are its State-Tracked Constrained Decoding (SCD) and
Reranking components. SCD dynamically incorporates appropriate API constraints
in the form of Token Search Trie for efficient and guaranteed generation
faithfulness with respect to the API documentation. The Reranking component
efficiently brings in the supervised signal by leveraging a lightweight model
as the discriminator to rerank the beam-searched candidate generations of the
large language model. We demonstrate the superior performance of FANTASE in API
call generation accuracy, inference efficiency, and context efficiency with
DSTC8 and API Bank datasets.
