  In the rapidly evolving field of natural language processing, the translation
of linguistic descriptions into mathematical formulation of optimization
problems presents a formidable challenge, demanding intricate understanding and
processing capabilities from Large Language Models (LLMs). This study compares
prominent LLMs, including GPT-3.5, GPT-4, and Llama-2-7b, in zero-shot and
one-shot settings for this task. Our findings show GPT-4's superior
performance, particularly in the one-shot scenario. A central part of this
research is the introduction of `LM4OPT,' a progressive fine-tuning framework
for Llama-2-7b that utilizes noisy embeddings and specialized datasets.
However, this research highlights a notable gap in the contextual understanding
capabilities of smaller models such as Llama-2-7b compared to larger
counterparts, especially in processing lengthy and complex input contexts. Our
empirical investigation, utilizing the NL4Opt dataset, unveils that GPT-4
surpasses the baseline performance established by previous research, achieving
an F1-score of 0.63, solely based on the problem description in natural
language, and without relying on any additional named entity information.
GPT-3.5 follows closely, both outperforming the fine-tuned Llama-2-7b. These
findings not only benchmark the current capabilities of LLMs in a novel
application area but also lay the groundwork for future improvements in
mathematical formulation of optimization problems from natural language input.
