  Large Language Models (LLMs) like GPT-4 have revolutionized natural language
processing, showing remarkable linguistic proficiency and reasoning
capabilities. However, their application in strategic multi-agent
decision-making environments is hampered by significant limitations including
poor mathematical reasoning, difficulty in following instructions, and a
tendency to generate incorrect information. These deficiencies hinder their
performance in strategic and interactive tasks that demand adherence to nuanced
game rules, long-term planning, exploration in unknown environments, and
anticipation of opponents' moves. To overcome these obstacles, this paper
presents a novel LLM agent framework equipped with memory and specialized tools
to enhance their strategic decision-making capabilities. We deploy the tools in
a number of economically important environments, in particular bilateral
bargaining and multi-agent and dynamic mechanism design. We employ quantitative
metrics to assess the framework's performance in various strategic
decision-making problems. Our findings establish that our enhanced framework
significantly improves the strategic decision-making capability of LLMs. While
we highlight the inherent limitations of current LLM models, we demonstrate the
improvements through targeted enhancements, suggesting a promising direction
for future developments in LLM applications for interactive environments.
