  Large Language Models have recently gained significant attention in
scientific discovery for their extensive knowledge and advanced reasoning
capabilities. However, they encounter challenges in effectively simulating
observational feedback and grounding it with language to propel advancements in
physical scientific discovery. Conversely, human scientists undertake
scientific discovery by formulating hypotheses, conducting experiments, and
revising theories through observational analysis. Inspired by this, we propose
to enhance the knowledge-driven, abstract reasoning abilities of LLMs with the
computational strength of simulations. We introduce Scientific Generative Agent
(SGA), a bilevel optimization framework: LLMs act as knowledgeable and
versatile thinkers, proposing scientific hypotheses and reason about discrete
components, such as physics equations or molecule structures; meanwhile,
simulations function as experimental platforms, providing observational
feedback and optimizing via differentiability for continuous parts, such as
physical parameters. We conduct extensive experiments to demonstrate our
framework's efficacy in constitutive law discovery and molecular design,
unveiling novel solutions that differ from conventional human expectations yet
remain coherent upon analysis.
