  Large Language Models (LLMs) create exciting possibilities for powerful
language processing tools to accelerate research in materials science. While
LLMs have great potential to accelerate materials understanding and discovery,
they currently fall short in being practical materials science tools. In this
position paper, we show relevant failure cases of LLMs in materials science
that reveal current limitations of LLMs related to comprehending and reasoning
over complex, interconnected materials science knowledge. Given those
shortcomings, we outline a framework for developing Materials Science LLMs
(MatSci-LLMs) that are grounded in materials science knowledge and hypothesis
generation followed by hypothesis testing. The path to attaining performant
MatSci-LLMs rests in large part on building high-quality, multi-modal datasets
sourced from scientific literature where various information extraction
challenges persist. As such, we describe key materials science information
extraction challenges which need to be overcome in order to build large-scale,
multi-modal datasets that capture valuable materials science knowledge.
Finally, we outline a roadmap for applying future MatSci-LLMs for real-world
materials discovery via: 1. Automated Knowledge Base Generation; 2. Automated
In-Silico Material Design; and 3. MatSci-LLM Integrated Self-Driving Materials
Laboratories.
