  Large language models excel in many human-language tasks but often falter in
highly specialized domains like scholarly astronomy. To bridge this gap, we
introduce AstroLLaMA, a 7-billion-parameter model fine-tuned from LLaMA-2 using
over 300,000 astronomy abstracts from arXiv. Optimized for traditional causal
language modeling, AstroLLaMA achieves a 30% lower perplexity than Llama-2,
showing marked domain adaptation. Our model generates more insightful and
scientifically relevant text completions and embedding extraction than
state-of-the-arts foundation models despite having significantly fewer
parameters. AstroLLaMA serves as a robust, domain-specific model with broad
fine-tuning potential. Its public release aims to spur astronomy-focused
research, including automatic paper summarization and conversational agent
development.
