  Optimizing scientific software is a difficult task because codebases are
often large and complex, and performance can depend upon several factors
including the algorithm, its implementation, and hardware among others. Causes
of poor performance can originate from disparate sources and be difficult to
diagnose. Recent years have seen a multitude of work that use large language
models (LLMs) to assist in software development tasks. However, these tools are
trained to model the distribution of code as text, and are not specifically
designed to understand performance aspects of code. In this work, we introduce
a reinforcement learning based methodology to align the outputs of code LLMs
with performance. This allows us to build upon the current code modeling
capabilities of LLMs and extend them to generate better performing code. We
demonstrate that our fine-tuned model improves the expected speedup of
generated code over base models for a set of benchmark tasks from 0.9 to 1.6
for serial code and 1.9 to 4.5 for OpenMP code.
