  Language Models have demonstrated remarkable capabilities on some tasks while
failing dramatically on others. The situation has generated considerable
interest in understanding and comparing the capabilities of various Language
Models (LMs) but those efforts have been largely ad hoc with results that are
often little more than anecdotal. This is in stark contrast with testing and
evaluation processes used in healthcare, radar signal processing, and other
defense areas. In this paper, we describe Test and Evaluation of Language
Models (TEL'M) as a principled approach for assessing the value of current and
future LMs focused on high-value commercial, government and national security
applications. We believe that this methodology could be applied to other
Artificial Intelligence (AI) technologies as part of the larger goal of
"industrializing" AI.
