  Energy economy optimization (EEO) models employ formal search techniques to
explore the future decision space over several decades in order to deliver
policy-relevant insights. EEO models are a critical tool for decision-makers
who must make near-term decisions with long-term effects in the face of large
future uncertainties. While the number of model-based analyses proliferates,
insufficient attention is paid to transparency in model development and
application. Given the complex, data-intensive nature of EEO models and the
general lack of access to source code and data, many of the assumptions
underlying model-based analysis are hidden from external observers. This paper
discusses the simplifications and subjective judgments involved in the model
building process, which cannot be fully articulated in journal papers, reports,
or model documentation. In addition, we argue that for all practical purposes,
EEO model-based insights cannot be validated through comparison to real world
outcomes. As a result, modelers are left without credible metrics to assess a
model's ability to deliver reliable insight. We assert that EEO models should
be discoverable through interrogation of publicly available source code and
data. In addition, third parties should be able to run a specific model
instance in order to independently verify published results. Yet a review of
twelve EEO models suggests that in most cases, replication of model results is
currently impossible. We provide several recommendations to help develop and
sustain a software framework for repeatable model analysis.
