  The rapid proliferation of generative AI has raised questions about the
competitiveness of lower-parameter, locally tunable, open-weight models
relative to high-parameter, API-guarded, closed-weight models in terms of
performance, domain adaptation, cost, and generalization. Centering
under-resourced yet risk-intolerant settings in government, research, and
healthcare, we see for-profit closed-weight models as incompatible with
requirements for transparency, privacy, adaptability, and standards of
evidence. Yet the performance penalty in using open-weight models, especially
in low-data and low-resource settings, is unclear.
  We assess the feasibility of using smaller, open-weight models to replace
GPT-4-Turbo in zero-shot, few-shot, and fine-tuned regimes, assuming access to
only a single, low-cost GPU. We assess value-sensitive issues around bias,
privacy, and abstention on three additional tasks relevant to those topics. We
find that with relatively low effort, very low absolute monetary cost, and
relatively little data for fine-tuning, small open-weight models can achieve
competitive performance in domain-adapted tasks without sacrificing generality.
We then run experiments considering practical issues in bias, privacy, and
hallucination risk, finding that open models offer several benefits over closed
models. We intend this work as a case study in understanding the opportunity
cost of reproducibility and transparency over for-profit state-of-the-art zero
shot performance, finding this cost to be marginal under realistic settings.
