  We present NUBIA, a methodology to build automatic evaluation metrics for
text generation using only machine learning models as core components. A
typical NUBIA model is composed of three modules: a neural feature extractor,
an aggregator and a calibrator. We demonstrate an implementation of NUBIA which
outperforms metrics currently used to evaluate machine translation, summaries
and slightly exceeds/matches state of the art metrics on correlation with human
judgement on the WMT segment-level Direct Assessment task, sentence-level
ranking and image captioning evaluation. The model implemented is modular,
explainable and set to continuously improve over time.
