  We present CAFA-evaluator, a powerful Python program designed to evaluate the
performance of prediction methods on targets with hierarchical concept
dependencies. It generalizes multi-label evaluation to modern ontologies where
the prediction targets are drawn from a directed acyclic graph and achieves
high efficiency by leveraging matrix computation and topological sorting. The
program requirements include a small number of standard Python libraries,
making CAFA-evaluator easy to maintain. The code replicates the Critical
Assessment of protein Function Annotation (CAFA) benchmarking, which evaluates
predictions of the consistent subgraphs in Gene Ontology. Owing to its
reliability and accuracy, the organizers have selected CAFA-evaluator as the
official CAFA evaluation software.
