  Proteins are the workhorses of life and gaining insight on their functions is
of paramount importance for applications such as drug design. However, the
experimental validation of functions of proteins is highly-resource consuming.
Therefore, recently, automated protein function prediction (AFP) using machine
learning has gained significant interest. Many of these AFP tools are based on
supervised learning models trained using existing gold-standard functional
annotations, which are known to be incomplete. The main challenge associated
with conducting systematic testing on AFP software is the lack of a test
oracle, which determines passing or failing of a test case; unfortunately, due
to the incompleteness of gold-standard data, the exact expected outcomes are
not well defined for the AFP task. Thus, AFP tools face the \emph{oracle
problem}. In this work, we use metamorphic testing (MT) to test nine
state-of-the-art AFP tools by defining a set of metamorphic relations (MRs)
that apply input transformations to protein sequences. According to our
results, we observe that several AFP tools fail all the test cases causing
concerns over the quality of their predictions.
