  The COVID-19 corona virus has claimed 4.1 million lives, as of July 24, 2021.
A variety of machine learning models have been applied to related data to
predict important factors such as the severity of the disease, infection rate
and discover important prognostic factors. Often the usefulness of the findings
from the use of these techniques is reduced due to lack of method
interpretability. Some recent progress made on the interpretability of machine
learning models has the potential to unravel more insights while using
conventional machine learning models. In this work, we analyze COVID-19 blood
work data with some of the popular machine learning models; then we employ
state-of-the-art post-hoc local interpretability techniques(e.g.- SHAP, LIME),
and global interpretability techniques(e.g. - symbolic metamodeling) to the
trained black-box models to draw interpretable conclusions. In the gamut of
machine learning algorithms, regressions remain one of the simplest and most
explainable models with clear mathematical formulation. We explore one of the
most recent techniques called symbolic metamodeling to find the mathematical
expression of the machine learning models for COVID-19. We identify Acute
Kidney Injury (AKI), initial Albumin level (ALBI), Aspartate aminotransferase
(ASTI), Total Bilirubin initial(TBILI) and D-Dimer initial (DIMER) as major
prognostic factors of the disease severity. Our contributions are- (i) uncover
the underlying mathematical expression for the black-box models on COVID-19
severity prediction task (ii) we are the first to apply symbolic metamodeling
to this task, and (iii) discover important features and feature interactions.
