  Detecting out of policy speech (OOPS) content is important but difficult.
While machine learning is a powerful tool to tackle this challenging task, it
is hard to break the performance ceiling due to factors like quantity and
quality limitations on training data and inconsistencies in OOPS definition and
data labeling. To realize the full potential of available limited resources, we
propose a meta learning technique (MLT) that combines individual models built
with different text representations. We analytically show that the resulting
technique is numerically stable and produces reasonable combining weights. We
combine the MLT with a threshold-moving (TM) technique to further improve the
performance of the combined predictor on highly-imbalanced in-distribution and
out-of-distribution datasets. We also provide computational results to show the
statistically significant advantages of the proposed MLT approach.
  All authors contributed equally to this work.
