  In this paper, we describe and study the indicator mining problem in the
online sex advertising domain. We present an in-development system, FlagIt
(Flexible and adaptive generation of Indicators from text), which combines the
benefits of both a lightweight expert system and classical semi-supervision
(heuristic re-labeling) with recently released state-of-the-art unsupervised
text embeddings to tag millions of sentences with indicators that are highly
correlated with human trafficking. The FlagIt technology stack is open source.
On preliminary evaluations involving five indicators, FlagIt illustrates
promising performance compared to several alternatives. The system is being
actively developed, refined and integrated into a domain-specific search system
used by over 200 law enforcement agencies to combat human trafficking, and is
being aggressively extended to mine at least six more indicators with minimal
programming effort. FlagIt is a good example of a system that operates in
limited label settings, and that requires creative combinations of established
machine learning techniques to produce outputs that could be used by real-world
non-technical analysts.
