  The creation of high-quality ontologies is crucial for data integration and
knowledge-based reasoning, specifically in the context of the rising data
economy. However, automatic ontology matchers are often bound to the heuristics
they are based on, leaving many matches unidentified. Interactive ontology
matching systems involving human experts have been introduced, but they do not
solve the fundamental issue of flexibly finding additional matches outside the
scope of the implemented heuristics, even though this is highly demanded in
industrial settings. Active machine learning methods appear to be a promising
path towards a flexible interactive ontology matcher. However, off-the-shelf
active learning mechanisms suffer from low query efficiency due to extreme
class imbalance, resulting in a last-mile problem where high human effort is
required to identify the remaining matches.
  To address the last-mile problem, this work introduces DualLoop, an active
learning method tailored to ontology matching. DualLoop offers three main
contributions: (1) an ensemble of tunable heuristic matchers, (2) a short-term
learner with a novel query strategy adapted to highly imbalanced data, and (3)
long-term learners to explore potential matches by creating and tuning new
heuristics. We evaluated DualLoop on three datasets of varying sizes and
domains. Compared to existing active learning methods, we consistently achieved
better F1 scores and recall, reducing the expected query cost spent on finding
90% of all matches by over 50%. Compared to traditional interactive ontology
matchers, we are able to find additional, last-mile matches. Finally, we detail
the successful deployment of our approach within an actual product and report
its operational performance results within the Architecture, Engineering, and
Construction (AEC) industry sector, showcasing its practical value and
efficiency.
