  Big Data today is being generated at an unprecedented rate from various
sources such as sensors, applications, and devices, and it often needs to be
enriched based on other reference information to support complex analytical
queries. Depending on the use case, the enrichment operations can be compiled
code, declarative queries, or machine learning models with different
complexities. For enrichments that will be frequently used in the future, it
can be advantageous to push their computation into the ingestion pipeline so
that they can be stored (and queried) together with the data. In some cases,
the referenced information may change over time, so the ingestion pipeline
should be able to adapt to such changes to guarantee the currency and/or
correctness of the enrichment results.
  In this paper, we present a new data ingestion framework that supports data
ingestion at scale, enrichments requiring complex operations, and adaptiveness
to reference data changes. We explain how this framework has been built on top
of Apache AsterixDB and investigate its performance at scale under various
workloads.
