  Relational database management systems (RDBMS) are widely used for the
storage and retrieval of structured data. To derive insights beyond statistical
aggregation, we typically have to extract specific subdatasets from the
database using conventional database operations, and then apply deep neural
networks (DNN) training and inference on these respective subdatasets in a
separate machine learning system. The process can be prohibitively expensive,
especially when there are a combinatorial number of subdatasets extracted for
different analytical purposes. This calls for efficient in-database support of
advanced analytical methods In this paper, we introduce LEADS, a novel
SQL-aware dynamic model slicing technique to customize models for subdatasets
specified by SQL queries. LEADS improves the predictive modeling of structured
data via the mixture of experts (MoE) technique and maintains inference
efficiency by a SQL-aware gating network. At the core of LEADS is the
construction of a general model with multiple expert sub-models via MoE trained
over the entire database. This SQL-aware MoE technique scales up the modeling
capacity, enhances effectiveness, and preserves efficiency by activating only
necessary experts via the gating network during inference. Additionally, we
introduce two regularization terms during the training process of LEADS to
strike a balance between effectiveness and efficiency. We also design and build
an in-database inference system, called INDICES, to support end-to-end advanced
structured data analytics by non-intrusively incorporating LEADS onto
PostgreSQL. Our extensive experiments on real-world datasets demonstrate that
LEADS consistently outperforms baseline models, and INDICES delivers effective
in-database analytics with a considerable reduction in inference latency
compared to traditional solutions.
