  Geo-distributed data analytics are increasingly common to derive useful
information in large organisations. Naive extension of existing cluster-scale
data analytics systems to the scale of geo-distributed data centers faces
unique challenges including WAN bandwidth limits, regulatory constraints,
changeable/unreliable runtime environment, and monetary costs. Our goal in this
work is to develop a practical geo-distribued data analytics system that (1)
employs an intelligent mechanism for jobs to efficiently utilize (adjust to)
the resources (changeable environment) across data centers; (2) guarantees the
reliability of jobs due to the possible failures; and (3) is generic and
flexible enough to run a wide range of data analytics jobs without requiring
any changes.
  To this end, we present a new, general geo-distributed data analytics system,
HOUTU, that is composed of multiple autonomous systems, each operating in a
sovereign data center. HOUTU maintains a job manager (JM) for a geo-distributed
job in each data center, so that these replicated JMs could individually and
cooperatively manage resources and assign tasks. Our experiments on the
prototype of HOUTU running across four Alibaba Cloud regions show that HOUTU
provides nearly efficient job performance as in the existing centralized
architecture, and guarantees reliable job executions when facing failures.
