  Several fundamental changes in technology indicate domain-specific hardware
and software co-design is the only path left. In this context, architecture,
system, data management, and machine learning communities pay greater attention
to innovative big data and AI algorithms, architecture, and systems.
Unfortunately, complexity, diversity, frequently-changed workloads, and rapid
evolution of big data and AI systems raise great challenges. First, the
traditional benchmarking methodology that creates a new benchmark or proxy for
every possible workload is not scalable, or even impossible for Big Data and AI
benchmarking. Second, it is prohibitively expensive to tailor the architecture
to characteristics of one or more application or even a domain of applications.
We consider each big data and AI workload as a pipeline of one or more classes
of units of computation performed on different initial or intermediate data
inputs, each class of which we call a data motif. On the basis of our previous
work that identifies eight data motifs taking up most of the run time of a wide
variety of big data and AI workloads, we propose a scalable benchmarking
methodology that uses the combination of one or more data motifs---to represent
diversity of big data and AI workloads. Following this methodology, we present
a unified big data and AI benchmark suite---BigDataBench 4.0, publicly
available from~\url{http://prof.ict.ac.cn/BigDataBench}. This unified benchmark
suite sheds new light on domain-specific hardware and software co-design:
tailoring the system and architecture to characteristics of the unified eight
data motifs other than one or more application case by case. Also, for the
first time, we comprehensively characterize the CPU pipeline efficiency using
the benchmarks of seven workload types in BigDataBench 4.0.
