  Although system heterogeneity has been extensively studied in the past, there
is yet to be a study on measuring the impact of heterogeneity on system
performance. For this purpose, we propose a heterogeneity measure that can
characterize the impact of the heterogeneity of a system on its performance
behavior in terms of throughput or makespan. We develop a mathematical model to
characterize a heterogeneous system in terms of its task and machine
heterogeneity dimensions and then reduce it to a single value, called
Homogeneous Equivalent Execution Time (HEET), which represents the execution
time behavior of the entire system. We used AWS EC2 instances to implement a
real-world machine learning inference system. Performance evaluation of the
HEET score across different heterogeneous system configurations demonstrates
that HEET can accurately characterize the performance behavior of these
systems. In particular, the results show that our proposed method is capable of
predicting the true makespan of heterogeneous systems without online
evaluations with an average precision of 84%. This heterogeneity measure is
instrumental for solution architects to configure their systems proactively to
be sufficiently heterogeneous to meet their desired performance objectives.
