  In mobile edge computing (MEC), resource scheduling is crucial to task
requests' performance and service providers' cost, involving multi-layer
heterogeneous scheduling decisions. Existing schedulers typically adopt static
timescales to regularly update scheduling decisions of each layer, without
adaptive adjustment of timescales for different layers, resulting in
potentially poor performance in practice.
  We notice that the adaptive timescales would significantly improve the
trade-off between the operation cost and delay performance. Based on this
insight, we propose EdgeTimer, the first work to automatically generate
adaptive timescales to update multi-layer scheduling decisions using deep
reinforcement learning (DRL). First, EdgeTimer uses a three-layer hierarchical
DRL framework to decouple the multi-layer decision-making task into a hierarchy
of independent sub-tasks for improving learning efficiency. Second, to cope
with each sub-task, EdgeTimer adopts a safe multi-agent DRL algorithm for
decentralized scheduling while ensuring system reliability. We apply EdgeTimer
to a wide range of Kubernetes scheduling rules, and evaluate it using
production traces with different workload patterns. Extensive trace-driven
experiments demonstrate that EdgeTimer can learn adaptive timescales,
irrespective of workload patterns and built-in scheduling rules. It obtains up
to 9.1x more profit than existing approaches without sacrificing the delay
performance.
