  In an overloaded FaaS cluster, individual worker nodes strain under
lengthening queues of requests. Although the cluster might be eventually
horizontally-scaled, adding a new node takes dozens of seconds. As serving
applications are tuned for tail serving latencies, and these greatly increase
under heavier loads, the current workaround is resource over-provisioning. In
fact, even though a service can withstand a steady load of, e.g., 70% CPU
utilization, the autoscaler is triggered at, e.g., 30-40% (thus the service
uses twice as many nodes as it would be needed). We propose an alternative: a
worker-level method handling heavy load without increasing the number of nodes.
  FaaS executions are not interactive, compared to, e.g., text editors:
end-users do not benefit from the CPU allocated to processes often, yet for
short periods. Inspired by scheduling methods for High Performance Computing,
we take a radical step of replacing the classic OS preemption by (1) queuing
requests based on their historical characteristics; (2) once a request is being
processed, setting its CPU limit to exactly one core (with no CPU
oversubscription).
  We extend OpenWhisk and measure the efficiency of the proposed solutions
using the SeBS benchmark. In a loaded system, our method decreases the average
response time by a factor of 4. The improvement is even higher for shorter
requests, as the average stretch is decreased by a factor of 18. This leads us
to show that we can provide better response-time statistics with 3 machines
compared to a 4-machine baseline.
