  The emergence of big data has caused a dramatic shift in the operating regime
for optimization algorithms. The performance bottleneck, which used to be
computations, is now often communications. Several gradient compression
techniques have been proposed to reduce the communication load at the price of
a loss in solution accuracy. Recently, it has been shown how compression errors
can be compensated for in the optimization algorithm to improve the solution
accuracy. Even though convergence guarantees for error-compensated algorithms
have been established, there is very limited theoretical support for
quantifying the observed improvements in solution accuracy. In this paper, we
show that Hessian-aided error compensation, unlike other existing schemes,
avoids the accumulation of compression errors on quadratic problems. We also
present strong convergence guarantees of Hessian-based error compensation for
stochastic gradient descent. Our numerical experiments highlight the benefits
of Hessian-based error compensation, and demonstrate that similar convergence
improvements are attained when only a diagonal Hessian approximation is used.
