  On a daily basis, data centers process huge volumes of data backed by the
proliferation of inexpensive hard disks. Data stored in these disks serve a
range of critical functional needs from financial, and healthcare to aerospace.
As such, premature disk failure and consequent loss of data can be
catastrophic. To mitigate the risk of failures, cloud storage providers perform
condition-based monitoring and replace hard disks before they fail. By
estimating the remaining useful life of hard disk drives, one can predict the
time-to-failure of a particular device and replace it at the right time,
ensuring maximum utilization whilst reducing operational costs. In this work,
large-scale predictive analyses are performed using severely skewed health
statistics data by incorporating customized feature engineering and a suite of
sequence learners. Past work suggests using LSTMs as an excellent approach to
predicting remaining useful life. To this end, we present an encoder-decoder
LSTM model where the context gained from understanding health statistics
sequences aid in predicting an output sequence of the number of days remaining
before a disk potentially fails. The models developed in this work are trained
and tested across an exhaustive set of all of the 10 years of S.M.A.R.T. health
data in circulation from Backblaze and on a wide variety of disk instances. It
closes the knowledge gap on what full-scale training achieves on thousands of
devices and advances the state-of-the-art by providing tangible metrics for
evaluation and generalization for practitioners looking to extend their
workflow to all years of health data in circulation across disk manufacturers.
The encoder-decoder LSTM posted an RMSE of 0.83 during training and 0.86 during
testing over the exhaustive 10 year data while being able to generalize
competitively over other drives from the Seagate family.
