  The traditional mode of recording faults in heavy factory equipment has been
via hand marked inspection sheets, wherein a machine engineer manually marks
the faulty machine regions on a paper outline of the machine. Over the years,
millions of such inspection sheets have been recorded and the data within these
sheets has remained inaccessible. However, with industries going digital and
waking up to the potential value of fault data for machine health monitoring,
there is an increased impetus towards digitization of these hand marked
inspection records. To target this digitization, we propose a novel visual
pipeline combining state of the art deep learning models, with domain knowledge
and low level vision techniques, followed by inference of visual relationships.
Our framework is robust to the presence of both static and non-static
background in the document, variability in the machine template diagrams,
unstructured shape of graphical objects to be identified and variability in the
strokes of handwritten text. The proposed pipeline incorporates a capsule and
spatial transformer network based classifier for accurate text reading, and a
customized CTPN network for text detection in addition to hybrid techniques for
arrow detection and dialogue cloud removal. We have tested our approach on a
real world dataset of 50 inspection sheets for large containers and boilers.
The results are visually appealing and the pipeline achieved an accuracy of
87.1% for text detection and 94.6% for text reading.
