  We propose end-to-end document classification and key information extraction
(KIE) for automating document processing in forms. Through accurate document
classification we harness known information from templates to enhance KIE from
forms. We use text and layout encoding with a cosine similarity measure to
classify visually-similar documents. We then demonstrate a novel application of
mixed integer programming by using assignment optimization to extract key
information from documents. Our approach is validated on an in-house dataset of
noisy scanned forms. The best performing document classification approach
achieved 0.97 f1 score. A mean f1 score of 0.94 for the KIE task suggests there
is significant potential in applying optimization techniques. Abation results
show that the method relies on document preprocessing techniques to mitigate
Type II errors and achieve optimal performance.
