  Low-precision is the first order knob for achieving higher Artificial
Intelligence Operations (AI-TOPS). However the algorithmic space for sub-8-bit
precision compute is diverse, with disruptive changes happening frequently,
making FPGAs a natural choice for Deep Neural Network inference, In this work
we present an FPGA-based accelerator for CNN inference acceleration. We use
{\it INT-8-2} compute (with {\it 8 bit} activation and {2 bit} weights) which
is recently showing promise in the literature, and which no known ASIC, CPU or
GPU natively supports today. Using a novel Adaptive Logic Module (ALM) based
design, as a departure from traditional DSP based designs, we are able to
achieve high performance measurement of 5 AI-TOPS for {\it Arria10} and project
a performance of 76 AI-TOPS at 0.7 TOPS/W for {\it Stratix10}. This exceeds
known CPU, GPU performance and comes close to best known ASIC (TPU) numbers,
while retaining the versatility of the FPGA platform for other applications.
