  The development of model compression is continuously motivated by the
evolution of various neural network accelerators with ASIC or FPGA. On the
algorithm side, the ultimate goal of quantization or pruning is accelerating
the expensive DNN computations on low-power hardware. However, such a
"design-and-deploy" workflow faces under-explored challenges in the current
hardware-algorithm co-design community. First, although the state-of-the-art
quantization algorithm can achieve low precision with negligible degradation of
accuracy, the latest deep learning framework (e.g., PyTorch) can only support
non-customizable 8-bit precision, data format, and parameter extraction.
Secondly, the objective of quantization is to enable the computation with
low-precision data. However, the current SoTA algorithm treats the quantized
integer as an intermediate result, while the final output of the quantizer is
the "discretized" floating-point values, ignoring the practical needs and
adding additional workload to hardware designers for integer parameter
extraction and layer fusion. Finally, the compression toolkits designed by the
industry are constrained to their in-house product or a handful of algorithms.
The limited degree of freedom in the current toolkit and the under-explored
customization hinder the prototype ASIC or FPGA-based accelerator design. To
resolve these challenges, we propose Torch2Chip, an open-sourced, fully
customizable, and high-performance toolkit that supports user-designed
compression followed by automatic model fusion and parameter extraction.
Torch2Chip incorporates the hierarchical design workflow, and the
user-customized compression algorithm will be directly packed into the
deployment-ready format for prototype chip verification with either CNN or
vision transformer (ViT). The code is available at
https://github.com/SeoLabCornell/torch2chip.
