  Depth factorization and quantization have emerged as two of the principal
strategies for designing efficient deep convolutional neural network (CNN)
architectures tailored for low-power inference on the edge. However, there is
still little detailed understanding of how different depth factorization
choices affect the final, trained distributions of each layer in a CNN,
particularly in the situation of quantized weights and activations. In this
study, we introduce a progressive depth factorization strategy for efficient
CNN architecture exploration under quantization constraints. By algorithmically
increasing the granularity of depth factorization in a progressive manner, the
proposed strategy enables a fine-grained, low-level analysis of layer-wise
distributions. Thus enabling the gain of in-depth, layer-level insights on
efficiency-accuracy tradeoffs under fixed-precision quantization. Such a
progressive depth factorization strategy also enables efficient identification
of the optimal depth-factorized macroarchitecture design (which we will refer
to here as FactorizeNet) based on the desired efficiency-accuracy requirements.
