  The cornerstone of autonomous vehicles (AV) is a solid perception system,
where camera encoders play a crucial role. Existing works usually leverage
pre-trained Convolutional Neural Networks (CNN) or Vision Transformers (ViTs)
designed for general vision tasks, such as image classification, segmentation,
and 2D detection. Although those well-known architectures have achieved
state-of-the-art accuracy in AV-related tasks, e.g., 3D Object Detection, there
remains significant potential for improvement in network design due to the
nuanced complexities of industrial-level AV dataset. Moreover, existing public
AV benchmarks usually contain insufficient data, which might lead to inaccurate
evaluation of those architectures.To reveal the AV-specific model insights, we
start from a standard general-purpose encoder, ConvNeXt and progressively
transform the design. We adjust different design parameters including width and
depth of the model, stage compute ratio, attention mechanisms, and input
resolution, supported by systematic analysis to each modifications. This
customization yields an architecture optimized for AV camera encoder achieving
8.79% mAP improvement over the baseline. We believe our effort could become a
sweet cookbook of image encoders for AV and pave the way to the next-level
drive system.
