  End-to-end Network has become increasingly important in multi-tasking. One
prominent example of this is the growing significance of a driving perception
system in autonomous driving. This paper systematically studies an end-to-end
perception network for multi-tasking and proposes several key optimizations to
improve accuracy. First, the paper proposes efficient segmentation head and
box/class prediction networks based on weighted bidirectional feature network.
Second, the paper proposes automatically customized anchor for each level in
the weighted bidirectional feature network. Third, the paper proposes an
efficient training loss function and training strategy to balance and optimize
network. Based on these optimizations, we have developed an end-to-end
perception network to perform multi-tasking, including traffic object
detection, drivable area segmentation and lane detection simultaneously, called
HybridNets, which achieves better accuracy than prior art. In particular,
HybridNets achieves 77.3 mean Average Precision on Berkeley DeepDrive Dataset,
outperforms lane detection with 31.6 mean Intersection Over Union with 12.83
million parameters and 15.6 billion floating-point operations. In addition, it
can perform visual perception tasks in real-time and thus is a practical and
accurate solution to the multi-tasking problem. Code is available at
https://github.com/datvuthanh/HybridNets.
