  Determining accurate bird's eye view (BEV) positions of objects and tracks in
a scene is vital for various perception tasks including object interactions
mapping, scenario extraction etc., however, the level of supervision required
to accomplish that is extremely challenging to procure. We propose a
light-weight, weakly supervised method to estimate 3D position of objects by
jointly learning to regress the 2D object detections and scene's depth
prediction in a single feed-forward pass of a network. Our proposed method
extends a center-point based single-shot object detector, and introduces a
novel object representation where each object is modeled as a BEV point
spatio-temporally, without the need of any 3D or BEV annotations for training
and LiDAR data at query time. The approach leverages readily available 2D
object supervision along with LiDAR point clouds (used only during training) to
jointly train a single network, that learns to predict 2D object detection
alongside the whole scene's depth, to spatio-temporally model object tracks as
points in BEV. The proposed method is computationally over $\sim$10x efficient
compared to recent SOTA approaches while achieving comparable accuracies on
KITTI tracking benchmark.
