  Computer vision applications in intelligent transportation systems (ITS) and
autonomous driving (AD) have gravitated towards deep neural network
architectures in recent years. While performance seems to be improving on
benchmark datasets, many real-world challenges are yet to be adequately
considered in research. This paper conducted an extensive literature review on
the applications of computer vision in ITS and AD, and discusses challenges
related to data, models, and complex urban environments. The data challenges
are associated with the collection and labeling of training data and its
relevance to real world conditions, bias inherent in datasets, the high volume
of data needed to be processed, and privacy concerns. Deep learning (DL) models
are commonly too complex for real-time processing on embedded hardware, lack
explainability and generalizability, and are hard to test in real-world
settings. Complex urban traffic environments have irregular lighting and
occlusions, and surveillance cameras can be mounted at a variety of angles,
gather dirt, shake in the wind, while the traffic conditions are highly
heterogeneous, with violation of rules and complex interactions in crowded
scenarios. Some representative applications that suffer from these problems are
traffic flow estimation, congestion detection, autonomous driving perception,
vehicle interaction, and edge computing for practical deployment. The possible
ways of dealing with the challenges are also explored while prioritizing
practical deployment.
