  Large Language Models (LLMs) have garnered significant attention for their
ability to understand text and images, generate human-like text, and perform
complex reasoning tasks. However, their ability to generalize this advanced
reasoning with a combination of natural language text for decision-making in
dynamic situations requires further exploration. In this study, we investigate
how well LLMs can adapt and apply a combination of arithmetic and common-sense
reasoning, particularly in autonomous driving scenarios. We hypothesize that
LLMs hybrid reasoning abilities can improve autonomous driving by enabling them
to analyze detected object and sensor data, understand driving regulations and
physical laws, and offer additional context. This addresses complex scenarios,
like decisions in low visibility (due to weather conditions), where traditional
methods might fall short. We evaluated Large Language Models (LLMs) based on
accuracy by comparing their answers with human-generated ground truth inside
CARLA. The results showed that when a combination of images (detected objects)
and sensor data is fed into the LLM, it can offer precise information for brake
and throttle control in autonomous vehicles across various weather conditions.
This formulation and answers can assist in decision-making for auto-pilot
systems.
