  The rapid advancement of the automotive industry towards automated and
semi-automated vehicles has rendered traditional methods of vehicle
interaction, such as touch-based and voice command systems, inadequate for a
widening range of non-driving related tasks, such as referencing objects
outside of the vehicle. Consequently, research has shifted toward gestural
input (e.g., hand, gaze, and head pose gestures) as a more suitable mode of
interaction during driving. However, due to the dynamic nature of driving and
individual variation, there are significant differences in drivers' gestural
input performance. While, in theory, this inherent variability could be
moderated by substantial data-driven machine learning models, prevalent
methodologies lean towards constrained, single-instance trained models for
object referencing. These models show a limited capacity to continuously adapt
to the divergent behaviors of individual drivers and the variety of driving
scenarios. To address this, we propose \textit{IcRegress}, a novel
regression-based incremental learning approach that adapts to changing behavior
and the unique characteristics of drivers engaged in the dual task of driving
and referencing objects. We suggest a more personalized and adaptable solution
for multimodal gestural interfaces, employing continuous lifelong learning to
enhance driver experience, safety, and convenience. Our approach was evaluated
using an outside-the-vehicle object referencing use case, highlighting the
superiority of the incremental learning models adapted over a single trained
model across various driver traits such as handedness, driving experience, and
numerous driving conditions. Finally, to facilitate reproducibility, ease
deployment, and promote further research, we offer our approach as an
open-source framework at \url{https://github.com/amrgomaaelhady/IcRegress}.
