  Large Language Vision Models (LLVMs) have demonstrated effectiveness in
processing internet videos, yet they struggle with the visually perplexing
dynamics present in Activities of Daily Living (ADL) due to limited pertinent
datasets and models tailored to relevant cues. To this end, we propose a
framework for curating ADL multiview datasets to fine-tune LLVMs, resulting in
the creation of ADL-X, comprising 100K RGB video-instruction pairs, language
descriptions, 3D skeletons, and action-conditioned object trajectories. We
introduce LLAVIDAL, an LLVM capable of incorporating 3D poses and relevant
object trajectories to understand the intricate spatiotemporal relationships
within ADLs. Furthermore, we present a novel benchmark, ADLMCQ, for quantifying
LLVM effectiveness in ADL scenarios. When trained on ADL-X, LLAVIDAL
consistently achieves state-of-the-art performance across all ADL evaluation
metrics. Qualitative analysis reveals LLAVIDAL's temporal reasoning
capabilities in understanding ADL. The link to the dataset is provided at:
https://adl-x.github.io/
