  Understanding human activity is a crucial yet intricate task in egocentric
vision, a field that focuses on capturing visual perspectives from the camera
wearer's viewpoint. Traditional methods heavily rely on representation learning
that is trained on a large amount of video data. However, a major challenge
arises from the difficulty of obtaining effective video representation. This
difficulty stems from the complex and variable nature of human activities,
which contrasts with the limited availability of data. In this study, we
introduce PALM, an approach that tackles the task of long-term action
anticipation, which aims to forecast forthcoming sequences of actions over an
extended period. Our method PALM incorporates an action recognition model to
track previous action sequences and a vision-language model to articulate
relevant environmental details. By leveraging the context provided by these
past events, we devise a prompting strategy for action anticipation using large
language models (LLMs). Moreover, we implement maximal marginal relevance for
example selection to facilitate in-context learning of the LLMs. Our
experimental results demonstrate that PALM surpasses the state-of-the-art
methods in the task of long-term action anticipation on the Ego4D benchmark. We
further validate PALM on two additional benchmarks, affirming its capacity for
generalization across intricate activities with different sets of taxonomies.
