  Weblogs, comprised of records detailing user activities on any website, offer
valuable insights into user preferences, behavior, and interests. Numerous
recommendation algorithms, employing strategies such as collaborative
filtering, content-based filtering, and hybrid methods, leverage the data mined
through these weblogs to provide personalized recommendations to users. Despite
the abundance of information available in these weblogs, identifying and
extracting pertinent information and key features from them necessitate
extensive engineering endeavors. The intricate nature of the data also poses a
challenge for interpretation, especially for non-experts. In this study, we
introduce a sophisticated and interactive recommendation framework denoted as
InteraRec, which diverges from conventional approaches that exclusively depend
on weblogs for recommendation generation. InteraRec framework captures
high-frequency screenshots of web pages as users navigate through a website.
Leveraging state-of-the-art multimodal large language models (MLLMs), it
extracts valuable insights into user preferences from these screenshots by
generating a textual summary based on predefined keywords. Subsequently, an
LLM-integrated optimization setup utilizes this summary to generate tailored
recommendations. Through our experiments, we demonstrate the effectiveness of
InteraRec in providing users with valuable and personalized offerings.
Furthermore, we explore the integration of session-based recommendation systems
into the InteraRec framework, aiming to enhance its overall performance.
Finally, we curate a new dataset comprising of screenshots from product web
pages on the Amazon website for the validation of the InteraRec framework.
Detailed experiments demonstrate the efficacy of the InteraRec framework in
delivering valuable and personalized recommendations tailored to individual
user preferences.
