  Interacting and understanding with text heavy visual content with multiple
images is a major challenge for traditional vision models. This paper is on
enhancing vision models' capability to comprehend or understand and learn from
images containing a huge amount of textual information from the likes of
textbooks and research papers which contain multiple images like graphs, etc
and tables in them with different types of axes and scales. The approach
involves dataset preprocessing, fine tuning which is by using instructional
oriented data and evaluation. We also built a visual chat application
integrating CLIP for image encoding and a model from the Massive Text Embedding
Benchmark which is developed to consider both textual and visual inputs. An
accuracy of 96.71% was obtained. The aim of the project is to increase and also
enhance the advance vision models' capabilities in understanding complex visual
textual data interconnected data, contributing to multimodal AI.
