  Large multimodal models (LMMs) extend large language models (LLMs) with
multi-sensory skills, such as visual understanding, to achieve stronger generic
intelligence. In this paper, we analyze the latest model, GPT-4V(ision), to
deepen the understanding of LMMs. The analysis focuses on the intriguing tasks
that GPT-4V can perform, containing test samples to probe the quality and
genericity of GPT-4V's capabilities, its supported inputs and working modes,
and the effective ways to prompt the model. In our approach to exploring
GPT-4V, we curate and organize a collection of carefully designed qualitative
samples spanning a variety of domains and tasks. Observations from these
samples demonstrate that GPT-4V's unprecedented ability in processing
arbitrarily interleaved multimodal inputs and the genericity of its
capabilities together make GPT-4V a powerful multimodal generalist system.
Furthermore, GPT-4V's unique capability of understanding visual markers drawn
on input images can give rise to new human-computer interaction methods such as
visual referring prompting. We conclude the report with in-depth discussions on
the emerging application scenarios and the future research directions for
GPT-4V-based systems. We hope that this preliminary exploration will inspire
future research on the next-generation multimodal task formulation, new ways to
exploit and enhance LMMs to solve real-world problems, and gaining better
understanding of multimodal foundation models. Finally, we acknowledge that the
model under our study is solely the product of OpenAI's innovative work, and
they should be fully credited for its development. Please see the GPT-4V
contributions paper for the authorship and credit attribution:
https://cdn.openai.com/contributions/gpt-4v.pdf
