  Significant advancements has recently been achieved in the field of
multi-modal large language models (MLLMs), demonstrating their remarkable
capabilities in understanding and reasoning across diverse tasks. However,
these models are often trained for specific tasks and rely on task-specific
input-output formats, limiting their applicability to a broader range of tasks.
This raises a fundamental question: Can we develop a unified approach to
represent and handle different multi-modal tasks to maximize the
generalizability of MLLMs? In this paper, we propose UnifiedMLLM, a
comprehensive model designed to represent various tasks using a unified
representation. Our model exhibits strong capabilities in comprehending the
implicit intent of user instructions and preforming reasoning. In addition to
generating textual responses, our model also outputs task tokens and grounding
tokens, serving as indicators of task types and task granularity. These outputs
are subsequently routed through the task router and directed to specific expert
models for task completion. To train our model, we construct a task-specific
dataset and an 100k multi-task dataset encompassing complex scenarios.
Employing a three-stage training strategy, we equip our model with robust
reasoning and task processing capabilities while preserving its generalization
capacity and knowledge reservoir. Extensive experiments showcase the impressive
performance of our unified representation approach across various tasks,
surpassing existing methodologies. Furthermore, our approach exhibits
exceptional scalability and generality. Our code, model, and dataset will be
available at \url{https://github.com/lzw-lzw/UnifiedMLLM}.
