  A multimodal AI agent is characterized by its ability to process and learn
from various types of data, including natural language, visual, and audio
inputs, to inform its actions. Despite advancements in large language models
that incorporate visual data, such as GPT-4V, effectively translating
image-based data into actionable outcomes for AI agents continues to be
challenging. In this paper, we introduce a multimodal model that incorporates
the concept of functional token specifically designed for AI agent
applications. To ensure compatibility with edge devices, our model is optimized
to a compact size of less than 1B parameters. Like GPT-4, our model can process
both English and Chinese. We demonstrate that this model is capable of
operating efficiently on a wide range of edge devices, including as constrained
as a Raspberry Pi.
