  In today's landscape, smartphones have evolved into hubs for hosting a
multitude of deep learning models aimed at local execution. A key realization
driving this work is the notable fragmentation among these models,
characterized by varied architectures, operators, and implementations. This
fragmentation imposes a significant burden on the comprehensive optimization of
hardware, system settings, and algorithms.
  Buoyed by the recent strides in large foundation models, this work introduces
a pioneering paradigm for mobile AI: a collaborative management approach
between the mobile OS and hardware, overseeing a foundational model capable of
serving a broad spectrum of mobile AI tasks, if not all. This foundational
model resides within the NPU and remains impervious to app or OS revisions,
akin to firmware. Concurrently, each app contributes a concise, offline
fine-tuned "adapter" tailored to distinct downstream tasks. From this concept
emerges a concrete instantiation known as \sys. It amalgamates a curated
selection of publicly available Large Language Models (LLMs) and facilitates
dynamic data flow. This concept's viability is substantiated through the
creation of an exhaustive benchmark encompassing 38 mobile AI tasks spanning 50
datasets, including domains such as Computer Vision (CV), Natural Language
Processing (NLP), audio, sensing, and multimodal inputs. Spanning this
benchmark, \sys unveils its impressive performance. It attains accuracy parity
in 85\% of tasks, demonstrates improved scalability in terms of storage and
memory, and offers satisfactory inference speed on Commercial Off-The-Shelf
(COTS) mobile devices fortified with NPU support. This stands in stark contrast
to task-specific models tailored for individual applications.
