  This paper presents ColonyOS, an open-source meta-operating system designed
to improve integration and utilization of diverse computing platforms,
including IoT, edge, cloud, and HPC. Operating as an overlay, ColonyOS can
interface with a wide range of computing environments, fostering creation of
so-called compute continuums. This makes it possible to develop AI workflows
and applications that can operate across platforms. At its core, ColonyOS
consists of distributed executors that integrate with various underlying
platforms based on a distributed microservice architecture. These executors
collectively form a colony, serving as a unified computing unit. To enable
secure integration of various platforms, each colony is provisioned with
precisely the resources needed, and all communication is confined within the
colony governed by a strict zero-trust security protocol. Interaction with
ColonyOS is done by submitting functional meta-descriptions of computational
tasks, called function specifications. These are sent to a Colonies server,
which acts as intermediary between applications and the executors. Upon
assignment, an executor interprets the meta-description and translates it into
an executable format, e.g. a Kubernetes deployment description, a Slurm script,
or a direct function call within the executor. Furthermore, a built-in
meta-file system enables data synchronization directives to be included in
meta-descriptions, enabling seamless data management across platforms.
Ultimately, ColonyOS paves the way for development of hyper-distributed
applications and workflows, which can seamlessly operate in a computing
continuum. The paper describes design principles and implementation details of
ColonyOS.
