  Artificial Intelligence (AI) systems planned for deployment in real-world
applications frequently are researched and developed in closed simulation
environments where all variables are controlled and known to the simulator or
labeled benchmark datasets are used. Transition from these simulators,
testbeds, and benchmark datasets to more open-world domains poses significant
challenges to AI systems, including significant increases in the complexity of
the domain and the inclusion of real-world novelties; the open-world
environment contains numerous out-of-distribution elements that are not part in
the AI systems' training set. Here, we propose a path to a general,
domain-independent measure of domain complexity level. We distinguish two
aspects of domain complexity: intrinsic and extrinsic. The intrinsic domain
complexity is the complexity that exists by itself without any action or
interaction from an AI agent performing a task on that domain. This is an
agent-independent aspect of the domain complexity. The extrinsic domain
complexity is agent- and task-dependent. Intrinsic and extrinsic elements
combined capture the overall complexity of the domain. We frame the components
that define and impact domain complexity levels in a domain-independent light.
Domain-independent measures of complexity could enable quantitative predictions
of the difficulty posed to AI systems when transitioning from one testbed or
environment to another, when facing out-of-distribution data in open-world
tasks, and when navigating the rapidly expanding solution and search spaces
encountered in open-world domains.
