  Inspired by the unique neurophysiology of the octopus, we propose a
hierarchical framework that simplifies the coordination of multiple soft arms
by decomposing control into high-level decision making, low-level motor
activation, and local reflexive behaviors via sensory feedback. When evaluated
in the illustrative problem of a model octopus foraging for food, this
hierarchical decomposition results in significant improvements relative to
end-to-end methods. Performance is achieved through a mixed-modes approach,
whereby qualitatively different tasks are addressed via complementary control
schemes. Here, model-free reinforcement learning is employed for high-level
decision-making, while model-based energy shaping takes care of arm-level motor
execution. To render the pairing computationally tenable, a novel
neural-network energy shaping (NN-ES) controller is developed, achieving
accurate motions with time-to-solutions 200 times faster than previous
attempts. Our hierarchical framework is then successfully deployed in
increasingly challenging foraging scenarios, including an arena littered with
obstacles in 3D space, demonstrating the viability of our approach.
