  Assist-as-needed (AAN) control aims at promoting therapeutic outcomes in
robot-assisted rehabilitation by encouraging patients' active participation.
Impedance control is used by most AAN controllers to create a compliant force
field around a target motion to ensure tracking accuracy while allowing
moderate kinematic errors. However, since the parameters governing the shape of
the force field are often tuned manually or adapted online based on simplistic
assumptions about subjects' learning abilities, the effectiveness of
conventional AAN controllers may be limited. In this work, we propose a novel
adaptive AAN controller that is capable of autonomously reshaping the force
field in a phase-dependent manner according to each individual's motor
abilities and task requirements. The proposed controller consists of a modified
Policy Improvement with Path Integral algorithm, a model-free, sampling-based
reinforcement learning method that learns a subject-specific impedance
landscape in real-time, and a hierarchical policy parameter evaluation
structure that embeds the AAN paradigm by specifying performance-driven
learning goals. The adaptability of the proposed control strategy to subjects'
motor responses and its ability to promote short-term motor adaptations are
experimentally validated through treadmill training sessions with able-bodied
subjects who learned altered gait patterns with the assistance of a powered
ankle-foot orthosis.
