  Environment awareness is crucial for enhancing walking safety and stability
of amputee wearing powered prosthesis when crossing uneven terrains such as
stairs and obstacles. However, existing environmental perception systems for
prosthesis only provide terrain types and corresponding parameters, which fails
to prevent potential collisions when crossing uneven terrains and may lead to
falls and other severe consequences. In this paper, a visual-inertial motion
estimation approach is proposed for prosthesis to perceive its movement and the
changes of spatial relationship between the prosthesis and uneven terrain when
traversing them. To achieve this, we estimate the knee motion by utilizing a
depth camera to perceive the environment and align feature points extracted
from stairs and obstacles. Subsequently, an error-state Kalman filter is
incorporated to fuse the inertial data into visual estimations to reduce the
feature extraction error and obtain a more robust estimation. The motion of
prosthetic joint and toe are derived using the prosthesis model parameters.
Experiment conducted on our collected dataset and stair walking trials with a
powered prosthesis shows that the proposed method can accurately tracking the
motion of the human leg and prosthesis with an average root-mean-square error
of toe trajectory less than 5 cm. The proposed method is expected to enable the
environmental adaptive control for prosthesis, thereby enhancing amputee's
safety and mobility in uneven terrains.
