Visual-inertial prediction of terrain-specific human locomotion modes
(2022)
Files
these.pdf
Open Access - Adobe PDF
- 25.83 MB
Details
- Authors
- Supervisors
- Ronsse, Renaud
- Abstract
- The exponential growth of technology opened the door for a new generation of locomotion assistive devices (e.g. prostheses and exoskeletons) to emerge. The latest generation is lighter weight due to the emergence of smart and composite materials, and advanced actuators and sensors. Controlling these devices to compensate for limb loss or walking disorder requires the generation of various gait patterns for different locomotion tasks like stairs ascending, descending, and level ground walking. However, this requires that these devices be able to detect the actions and intentions of their human user when traversing from one type of terrain to another. Such transition requires switching to the new task in a timely manner; otherwise, the user is more vulnerable to falls and injuries. Typical solutions to detect a new locomotion task are based either on sensors that measure the user’s kinematics or on a voluntary action of the user, such has tilting their chest or pushing a button. While the former suffers from a delay, the latter provides an unnatural way of behaving during locomotion. Recently, alternative methods have been proposed that uses wearable ranging sensors or camera. However, the implemented approaches for terrain identification with such sensors have been only validated in single indoor and/or outdoor path(s). Also, the system detection ability in a path being partially occluded by other walkers has not been addressed yet. Last but not least, the detection results are obtained based on a pre-specified region of interest in front of the user, without applying extra processing to predict the terrain information for several upcoming steps. In this thesis, we introduce a computer vision-based terrain detection algorithm that relies on a depth camera attached to the user’s chest. The camera, therefore, could capture the scene in front of the user in a point cloud data format, while the algorithm applies several steps on the captured data in order to achieve environmental feature extraction. The features are passed to a trained classifier that predicts so-called locomotion affordances. The algorithm further processes the detected affordances to obtain the terrain type and its geometrical features for three steps in advance with the adaptation to the instantaneous user’s step size. Additionally, the system can provide an online estimate of the walking stride length, speed, and turning angle based on a single inertial sensor that is built within the depth camera. The estimated speed and turning angle pave the way to an activity tracking method that could be beneficial to evaluate the assistive device behavior across various environmental terrains without the need for dedicated equipment. The algorithm was validated with 8 participants in several indoor and outdoor paths with a variety of terrain types. The validation process consisted, also, in conducting the experiments with clear and partially occluded path conditions. The system accomplishes remarkable results with a grand average detection accuracy above 90% over all locomotion modes with clear path conditions. Moreover, it demonstrates the ability to recognize various terrain types even with partially occluded path conditions at least for a single step in front of the user. Finally, the estimated geometrical features of most terrain types were closed to real observed values, even with partially covered paths. Similar low error rates were observed for the walking speed and step size estimation approaches when evaluated on several participants walking across paths designed for that purpose. The full package which consists of terrain detection for multiple steps, geometrical features, speed, and turning angle estimations can provide a valuable tool for a wide variety of disabled people including amputees with an active prosthesis, visually impaired, or blind people who face special locomotion difficulties in daily life.
- Affiliations
Citations
Al-Dabbagh, A. (2022). Visual-inertial prediction of terrain-specific human locomotion modes. https://hdl.handle.net/2078.5/27792