Polymodal Menus: A Model-based Approach for Designing Multimodal Adaptive Menus for Small Screens
(2017) Proceedings of the ACM on Human-Computer Interaction - EICS — Vol. 1, n° 1, p. 1-19 (2017)
Files
Bouzit-EICS2017.pdf
Open Access - Adobe PDF
- 1.33 MB
Details
- Authors
- Abstract
- This paper presents a model-based approach for designing Polymodal Menus, a new type of multimodal adaptive menu for small screen graphical user interfaces where item selection and adaptivity are responsive to more than one interaction modality: a menu item can be selected graphically, tactilely, vocally, gesturally, or any combination of them. The prediction window containing the most predicted menu items by assignment, equivalence, or redundancy is made equally adaptive. For this purpose, an adaptive menu model maintains the most predictable menu items according to various prediction methods. This model is exploited throughout various steps defined on a new Adaptivity Design Space based on a Perception-Decision-Action cycle com-ing from cognitive psychology. A user experiment compares four conditions of Polymodal Menus (graphical, vocal, gestural, and mixed) in terms of menu selection time, error rate, user subjective satisfaction and user preference, when item prediction has a low or high level of accuracy. Polymodal Menus offer alternative input/output modalities to select menu items in various contexts of use, especially when graphical modality is constrained.
- Affiliations
Citations
Bouzit, S., Calvary, G., Chêne, D., & Vanderdonckt, J. (2017). Polymodal Menus: A Model-based Approach for Designing Multimodal Adaptive Menus for Small Screens. Proceedings of the ACM on Human-Computer Interaction - EICS, 1(1), 1-19. https://doi.org/10.1145/3099585 (Original work published 2017)