Gestural-Vocal Coordinated Interaction on Large Displays

Parthiban, Vik;Maes, Pattie;Sellier, Quentin;Sluÿters, Arthur;Vanderdonckt, Jean
(2022) ACM SIGCHI Symposium on Engineering Interactive Computing Systems (EICS ’22) — Location: Sophia Antipolis, France (21.June.2022)

Files

EvaLUI-EICS2022.pdf
  • Open Access
  • Adobe PDF
  • 4.5 MB
Download

Details

Authors
  • Parthiban, VikMassachusetts Institute of Technology, MIT Media Lab
    Author
  • Maes, PattieMassachusetts Institute of Technology, MIT Media Lab
    Author
  • Sellier, Quentinorcid-logoUCLouvain
    Author
  • Sluÿters, Arthurorcid-logoUCLouvain
    Author
  • Vanderdonckt, Jeanorcid-logoUCLouvain
    Author
Abstract
On large displays, using keyboard and mouse input is challenging because small mouse movements do not scale well with the size of the display and individual elements on screen. We present “Large User Interface” (LUI), which coordinates gestural and vocal interaction to increase the range of dynamic surface area of interactions possible on large displays. The interface leverages real-time continuous feedback of free-handed gestures and voice to control a set of applications such as: photos, videos, 3D models, maps, and a gesture keyboard. Utilizing a single stereo camera and voice assistant, LUI does not require calibration or many sensors to operate, and it can be easily installed and deployed. We report results from user studies where participants found LUI efficient, learnable with minimal instruction, and preferred it to point-and-click interfaces.
Affiliations

Citations

  • APA
  • Chicago
  • FWB

Parthiban, V., Maes, P., Sellier, Q., Sluÿters, A., & Vanderdonckt, J. (2022). Gestural-Vocal Coordinated Interaction on Large Displays. Companion of the 2022 ACM SIGCHI Symposium on Engineering Interactive Computing Systems, p. 7. https://doi.org/10.1145/3531706.3536457