Modelling of User Interfaces based on State-Charts to Ease the Test and Evaluation of Multimodal Interactions

Sebastian Feuerstack and Mauro Dos Santos Anjo and Jessica Colnago and Ednaldo Pizzolato
Proceedings of INFORMATIK 2011
New forms of interactions such as the gesture-based control of interfaces could enable interaction in situations where hardware controls are missing and support impaired people where other controls fails. The rich spectrum of combining hand pos- tures with movements offers great interaction possibilities but requires extensive user testing to figure out a user interface navigation control with a sufficient performance and a low error rate. In this paper we describe a model-based interface design based on assembling in- teractors and multimodal mappings to design multimodal interfaces. We propose to use state charts for the rapid generation of different user interface controls to acceler- ate user testing. First we describe how our approach can be applied to design direct manipulation interfaces that rely on a mouse-based interface control. Then we present how we applied the interactor based design to quickly generate several variants of a gesture-based interface navigation control to demonstrate our approach. We compare the three most promising variants in a user test and report about the test results.
4 / 2011
Hans-Ulrich Hei�, Peter Pepper and Holger Schlingloff, J�rg Schneider