The openinterface framework: a tool for multimodal interaction.

ABSTRACT

The area of multimodal interaction has expanded rapidly. However, the implementation of multimodal systems still remains a difficult task. Addressing this problem, we describe the OpenInterface (OI) framework, a component-based tool for rapidly developing multimodal input interfaces. The OI underlying conceptual component model includes both generic and tailored components. In addition, to enable the rapid exploration of the multimodal design space for a given system, we need to capitalize on past experiences and include a large set of multimodal interaction techniques, their specifications and documentations. In this work-in-progress report, we present the current state of the OI framework and the two exploratory test-beds developed using the OpenInterface Interaction Development Environment.

References

1. Benoit, A., Bonnaud, L., Caplier, A., Damousis, I., Tzovaras, D., Jourde, E., Nigay, L., Serrano, M. and Lawson, J-Y. 2006. Multimodal Signal Processing and Interaction for a Driving Simulation: Component-based Architecture. Journal on Multimodal User Interfaces,1, 1,49--58.
2. Bouchet, J., Nigay, L. (2004). ICARE: A Component-Based Approach for the Design and Development of Multimodal Interfaces. Extended Abstracts CHI'04, ACM, pp. 1325--1328.
3. Buxton, W. (1983). Lexical and Pragmatic Considerations of Input Structures. ACM SIGGRAPH CG, 17(1), pp. 31--37
4. Dietz, P. and Leigh, D. (2001). DiamondTouch: a multi-user touch technology. Proc. of the 14th annual ACM symposium on User interfaces software and technologies, UIST'01, pp. 219--226.
5. Dragicevic, P., and Fekete, J. D. (2001). Input device selection and interaction configuration with ICON. Joint Proc. of IHM'01 and HCI'01,Springer Verlag, pp. 543--558.
6. Foley, J., Wallace, V. L. and Chan, P. (1984) . The human factors of graphics interaction techniques. IEEE Computer Graphics and Applications, 11, pp. 13--48
7. Greenberg, S. and Fitchett, C. (2001). Phidgets: Easy Development of Physical Interfaces throught Physical Widgets. In Proc. of the UIST 2001 Annual ACM Symposium on User Interface Software and Technology, ACM Press
8. Interface-Z, www.interface-z.com.
9. Maynes-Aminzade, D., Winograd, T., and Igarashi, T. (2007). Eyepatch: Prototyping Camera-based interaction through Examples. In Proc. UIST'07
10. Nigay, L., Coutaz, J. (1997). Multifeature Systems: The CARE Properties and Their Impact on Software Design. Intelligence and Multimodality in Multimedia Interfaces, AAAI Press.
11. OpenInterface European project. IST Framework 6 STREP funded by the European Commission (FP6- 35182). www.oi-project.org.
12. Williamson, J., Murray-Smith, R., and Hugues, S. (2007). Shoogle: Ecitatory Multimodal Interaction on Mobile Devices. In Proc. CHI'07, ACM Press, pp. 121--1

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction devices

      • Touch screens

    • Interaction paradigms

    • Interactive systems and tools

      • User interface management systems

  • Interaction design

    • Interaction design process and methods

      • Interface design prototyping

• Software and its engineering

  • Software notations and tools

    • Development frameworks and environments

      • Integrated and visual development environments