Seminar Multimodal perception and interaction

Website:website containing additional information
Course code:INFOMMPI
Credits:7.5 ECTS
Period:periode 4 (week 17 t/m 27, dwz 23-4-2007 t/m 6-7-2007; herkansing week 35)
Timeslot:D
Participants:up till now 7 subscriptions
Schedule:Dit is een oud rooster!
formgrouptimeweekroomteacher
seminar   ma 09-1722 BBL-468 Peter Werkhoven
  
di 09-1722 BBL-468
wo 09-1722 BBL-468
do 09-1722 BBL-468
vr 09-1717 BBL-468
22 BBL-468
26 BBL-468
Contents: The seminar "Multimodal perception & Interaction" is about the fascinating world of human visual, auditory and tactile perception and the use of its potential in designing novel interfaces for interacting with virtual worlds. We will read and discuss papers that address the following topics:
  1. some basics of visual, auditory and tactile perception and the effects of combining them (multimodal presentation and interface design guidelines)
  2. visual communication interfaces in virtual worlds (effects of non-verbal facial communication)
  3. navigation interfaces in virtual worlds (head tracked visualization and disorientation issues)
  4. manipulation in virtual worlds (traditional mouse-cursor interfaces versus virtual hand control)
  5. virtual worlds through mobile displays (scrolling interfaces versus virtual windows)
  6. emerging interface technology (synaesthetic media and brain machine interfaces)
If possible we will organize a visit to the TNO Human Factors laboratory in Soesterberg.
Literature:The literature consists of 16 research papers, all of which are available in electronic form.
Course form:All of the 8 participants will read all 16 papers, write down issues for discussion for each paper and prepare the presentation of two of the 16 papers. We will get together on three days:
  • April 27 (9.00-17.00): Werkhoven will introduce the subjects of the various papers and their relations
    (participants read papers 1 to 8, send in discussion issues and prepare one paper for presentation)
  • June 01 (9.00-17.00): each participant presents his/her paper and discusses this paper with the group
    (participants read papers 9-16, send in discussion issues and prepare one paper for presentation)
  • June 29 (9.00-17.00): each participant presents his/her paper and discusses this paper with the group
Exam form:The grade is based on the quality of the presentation, on the quality of the discussion questions a participant contributes, and on the classroom activities. There is no exam afterwards.
Minimum effort to qualify for 2nd chance exam:There is no second chance exam.
Description:Papers:
  • Some basics of visual, auditory and tactile perception and the effects of combining them (multimodal presentation and interface design guidelines)
    1. Goldstein E.B. (2007). Perceiving depth and size. In: Sensation and Perception (7th Edition). Pacific Grove, CA: Wadsworth, pages 167-193.
    2. Goldstein E.B. (2007). The cutaneous senses. In: Sensation and Perception (7th Edition). Pacific Grove, CA: Wadsworth, pages 303-325.
    3. Ernst, M.O. & Bulthoff, H.H. (2004). Merging the senses into a robust percept. Trends in Cognitive Sciences 8(4), 162-169.
    4. Sarter, N.B. (2006). Multimodal information presentation: Design guidance and research challenges. International Journal of Industrial Ergonomics. Volume 36, Issue 5 , Pages 439-445.
  • Visual communication interfaces in virtual worlds (effects of non-verbal facial communication)
    1. Werkhoven P.J., Schraagen J.M.C. and Punte P.A.J. (2001). Seeing is believing: communication performance under isotropic video-conferencing conditions. Displays, 22, pp. 137 - 149.
    2. Fabri, M., Moore, D.J., Hobbs, D.J (2004). Mediating the Expression of Emotion in Educational Collaborative Virtual Environments: An Experimental Study. International Journal of Virtual Reality, 7(2), pp 66-81.
  • Navigation interfaces in virtual worlds (head tracked visualization and disorientation issues)
    1. Péruch, P., Vercher, J.-L., & Gauthier, G. M. (1995). Acquisition of spatial knowledge through visual exploration of simulated environments. Ecological Psychology, 7(1), 1-20.
    2. Bakker, N.H., Passenier, P.O, & Werkhoven, P.J. (2003). The Effects of Head-Slaved Navigation and the use of Teleports on Spatial Orientation in Virtual Environments (VE). Human Factors, 45(1), pp. 160-169.
  • Manipulation in virtual worlds (traditional mouse-cursor interfaces versus virtual hand control)
    1. Kok, A. & Liere R. van (2007). A multimodal virtual reality interface for 3D interaction with VTK. Knowledge and Information Systems, vol 11 (3), 2007.
    2. Werkhoven P. & Groen J. (1998). Manipulation performance in interactive virtual environments. Human Factors. Vol. 40 Nr. 3, pp. 432-442.
  • Virtual worlds through mobile displays (scrolling interfaces versus virtual windows)
    1. Yee, K. (2003). Peephole Displays: Pen Interaction on Spatially Aware Hand-held Computers. Proceedings of the ACM Conference on Computer-Human Interaction.
    2. Mehra, S., Werkhoven P. & Worring M. (2006) Navigating on hand held displays: Dynamic versus Static Peephole Navigation. ACM Transactions on Computer Human Interaction.
  • Emerging interface technology (synaesthetic media and brain machine interfaces)
    1. Ramachandran V.S. and Hubbard E.M. (2003). Hearing Colors, Tasting Shapes, Scientific American, April 15.
    2. Waterworth J.A. (1997). Creativity and Sensation: The Case for Synaesthetic Media. Leonardo, Vol. 30, No. 4 (1997), pp. 327-330.
    3. Lebedev MA, Nicolelis MA (2006). Brain-machine interfaces: past, present and future. Trends Neuroscience 29: 536-546.
    4. Friedman D. Leeb R., Guger C., Steed A., Pfurtscheller G., Slater M. (2007). Navigating Virtual Reality by thought: What is it like? Presence Vol. 16, No. 1, pp. 100-110.
wijzigen?