Seminar Multimodal perception and interaction

Website:website containing additional information
Course code:INFOMMPI
Credits:7.5 ECTS
Period:periode 4 (week 17 t/m 27, dwz 21-4-2008 t/m 4-7-2008; herkansing week 35)
Timeslot:D2+D3
Participants:up till now 17 subscriptions
Schedule:Dit is een oud rooster!
formgrouptimeweekroomteacher
college   vr 09-1717 BBL-426
19,21,25 BBL-461
practicum   vr 09-1722, 23, 24 BBL-408
seminar          Peter Werkhoven
Joske Houtkamp
    
Contents: The seminar "Multimodal perception & Interaction" is about the fascinating world of human visual, auditory and tactile perception and the use of its potential in designing novel interfaces for interacting with virtual worlds. The seminar consists of five building blocks:
  1. Designing virtual worlds for training purposes: cases, design issues and research challenges (teacher/guest presentations).
  2. Basic knowledge on perception, navigation, manipulation and communication in VR (teacher presentations)
  3. Student presentations and discussions of papers related to the topics under 2. Each participant presents his/her paper and discusses this paper with the group. We will read and discuss papers that address the following topics:
    - some basics of visual, auditory and tactile perception and the effects of combining them (multimodal presentation and interface design guidelines)
    - visual communication interfaces in virtual worlds (effects of non-verbal facial communication)
    - navigation interfaces in virtual worlds (head tracked visualization and disorientation issues)
    - manipulation in virtual worlds (traditional mouse-cursor interfaces versus virtual hand control)
    - virtual worlds through mobile displays (scrolling interfaces versus virtual windows)
    - emerging interface technology (synaesthetic media and brain machine interfaces)
  4. Experiments: evaluation of situation awareness and/or affective appraisal in virtual worlds. Students will design and perform experiments to assess the user's response to a multimodal virtual environment.
  5. Future interfaces: tactile and multimodal interfaces, synaesthetic media, brain-machine interfaces (teacher presentations).
If possible we will organize a visit to the TNO Human Factors laboratory in Soesterberg.
Literature:The literature consists of 16 research papers, all of which are available in electronic form.
Course form:All of the participants will read all 16 papers, write down issues for discussion for each paper and prepare the presentation of two of the 16 papers.
We will get together on four days: April 25, May 9, May 23, June 20.
In weeks 6,7 and 8 you will work together on experiments.
Exam form:The grade is based on the quality of the presentation, on the quality of the discussion questions a participant contributes, and on the classroom activities. There is no exam afterwards.
Minimum effort to qualify for 2nd chance exam:Om aan de aanvullende toets te mogen meedoen moet de oorspronkelijke uitslag minstens 4 zijn.
Description:Papers
  • Some basics of visual, auditory and tactile perception.
    1. Goldstein E.B. (2007). Perceiving depth and size. In: Sensation and Perception (7th Edition). Pacific Grove, CA: Wadsworth, pages 167-193.
    2. Goldstein E.B. (2007). The cutaneous senses. In: Sensation and Perception (7th Edition). Pacific Grove, CA: Wadsworth, pages 303-325.
  • Multimodal perception
    1. Ernst, M.O. & Bulthoff, H.H. (2004). Merging the senses into a robust percept. Trends in Cognitive Sciences 8(4), 162-169.
    2. Sarter, N.B. (2006). Multimodal information presentation: Design guidance and research challenges. International Journal of Industrial Ergonomics. Volume 36, Issue 5 , Pages 439-445.
  • Visual communication interfaces in virtual worlds (effects of non-verbal facial communication)
    1. Werkhoven P.J., Schraagen J.M.C. and Punte P.A.J. (2001). Seeing is believing: communication performance under isotropic video-conferencing conditions. Displays, 22, pp. 137 - 149.
  • Affective qualities and subjective experience in virtual environments
    1. Wilfred, L., Hall, R., Hilgers, M., Leu, M., Hortenstine, J., Walker, C. & Reddy, M. (2004). Training in Affectively Intense Virtual Environments. In G. Richards (Ed.), Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2004 (pp. 2233-2240). Chesapeake, VA: AACE.
    2. Benyon, D., Smyth, M., O'Neill, S., McCall, R., and Carroll, F. 2006. The place probe: exploring a sense of place in real and virtual environments. Presence: Teleoper. Virtual Environ. 15, 6, 668-687.
    3. Freeman, J., Lessiter, E. Keogh, F.W. Bond and K. Chapman. Relaxation Island: virtual, and really relaxing, Presence 2004: 67-72, http://dx.doi.org/10.1162/pres.15.6.668
  • Navigation interfaces in virtual worlds (head tracked visualization and disorientation issues)
    1. Bakker, N.H., Passenier, P.O, & Werkhoven, P.J. (2003). The Effects of Head-Slaved Navigation and the use of Teleports on Spatial Orientation in Virtual Environments (VE). Human Factors, 45(1), pp. 160-169.
  • Manipulation in virtual worlds (traditional mouse-cursor interfaces versus virtual hand control)
    1. Werkhoven P. & Groen J. (1998). Manipulation performance in interactive virtual environments. Human Factors. Vol. 40 Nr. 3, pp. 432-442.
  • Virtual worlds through mobile displays (scrolling interfaces versus virtual windows)
    1. Yee, K. (2003). Peephole Displays: Pen Interaction on Spatially Aware Hand-held Computers. Proceedings of the ACM Conference on Computer-Human Interaction.
    2. Mehra, S., Werkhoven P. & Worring M. (2006) Navigating on hand held displays: Dynamic versus Static Peephole Navigation. ACM Transactions on Computer Human Interaction.
  • Emerging interface technology (synaesthetic media and brain machine interfaces)
    1. Ramachandran V.S. and Hubbard E.M. (2003). Hearing Colors, Tasting Shapes, Scientific American, April 15.
    2. Bach-y-Rita, P. and Kercel, S. W. (2003). Sensory substitution and the human-machine interface. Trends in Cognitive Sciences 7, pp. 541-546.
    3. Lebedev MA, Nicolelis MA (2006). Brain-machine interfaces: past, present and future. Trends Neuroscience 29: 536-546.
    4. Friedman D. Leeb R., Guger C., Steed A., Pfurtscheller G., Slater M. (2007). Navigating Virtual Reality by thought: What is it like? Presence Vol. 16, No. 1, pp. 100-110.
    5. Magerkurth, C., Cheok A. D., Mandryk, R. L. and Nilsen, T. (2005) Pervasive games: bringing computer entertainment back to the real world. ACI Computers in Entertainment 3 (3), pp. 11-29.
wijzigen?