Seminar Multimodal perception and interaction

Website:website met extra informatie
Onderwijs:Het vak INFOMMPI wordt in 2012/2013 niet aangeboden.
Onderwijs:Het is nog niet zeker of en zo ja in welke periode van 2012/2013 dit vak wordt aangeboden.
Nota bene:Er is geen recente vakbeschrijving beschikbaar.
Onderstaande tekst is een oude vakbeschrijving uit collegejaar 2008/2009
Inhoud: The seminar "Multimodal perception & Interaction" is about the fascinating world of human visual, auditory and tactile perception and the use of its potential in designing novel interfaces for interacting with virtual worlds. The seminar consists of four building blocks:
  1. Designing virtual worlds for training purposes: cases, design issues and research challenges (teacher/guest presentations).
  2. Basic knowledge on perception, navigation, manipulation and communication in VR (teacher presentations)
  3. Student presentations and discussions of papers related to the topics under 2. Each participant presents his/her paper and discusses this paper with the group. We will read and discuss papers that address the following topics:
    - some basics of visual, auditory and tactile perception and the effects of combining them (multimodal presentation and interface design guidelines)
    - visual communication interfaces in virtual worlds (effects of non-verbal facial communication)
    - navigation interfaces in virtual worlds (head tracked visualization and disorientation issues)
    - manipulation in virtual worlds (traditional mouse-cursor interfaces versus virtual hand control)
    - virtual worlds through mobile displays (scrolling interfaces versus virtual windows)
    - emerging interface technology (synaesthetic media and brain machine interfaces)
  4. Students will design and perform experiments on multimodal perception, affective appraisal in virtual worlds, or brain-machine interfaces.
If possible we will organize a visit to the TNO Human Factors laboratory in Soesterberg.
Literatuur:kan veranderen!
The literature consists of 16 research papers, all of which are available in electronic form.
Werkvorm:All of the participants will read all 16 papers, write down issues for discussion for each paper and prepare the presentation of two of the 16 papers.
We will get together on four days (dates to be determined).
In weeks 6,7 and 8 you will work together on experiments.
Toetsvorm:The final grade is based on the quality of the presentation, on the quality of the discussion questions a participant contributes, and on the classroom activities (together 50 %) and the report on the experiment (50 %). Each of the grades should be 5.6 or higher. There is no exam afterwards.
Inspanningsverplichting voor aanvullende toets:Om aan de aanvullende toets te mogen meedoen moet de oorspronkelijke uitslag minstens 4 zijn.
Beschrijving:Papers (NB some of these will be updated before the start of the course)
  • Some basics of visual, auditory and tactile perception.
    1. Goldstein E.B. (2007). Perceiving depth and size. In: Sensation and Perception (7th Edition). Pacific Grove, CA: Wadsworth, pages 167-193.
    2. Goldstein E.B. (2007). The cutaneous senses. In: Sensation and Perception (7th Edition). Pacific Grove, CA: Wadsworth, pages 303-325.
  • Multimodal perception
    1. Ernst, M.O. & Bulthoff, H.H. (2004). Merging the senses into a robust percept. Trends in Cognitive Sciences 8(4), 162-169.
    2. Sarter, N.B. (2006). Multimodal information presentation: Design guidance and research challenges. International Journal of Industrial Ergonomics. Volume 36, Issue 5 , Pages 439-445.
  • Visual communication interfaces in virtual worlds (effects of non-verbal facial communication)
    1. Werkhoven P.J., Schraagen J.M.C. and Punte P.A.J. (2001). Seeing is believing: communication performance under isotropic video-conferencing conditions. Displays, 22, pp. 137 - 149.
  • Affective qualities and subjective experience in virtual environments
    1. Wilfred, L., Hall, R., Hilgers, M., Leu, M., Hortenstine, J., Walker, C. & Reddy, M. (2004). Training in Affectively Intense Virtual Environments. In G. Richards (Ed.), Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2004 (pp. 2233-2240). Chesapeake, VA: AACE.
    2. Benyon, D., Smyth, M., O'Neill, S., McCall, R., and Carroll, F. 2006. The place probe: exploring a sense of place in real and virtual environments. Presence: Teleoper. Virtual Environ. 15, 6, 668-687.
    3. Freeman, J., Lessiter, E. Keogh, F.W. Bond and K. Chapman. Relaxation Island: virtual, and really relaxing, Presence 2004: 67-72, http://dx.doi.org/10.1162/pres.15.6.668
  • Navigation interfaces in virtual worlds (head tracked visualization and disorientation issues)
    1. Bakker, N.H., Passenier, P.O, & Werkhoven, P.J. (2003). The Effects of Head-Slaved Navigation and the use of Teleports on Spatial Orientation in Virtual Environments (VE). Human Factors, 45(1), pp. 160-169.
  • Manipulation in virtual worlds (traditional mouse-cursor interfaces versus virtual hand control)
    1. Werkhoven P. & Groen J. (1998). Manipulation performance in interactive virtual environments. Human Factors. Vol. 40 Nr. 3, pp. 432-442.
  • Virtual worlds through mobile displays (scrolling interfaces versus virtual windows)
    1. Yee, K. (2003). Peephole Displays: Pen Interaction on Spatially Aware Hand-held Computers. Proceedings of the ACM Conference on Computer-Human Interaction.
    2. Mehra, S., Werkhoven P. & Worring M. (2006) Navigating on hand held displays: Dynamic versus Static Peephole Navigation. ACM Transactions on Computer Human Interaction.
  • Emerging interface technology (synaesthetic media and brain machine interfaces)
    1. Ramachandran V.S. and Hubbard E.M. (2003). Hearing Colors, Tasting Shapes, Scientific American, April 15.
    2. Bach-y-Rita, P. and Kercel, S. W. (2003). Sensory substitution and the human-machine interface. Trends in Cognitive Sciences 7, pp. 541-546.
    3. Lebedev MA, Nicolelis MA (2006). Brain-machine interfaces: past, present and future. Trends Neuroscience 29: 536-546.
    4. Friedman D. Leeb R., Guger C., Steed A., Pfurtscheller G., Slater M. (2007). Navigating Virtual Reality by thought: What is it like? Presence Vol. 16, No. 1, pp. 100-110.
    5. Magerkurth, C., Cheok A. D., Mandryk, R. L. and Nilsen, T. (2005) Pervasive games: bringing computer entertainment back to the real world. ACI Computers in Entertainment 3 (3), pp. 11-29.
wijzigen?