The term “perceptualization“ was coined specifically for this course. It describes the translation of signals and information to modalities that appeal to any of the human senses. As such, it generalizes the terms “visualization“ and “sonification“ to include all other senses. We study such perceptualizations, with particular focus on how properties of a perception systems can be used to optimally convey information. Theory, practice, history and examples of information perceptualization are studied and discussed. Lectures are combined with reading homework, student presentations and a student project. Full attendance is compulsory.

This course is offered as part of the Media Technology MSc program at Leiden University. Students from outside the Media Technology programme who wish to participate should contact the program coordinator.

Lecturers	Maarten Lamers and Edwin van der Heide of LIACS
Level/credits	level 400 (specialized course), 2 EC
Lecture dates	Fall semester, see the Media Technology calendar
Location	room 413, Snellius building
Communication	via the Media Technology forum under "course announcements". Check it regularly!
Course vault	the course vault is only accessible with the username/password combination supplied in class.
Grading	Homework tests (10%), assignments 1 (25%), 2 (25%) and 3 (40%), rounded to the nearest grade accepted by Leiden University's grades administration system.

Lecture	Topic	Homework due (before class)
1	course intro & introducing the senses, part I	-
2	introducing the senses, part II	Read Introduction (pp 1-4) and Chapter 2 (Theory of Sonification) of Hermann 2011, Cain, and Salisbury
3	translation & interpretation, part I	Read Gilbert and Bach-y-Rita
4	translation & interpretation, part II	Assignment 1 (see below)
5	examples of perceptualization systems	Assignment 2 (see below)
6	project presentations	Assignment 3 (see below)

Find a project that exploits an anomaly or "side-effect" of the visual, auditory, olfactory, ..... , perceptual system. For example, it could exploit:

• our visual blind-spot
• saccadic movements of our eyes
• inter-species differences in perception
• hearing through cranial vibrations
• etcetera...

Describe the project and how it exploits the perceptual anomaly/side-effect in 1 page. Include some form of reference to the project. Send us this page via e-mail before lecture 4. Use PDF format if possible. This assignment must be done individually.

Find an existing perceptualization system. It should make a signal or stream of information perceivable and interpretable, in a way other than visualization. It can be a scientific or artistic work.

Write a paper of max 2 pages about your finding, in which you describe the system and create a debate around it. Dedicate one-third of the paper to decribing the system, and two-thirds to the debate and reflection. Be critical, and include at least some critique. Main aspects to consider are the translation and interpretation of information. What do you think of the choices made in the system? What did they neglect? Discuss your findings in a broader context then just the system itself (compare it to alternative systems or current theories).

This assignment must be done individually. Send us your paper via e-mail before lecture 5. Use PDF format if possible. Also, prepare a short presentation (under 5 minutes) about it to present to the class.

Create a perceptualization system. It should make a signal or stream of information perceivable and interpretable, in a way other than projecting it onto a display screen. It should map dimensions (or qualities) of the input signal/data onto dimensions (or qualities) of the human sense(s). How these dimensions are mapped onto each other is most important in the evaluation/grading of the work.

Demonstrate your system to the class in under 5 minutes, in such a way that others can experience it. Provide at least 2 different input streams or data sets for the demonstration. Explain the mapping/translation that you made, and give your system a name.

In 1 page, describe your system, the mapping, and language/interpretation issues such as strengths and limitations. Send us your paper via e-mail before the presentation session. Use PDF format if possible. This assignment must be done in teams of 2 students.

For reasons of copyright protection, some of the articles can be downloaded from within the university's network only.

[Hermann 2011]	T Hermann, A Hunt, and JG Neuhoff (Eds.), The Sonification Handbook, Logos Verlag Berlin GmbH, 2011
[Cain]	WS Cain, R de Wijk, C Lulejian, F Schiet, and L-C See (1998), Odor Identification: Perceptual and Semantic Dimensions, Chemical Senses Vol. 23, pp 309-326
[Salisbury]	K Salisbury, F Conti, and F Barbagli (2004), Haptic Rendering: Introductory Concepts, IEEE Computer Graphics and Applications, March/April 2004, pp 24-32
[xSonify]	Introduction of Sonification, Chapter 1 of documentation for xSonify sonification software, NASA
[Sturm]	Bob L. Sturm (2002), Surf Music: Sonification of Ocean Buoy Spectral Data, Proceedings of the International Conference for Auditory Display, Kyoto (Japan), July 2002
[ICAD]	International Community for Auditory Display, www.icad.org
[Hermann 1999]	T. Hermann and H. Ritter (1999), Listen to your Data: Model-Based Sonification for Data Analysis, In Advances in Intelligent Computing and Multimedia Systems, G.E. Lasker (Editor), International Institute for Advanced Studies in System Research and Cybernetics, pp. 189–194
[Gilbert]	Avery N. Gilbert, Robyn Martin, and Sarah E. Kemp (1996), Cross-modal Correspondence Between Vision and Olfaction: The Color of Smells, The American Journal of Psychology, Vol 109 Num 3, pp. 335-351
[Bach-y-Rita]	Paul Bach-y-Rita and Stephen W. Kercel (2003), Sensory Substitution and the Human-Machine Interface, Trends in Cognitive Sciences, Vol 7 Num 12, December 2003, pp. 541-546
[Robles-De-La-Torre]	Gabriel Robles-De-La-Torre (2006), The Importance of the Sense of Touch in Virtual and Real Environments, IEEE Multimedia, July-September 2006, pp 24-30
[Brewster]	S.A. Brewster and L.M. Brown (2004), Tactons: structured tactile messages for non-visual information display, Australasian User Interface Conference, 18-22 January 2004, ACS Conferences in Research and Practice in Information Technology, Vol 28, pp. 15-23
[Benali-Khoudja]	Mohamed Benali-Khoudja, Moustapha Hafez, Jean-Marc Alexandre, and Abderrahmane Kheddar (2004), Tactile Interfaces: a State-of-the-Art Survey, 35th International Symposium on Robotics, Paris, March 2004
[Elsenaar]	Arthur Elsenaar and Remko Scha (1997), Arthur & The Solenoids, video on youtube.com
[Ando]	Hideyuki Ando, Tomofumi Yoshida, Taro Maeda, and Junji Watanabe (2007), Save YourSelf: Galvanic Vestibular Stimulation Interface, website
[Kaye]	Joseph Kaye (2004), Making Scents: Aromatic Output for HCI, Interactions, Vol 11 Issue 1, Jan/Feb 2004
[Washburn]	Donald A. Washburn and Lauriann M. Jones (2004), Could Olfactory Displays Improve Data Visualization?, Computing in Science and Engineering, Vol 6 Num 6, pp. 80-83, Nov/Dec 2004
[Turin]	Luca Turin (2005), Luca Turin on the Science of Scent, talk on TED.com, February 2005
[Loftin]	R. Bowen Loftin (2003), Multisensory Perception: Beyond the Visual in Visualization, Computing in Science and Engineering, Vol. 5 Num 4, pp. 56-58, Jul/Aug 2003
[Irving]	Lucy Irving, Sensory Substitution project, www.sensorysubstitution.co.uk
[Salisbury]	Madeline Fisher (2007), Balancing Act (about the work of Paul Bach-y-Rita and Mitch Tyler), On Wisconsin magazine, Spring 2007
[Fisher]	Madeline Fisher (2007), Balancing Act (about the work of Paul Bach-y-Rita and Mitch Tyler), On Wisconsin magazine, Spring 2007
[Warwick 2004]	Kevin Warwick and Mark Gasson (2004), Practical Interface Experiments with Implant Technology, in "Computer vision in human-computer interaction: ECCV 2004 Workshop on HCI", Nicu Sebe, Michael Lew, Thomas Huang (eds), LNCS 3058, May 2004, pp. 7-16.
[Warwick 2005]	Kevin Warwick, Mark Gasson, B Hutt, and I Goodhew (2005), An attempt to extend human sensory capabilities by means of implant technology, IEEE Int Conf on Systems, Man and Cybernetics, October 2005, pp. 1663-1668
[Warwick 2008]	Kevin Warwick (2008), Upgrading Humans via Implants - Why Not?, 19: Interdisciplinary Studies in the Long Nineteenth Century, Issue 7 "Mind, Body, Machines", October 2008
[Heimbecker]	Steve Heimbecker (2003), Wind Array Cascade Machine
[Haptic Radar]	Alvaro Cassinelli, Carson Reynolds, and Masatoshi Ishikawa (2006), The Haptic Radar; Extended Skin Project
[Infosthetics]	Information Aesthetics website, infosthetics.com