Sterne and Akiyama

The Recording that Never Wanted to be Heard and Other Stories of Sonification

sonificaition - “the transformation of nonsonic data into audible sound”

Main goals:

1. “showing how sonification articulates a range of practices that render data for the ear and exploring ideas and practices articulated to hearing within sonification”
2. larger historical point is that “the articulations between particular senses and kinds of sense data are incredibly weak”

They argue for:

1. attending to the modularity of sensory technologies
2. for the modularity of the relations between the senses, subjects, and technologies
3. for the modularity of the senses themselves ** modularity means “the degree to which densely connected compartments within a system can be decoupled into separate communities or clusters which interact more among themselves rather than other communities.”

Gregory Kramer and Bruce Walker (2004) break sonification into three categories

1. alerts and notifications - communicate a binary possibility
2. auditory icons/’earcons’ - give feedback on a process (ie: a completion noise); vary in pitch and timbre but only one kind of noise
3. audification - renders data meaningful to a listener (ie: listening to data or data from images) via transcoding (shifting data to sound)
   • audification requires an assumption that the sound produced is made directly of the object being sonified - patterns in sound come from patterns in data
     • it is an INDEX not a symbol (re: Charles Sanders Pierece’s 1955 semiotic theory - index has a causal relationship with its signifier; symbol has an arbitrary relationship to its signifier)

Sonification discourse suggests that hearing is more sensitive to affect than the eye and sonification is attached to the idea that the ear and the sonic arts are more inherently closer to emotion and affect - which is rooted in Western traditions that shape the way we think about music, speech, other auditory expression (pg 550); S and A argue that [they] do not need to believe in the truth of these assertions to appreciate their effectiveness in shaping how practictioners relate to, make use of, and construct arguments about sonification.

Currently, there is a split between data-driven empirical sonification and artistic sonification bc data scientists feel the art aspect undervalues the potential for its use in science.

Also discusses the merge with sound art and art-science, sonification for accessibility and emplacement (immediate experience of a space).