I am Sound

Home

Video/Music
portraits

Cubist
portraits

Concept

Partners

Contact

Concept:

"I am Sound" interweaves contemporary concerns with surveillance, face tracking and use of personal data into a mediated synesthetic experience of transforming image into sound - transforming the participant's face into a musical portrait.

The participant lends their face to the musical system to produce a custom musical composition

“I Am Sound” uses custom software developed over the last several years by Christoph Reiserer, a prize winning composer of both acoustic and electronic music, to generate musical compositions in real time.

First, Tamiko Thiel's image processing software* polls the video camera input to register a person entering the cone of light and dwelling in front of the camera, at which point an image is taken and projected on the metallophone. The metallophone, based on a design by Reiserer, consists of 12 aluminum plates arranged by Thiel into a spatial composition that fractures the projection into a cubist portrait.

Then the software analyses the low resolution image as a whole and on specific plates for four characteristic light/dark chiaroscuro values. These values are sent as initial parameters to Reiserer's music composition software, which he uses to determine the fundamental pitch, the structure of the harmonic spectrum and the melodic order of the tones. Thiel's software further scans the forehead, eyes, nose and mouth regions of the portrait - visible to the participant as a moving green box - for additional chiaroscuro values which Reiserer uses to create variations over time on the initial struture of the music.

Music composition and technology

The output frequencies of this composition are turned into audio signals for the installation with twelve sample players arranged in an equally tempered twelve tone scale. Each sample player uses four different sound samples, produced using the aluminium plates, and their octave frequency transpositions.

Sound production

Twelve thin aluminium plates are tuned to each one of the twelve fundamental frequencies and hung from the ceiling to form the curtain-like metallophone.

The metallophone, with Tamiko Thiel (left) and Christoph Reiserer (right).

Piezoelectric transducers glued onto the back of each plate are used to cause the plate to vibrate as it receives its amplified output signals. In order that the twelve equally tempered aluminium plates can vibrate freely, they are attached with rubber washers to struts on which the plate is hung from the ceiling.

Back side of metallophone, showing holders that contain the transducers

As opposed to usual loudspeakers, the signal will not be amplified via a membrane, but via 12 fixed tones. The fundamental frequency of each carefully tuned plate will be determined and the sound generators calibrated to these pitches, such that the plates will be able to vibrate at the fundamental frequency and its octaves.

The sound produced by the plates is very quiet, however, similar to that of a vibraphone played with the bow of a double bass. If there is substantial ambient noise in the space, the sound may need to be amplied via loudspeakers, but this should only be done if absolutely necessary.

* Face tracking courtesy of Jean-Marc Pelletier's cv.jit externals for Max MSP.