Sound to video

Fall 2007/Spring 2008
Max/MSP/Jitter

Sound to video is an experiment involving using sound analysis to add live visual effects to input audio. Using Tristan Jehan’s analyzer~ object, I looked at the rhythm, loudness, brightness and noisiness of the incoming audio. Using the data from that, I ran it through many complex algorithms to test for different thresholds. I was looking for prolonged periods of music being quiet vs. loud or chaotic vs. simple. These algorithms would then trigger different video effects or jump cuts in the visuals.

Everything was calculated in real time, so all I needed was a sound file or microphone and a source video and I would get a sort of instant music video. However, I eventually hit the wall in terms of computing power and programming ability, so I have left the program aside for now. I also felt very constricted by mapping certain music events to exact visual changes, so I would want to do a better job at keeping the interaction more organic next time. I would like to revisit parts of the project a bit so that I can incorporate some of the elements into my own performance patch.