PLAY MUSIC, videogames and music.

In 1995 the now disappeared but profoundly influential Tokyo based publisher Digitalogue published "The Reactive Square" the first of John Maeda's "Reactive Books", careful explorations of paper and computer based art. The diskette contained a collection of 10 stylish games that used sound input to modify the image of a square. A very simple appearance belies the complexity of the program. It is a minimalist masterpiece. (1) 

3 years later, in 1998, Tota Hasegawa published Microphone Fiend - Interactive Sound Plaything - also on Digitalogue. 9 small windows around a gangster's face react to microphone input to explore different aspects of the yakuza life - « driving fast », « I don’t need a woman », « I'm not crying » (as a tear rolls down his face), « feeling hungry ? », « cant stand him », « find his weak point » …. 

These 2 pieces explored the effect of sound on an image and as such are not games as musical instruments but rather games that use sound as the manner of playing. i mention them here for their conceputal importance and to constitute the trilogy of play that we will be exploring - games as instruments, musical games and sound inputs as interface. They are playful experiments that establish the relationship of sound and image in an indissociable loop, that would be explored profoundly from the end of the  90's with newly available audiovisual tools like the nato video objects for Max/MSP and the work on games as audiovisual instruments.

The classic "Shift+Control" by the british group Audiorom was released in the same year as Microphone Fiend. The CD-ROM contained a collection of playful musical instruments, sequencers and rythm machines coming out of the design/music/computer crossover. In 99 it was re-released & re-designed (the original design was by Neville Brody's studio) -the new version was recognised by a prize in Cannes at the Milia. It was the first real collection of "soundtoys", a genre that would become important later on.(2)

This use of playing with the image to make music soon became more sophisticated and another a classic of the genre is "Small Fish" by Kiyoshi Furukawa, Masaki Fujihata & Wolfgang Munch published in 1999 by the ZKM institute in Karlsruhe, Germany. Simple almost naïve colored forms and gentle melodies express the utter well-being that one feels playing with this piece. Small Fish is a very fine piece of object oriented programming - each visual or audio element having its own behavior which is then modified on contact with another behavior. The result is an almost infinite combination of visual and audio notes. (3)

At the same time, in 1998, a parallel event of great importance happened - the release of the first arcade version of Dance Dance Revolution and it was indeed a revolution. I remembering seeing long cues in Tokyo arcades to use it. It was the first widespread use of a physical interface in a game. A game that was about the music, not the music that accompanies the game. This game has been phenomenally successful - it has been out on playstation for several years and is accompanied with a "dance mat" - a hardware that permits to connect your dance moves to the computer. 

In 2005 the DDR Mario Mix was released (4), spectacularaly splicing genres and this year, 2006, is revival ; Konami have rereleased the original arcade version as My First Dance Dance Revolution ! "My First Dance Dance Revolution you can groove and move to the tunes just like the original arcade game!" Its already nostalgic - as Feel Action sung "pasan las horas i no te das cuenta, es ke vuelven las noventa".

I think this game was important because it completely generated a new understanding of computer games and their market. That from now on games and music would be indissociable. It paved the way for large companies to start taking some risks on musical games.

The first in this sense was REZ in 2001 and we hailed it at the time as a breakthrough. Here was a musical instrument disguised as a first person spaceship shooter and released by a major game company. (5)

The same year things were shaking up all over game land - the figure of the independent game creator was emerging and in the computer art underground people like Jodi, Retroyou or Glaznost were hacking existing game engines to make new pieces. JODI's rework of QUAKE, Retroyou's fuck the gravity code. Both these artists made sound pieces recycling game engines. The game as audiovisual instrument or the vice versa.

Su-studio in Barcelona or lab-au in Brussels (6) also investigated this line, but building from the bottom up. In the case of Su-Studio using VirTools they created a complex game called « Commandos » (2001) where an impressive 3D environment served as a musical space. (7)

By the end of 2002 musical gaming - either as dancing to music or creating it - was here to stay in commercial games - the following musical games for Playstation 2 were shown in an event linnking games and music in Art Futura  :: Rez, Space Channel Five w. dance pad, Frequency, Britney Spears Dance w. dance pad, Guitaroo Man.  (8)

In 2004 Julian Oliver and Pix premiered in Sonar an audiovisual performance engine called FIJUU, they call it "game-based audiovisual performance". (9) In many ways it was the culmination of the direction that Commandos had begun to explore 3 years earlier. Oliver had already explored the use of the Quake Engine as a musical tool - connecting the multiple events of the game via OSC to Pure Data patches which assured a high quality intense sound. Oliver and his selectparks.net site have been fundamental in the the appraisal and development of independent gaming. 

This development of games as musical instruments has culminated in 2006's most important musical gaming release by the artist Toshio Iwai. Iwai dreamed of a game that would combine "microscope, cassette recorder, synthsiser and game console" and he persuaded Nintendo to let him make it. (10)
In Sonar festival 2006 he performed a memorable concert using this game. A game that is an instrument too. In the same festival a concert was played by Günter Geiger, Martin Kaltenbrunner, Sergi Jordà & Marcos Alonso using the ReacTable.(11)

Sergi Jordà and Guenter Geiger are pioneers each in their specific fields. From 1997 Jordà and his team developed the FMOL instrument (12) - a highly advanced "soundtoy" that was the main instrument for the group FMOL trio, while Geiger is one of the developers of Pure Data, a real time audio and video environment. The combined skills of these men with long experience in real time resulted in the ReacTable - an improvisational instrument that plays like a collective game. 

Topology of playing and sound. Game as improvisational environment, game as music, music as game, sound as a way of playing with image ... And we loop back to where we started this text, sound as input, with this game hack from 2005 - "a live instrument battle channeled through the video game Mortal Kombat. Audio from two classical guitars controls movements of characters in Mortal Kombat Trilogy- acoustic-electric instruments have replaced the typical video game controller .... Modal Kombat delivers an engaging combination of music, choreography and good old-fashioned fantasy street fighting." (13)

(1)Today u can download this from maeda's web - it weighs only 60 kb, but you need Mac OS9 ! http://www.maedastudio.com/2004/rbooks2k/rsquare.html
(2)http://soundtoys.net
(3)Small Fish, Microphone Fiend and other classic CD-ROMs can be consulted in the fiftyfifty collection housed in the Sonoscope consultation space in CCCB, Barcelona.
(4)http://www.ddrmariomix.com/
(5)http://www.sonicteam.com/rez/
(6)http://www.lab-au.com/ 
(7)http://www.soniacillari.net/komandos.htm
(8)http://www.fiftyfifty.org/playtime/
(9)http://fijuu.com/
(10)http://electroplankton.nintendods.com/
(11)http://www.iua.upf.es/mtg/reacTable/
(12)http://www.iua.upf.es/~sergi/FMOL/
(13)http://www.modalkombat.com/