The history of cinema and of visual arts is rich with moments in which visionary directors take their work into hybrid directions where physics, science, technology and music intersect. Chromatic futurist music, the abstract animations of people like Ruttman, Eggelin and Fischinger, the many visual music experiments of the beginning of the nineteenth century can be seen as the forerunners of contemporary forms of digital audio-visual interaction.
Looking at the works of these artists it is tempting to imagine what their reaction would be if their could experience the infinite possibilities of the encounter between audio and visual communication forms that the digital meta-language allows. In this perspective one that links the history of the audio-visual to its contemporary development- it is even more interesting to discover the experiences of artists that prefer to recuperate almost forgotten technologies that belong to media-history rather than using digital ones.
One of these technologies is the optical synthesis of sound. It is based on Fourier‘s discovery that sound is a wave, and was developed during the nineteenth century by physicians, animation experts and artists. Examples of contemporary users of optical sound are the live hyper-minimal Fourier-Tanzformation I+II dei Mikomiokona and the performance of Tonewheels. Tonewheels is born out of the collaboration between Derek Holzer and Sara Kostler, and was presented at the Festival Netmage 2008. The Dutch duo, which has been part of the Umatic collective for a number of years, considers the optical synthesis of sound an exciting challenge in the arena of chaos manipulation. This technique allows the artists to control the flow of energy provoked by the encounter between light and electricity with their hands. It allows them to intervene in the unstable balance between sound shape and graphic design, thus creating a tension between the performance act and the audience through the material nature of the audiovisual creation.
Claudia D’Alonzo: Tell me about your respective artistic journeys and about your collaboration
Derek Holzer: My background originally was in radio and media-arts, including college, community, pirate and internet radio. This kind of work led very quickly into field recordings and from there to the other aspects of sound art which I am still busy with to this day. I spent quite a bit of time between making environmental recordings as well as media-arts projects surrounding them (such as Soundtransit.nl) and making and programming digital sound through free and open source applications such as Pure Data.
Sara and I have been working together since 2001, when we went together to the Acoustic.Space.Lab symposium, at the site of a 32m dish antenna used by the KGB as a spy station during the Soviet era in Latvia . Since then, we have collaborated on videos, installations, websites, workshops and live audiovisual performances. Each one of us is totally dedicated, perhaps even immersed, in our own chosen media, which is what makes our collaborations so strong.
Sara Kolster: Although my background is design, I never really felt myself a designer. The focus of my work soon shifted towards still and moving image, resulting in collaborations with media- and sound artists and documentary makers. I have a very research-based approach, ranging from a more formalist approach in the use of image and sound (like the Tonewheels project), to a more content based perspective used in disciplines such as journalism, documentary and archeology (like the projects Ossea and Living Spaces). The last few years this resulted in a wide variety of projects involving installations, webbased projects such as SoundTransit, single channel video work, live performances and animation. Besides my artistic work, I really enjoyed giving lectures and workshops in art centers or Universities in various countries.
As Derek, I don’t really believe in working alone and prefer to collaborate with other artists – each with their own talent – coming from different backgrounds. For me, this process of collaboration is an essential aspect of my work. It pushes me and my work into areas I didn’t think of before.
Claudia D’Alonzo: When and how was your interest in optical sound technology born? When did you decide to begin to recuperate historical instruments for a/v experimentation for the performance Tonewheels?
Sara Kolster: In 2003 we started the project Visible Sound/Audible Image which involved a series of live av-performances, workshops and screenings in the Baltic States with a central focus on the direct interrelation of image and sound. The live-performance resonanCITY was part of this project and showed already our interest in using analogue material, such as found objects, medium format slides and film footage to create sound and images from. You can see it as a preliminary state of tonewheels, especially the visual part of it, since I used a small lightbox and a camera to create a live projection. The overhead projector I use in Tonewheels is nothing more than an enlarged lightbox; with the only difference that the projection directly shows what my hands are doing without any digital interference of a video camera or a beamer. There’s no delay or buffer between the “real” image and the projected image which makes the performance extremely “live”.
Derek Holzer: I’ve always been fascinated by the ANS synthesizer, which was conceived by Russian inventor Evgeny Murzin in 1938 as a way of creating music from a score without an orchestra. It’s a bit like what we expect from music software nowadays, actually! The “score” of the ANS is a glass plate covered in a black wax. You scratch through the wax, and this lets light into the synthesizer. Where the light shines in determines the pitches that the synthesizer plays, and because you roll the glass plate through the synthesizer (a bit like a printing press, actually), the pitches can change depending on how your scratched and how fast you move the plate. The pure analog simplicity of this instrument is quite striking!
So with this idea in mind, I went together with Sara last summer to the “Workshop for Art and Music with the Overhead Projector”, organized by Ralf Schreiber, Christian Faubel and Tina Tongerel at the Moltkerei in Cologne. I originally thought to make an ANS wall-installation, which could be played by scores drawn for the overhead projector. But then I became more interested in actually drawing the waveforms of the sound, which is exactly what the tonewheels we use are doing. I was also impressed by the work of New York ‘s Loud Objects group, who were at the workshop as well. They solder together simple one-bit microcontroller-based synthesizers live on the overhead projector, and this approach of making the technology more transparent in some way is very important in an era of laptops and black boxes, where the audience has no idea how the sounds they hear are being produced.
Because of my interest in the ANS, which is housed at the Moscow State Conservatory, as well as in other pioneering electronic music instruments such as the Theremin, I have been in touch with Andrei Smirnov from the Theremin Center for quite some time. His knowledge and resources in the area of direct optical synthesis are unparalleled, and the historical information he has given me was extremely useful in conceiving what kind of project this could turn out to be. Andrei was more than happy to look over the research I did, and to offer suggestions and (sometimes blunt) criticisms.
Claudia D’Alonzo: Could you describe Tonewheels’ set? How does your live performance take place and how do you generate the sound and the images?
Derek Holzer: Tonewheels is an experiment in converting graphical imagery to sound, inspired by some of the pioneering 20th Century electronic music inventions. Transparent tonewheels with repeating patterns are spun over light-sensitive electronic circuitry to produce sound and light pulsations and textures, while projected graphical loops and textures add richness to the visual environment. This all-analog set is performed entirely live, using only overhead projectors as light source, performance interface and audience display. In this way, Tonewheels aims to open up the “black box” of electronic music and video by exposing the working processes of the performance for the audience to see.
Sara Kolster: The projection I create is not static, but exists of graphical film “loops”. Besides this constant movement, I manually move translucent material in another layer. In the future i would like to work with more than one overhead projector. This would give me more freedom to experiment with more complex patterns and light-layers, switching from one to the other projector. You could see it as an extreme low-tech video mixing system!
Derek Holzer: Currently, the audio comes from my side alone, and we both contribute to the visual elements. The audience can see the spinning tonewheels and circuits of the interface on my overhead projector, and on Sara’s projector there are running patterns and colors which overlap on the screen with the image of my interface. Perhaps later on, we will add some circuitry which allows Sara’s graphical patterns to influence the sound as well. The whole project is quite new, and is very intensive to develop, so additions come one at a time for every performance.
Claudia D’Alonzo: How do you prepare and choose the graphic patterns that you use during the show? For example I know that you use patterns inspired by traditional and folk decorative art. Do you always use the same graphic material or do you modify it for every performance? What have you prepared for Netmage?
Sara Kolster: For our set in Poland , I created graphics based on folk patterns used in traditional Polish clothing. When we visited Warsaw the first time in order to investigate the performance setting and to do research in order to develop our tonewheels-set, we found a book about traditional Polish folk-art and the idea arose to use it as a starting point of our set. From the book I manually reproduced these graphics on the computer, resulting in 15 different tonewheels and film-strips which we used during our set in September of this year. Besides these folk-patterns – which were quite complex – I designed several other black & white graphics – more quiet ones – which would make the projection more interesting. Besides the b&w-graphics, I use translucent material such as color light-filters and masks. For Netmage I will use a combination of the Polish patterns and newly created ones.
Derek Holzer: The Polish folk patterns looked quite beautiful, but acoustically they were usually quite similar…they all had the kind of buzzing sound of a square or sawtooth waveform. I had been looking for a while at Edwin Emil Welte’s Light-Tone organ, which used spinning glass discs with various harmonic waveshapes painted on them to create sound. So I asked Sara to imitate one of these discs from a photograph. While it wasn’t harmonically perfect, it did have a very different sound from the others, and it became one of my favorite tonewheels to play because of its unique timbre.
What’s interesting is that this discussion of designing the tonewheels from the graphical point of view versus designing from the sonic point of view is an exact reproduction of the debates which took place at the dawn of this technology in the 1930’s. Animator Oskar Fischinger was working on painting or photographing graphical shapes and patterns directly into the soundtrack strip of motion picture film, to see how certain shapes sounded. On the other hand, the technician Rudolf Pfenninger was interested in creating a “vocabulary” of waveshapes which corresponded to different instrument or voice sounds, so that he could compose film music graphically without the need of performers (again, the main idea behind most computer audio software).
The Netmage performance was most likely use all the different tonewheels we have developed so far. Each has its own special characteristic and sound.
Claudia D’Alonzo: Talking about the audio, what kind of sound are you searching for and what control do you have on the output?
Derek Holzer: Pre-programmed, scored, scripted or sequenced music doesn’t interest me at all. My performance strategy has always been to create a situation with a number of possibilities (instruments, objects, timbres, etc) and explore those through improvisation. Live performance for me has always been something like a struggle to gain control of what is often a very chaotic system. The sound which happens in the Tonewheels performance is primarily the sound of modulated electricity, sent directly to the mixer. But as it is direct current voltage, I can also send it to the analog modular synthesizer and create new sounds with it there, and this is where it becomes exceptionally chaotic! Beyond that, I like to see what happens in the live scenario, rather than try to predetermine what I or the audience will hear.
Claudia D’Alonzo: How does the possibility of touching, of literally dismantling the audio and visual source, influence the live experience? As performers do you have a different kind of involvement compared to playing with a software and a laptop? I can imagine it being a very involving experience on a sensorial level
Derek Holzer: In the beginning of 2007, I made a small promise to myself to slowly get rid of the laptop from my live performances. Not all at once, but in stages. I started to realize that laptop performance, outside of a few people who really critique it like Mattin, is an absolute dead end. I’m not saying that computer music is a dead end, or that music should not be created or composed on the computer. But I question projects like all the laptop quartets and orchestras which have popped up lately. It’s the visual equivalent of watching the window of an internet cafe! There is simply nothing performative in it, so why expect a paying audience to sit and watch it? Gadgets and Wii-motes and sensors and these kind of things people have been working with the last 10 years or so add some performativity back in, but in the end the big challenge is to involve the audience’s attention in a meaningful way. The computer allows for so much complexity, and for so much pre-planning, that very little is actually spontaneous. And even less of it has a sense of danger to it.So I started acquiring the various parts of an analog modular synthesizer, and I picked up my old hobby of DIY electronics in order to build the parts which I could not afford or find in any other way. The synthesizer works in a very physical and direct way, outside of this imaginary dataspace of the computer. It’s mechanics and electrons moving in absolute real time, and it’s full of risks for the live situation, and that’s what convinced me it was the way forward.
Sara Kolster: In our previous performance resonanCITY I already experimented with the use of analogue material such as medium-format slides and 16mm film. But I never was very satisfied by the quality of the projection; it never was as crisp as it would have been when it would be a direct light-projection, using a slide-projector for instance. For quite some time I was looking for a more direct way of projecting and creating moving image. When this workshop came across last summer, I decided to experiment with the overhead projector, resulting in the tonewheels set.
Claudia D’Alonzo: It seems clear that behind a piece like Tonewheels there is a great amount of research, not only from a technical point of view but also a research on the history of cinema an of Nineteenth century vanguards. And at the same time you both use open source software like Pure Data and you hold workshops to share your knowledge of this program. It seems to me that your way of using technology is very artisanal: what relationship do you have with the instruments with which you work?
Derek Holzer: I’ve always believed that the first step in any kind of technological art is the creation of one’s own tools, rather then buying or downloading some ready-made solution based on someone else’s idea of how art should be made. So PD appealed to this side of me very much, and I created almost every instrument I used with it over the years. Now that I’m working with analog electronics more and more, I find that there is a similar community of people who are constantly inventing new sonic machines, and they have inspired me greatly. From musicians like Jessica Rylan, who built her own performance instruments from the circuit-boards up, and instrument builders like Tom Bugs, who cranks out a new quirky noise-box design every month or so, to true electro-mystics like Martin Howse, who concerns himself in an almost alchemical way with the materials and hidden potentials of electrical circuitry as it interacts with physical matter and bodies–all these people and more have shown me new ideas and directions over the last year.
Claudia D’Alonzo: Do you consider the recuperation of these techniques a sort of audiovisual archeology or do you think that at the moment there is a push in the electronic world to develop trajectories that presuppose a physical contact with the audiovisual material?
Derek Holzer: Walking around on many media-art festivals, I was always surprised by the hype around new gadgets. A lot of times it felt like a sort of fair for tech-fetishists – whether it was GPS or motion controlled surveillance cams – only used to mystify the audience. Most of these – especially digital – techniques create a distance between the audience and the work which is shown, since the way it functions is completely hidden. When you think for example of av-laptop performances, the audience has no clue what is happening, where the sound and images are coming from. Above it all, the static behavior of the performer behind his/her laptop does not reveal any empathy for the audience.
I think a lot of artists come back from the use of digital media, or at least become more critical and aware about the fact why they use a laptop or other new technology. I do believe that the era of a computer-as-end-interface is over. Instead, its role becomes more complex; as a link in a chain of analogue devices (whether mechanical or electronic) it opens many interesting artistic possibilities to discover and explore.
Derek Holzer: It wouldn’t be difficult to characterize much media art of the last ten years or so as having a euphoric–no, actually more than euphoric, even beyond utopian–vision of the possibilities of disembodied data. The media activist can suddenly make a radio show or magazine without the traditional media infrastructure. The digital sculptor can fashion 3D models of impossible objects existing in worlds with invented laws of physics. The laptop composer has an endlessly recursive strange attractor of fantastic instruments and orchestras to do their bidding. Or one can even sample their favorite performers and play along with that.
But all this work exists only in the imaginary dataspace, to be played out in the dark corners of the internet, via solitary explorations with the home computer. When you bring a group of people together (in a real room, often with ugly carpet or perhaps the smell of unwashed feet) around computer art, what you often get is a situation where one person is “inside” the work, either as artist or audience, and the rest are watching someone else be “inside” something without knowing exactly what it is they are experiencing. Even this “locative media” craze, which was supposed to be about giving physical location to this disembodied data, actually required that all of us walk around staring at screens or plugged into headphones the whole time. Alone, waiting for a message, like a participant at the hacker conferences where they still prefer to use IRC even when they are in the same room (purple carpet, stinky feet..). I, for one, decided that I’d had enough of that. And I sincerely hope there are others who are willing to join me.