Design And Performance Lab
|
Performance Systems
Johannes Birringer (c) 2003
It is the particular relationship of dance to technology which motivates me to look more broadly at the changing production environments in which we develop new performance concepts, while also keeping historical precedents in perspective. For example, today's motion capture-based animations, created in the commercial film industry but also in computer science, biomechanics, and art/graphic design departments who always look for performers to be the "subjects" for capture, find their historical roots in late 19th century motion studies in chronophotography and early cinema (Muybridge, Marey, Méliès).(2) Furthermore, dance makers, researchers and teachers have used film or video as a vital means of documenting or analyzing existing choreographies that have been passed on, in practice, from performing bodies to other performing bodies. Some scholars and software programmers published tools (LabanWriter, LifeForms) that attracted attention in the field of dance notation and preservation as well as among choreographers (e.g. Merce Cunningham, Pablo Ventura) who wanted to utilize the computer for the invention and visualization of new movement possibilities.(3) At the turn of the new century, overlapping interests in related fields -- film, electronic music, digital art, science and technology, design, engineering, robotics, telecommunications -- advance our understanding of the complementary thinking processes that drive new interdisciplinary research and conceptual models influenced by the computer's information processing capabilities and the internet's global reach. Like music before it, dance has begun to incorporate instruments (cameras, video-projectors, microphones, sensors, computers) and software tools which allow it to structure and control the various components of any performance event, i.e. sound, video, movement. It is the convergence of choreography and movement design with the languages of programming, electronic music and film editing in interactive, real-time performance processing that I want to define as a "performance system." I do not look at human performers in the interface as separate from an interactive software system or programming environment (such as MAX/MSP, VNS, BigEye, EyeCon, Isadora, Keystroke, Choreograph, etc).(4) Rather, I want to treat the entire interface environment as performance process, as an emergent system. In the following, I propose to examine a new aesthetics of interactivity which recognizes the technical context of programming languages and the artistic challenges we face when physically working with real-time processing. In this context, the proscenium and conventional production processes have become inadequate. New dance, involving technologies and interactive designs from the conceptual starting point, requires a different environment for its evolution. I offer an overview of the terms crucial for an understanding of interactive dance, especially as they relate to new training environments and performance systems. These terms have gained currency in the emerging field of dance and technology, while being intrinsically connected to music, electronic instrument design, architecture, digital art, and the popular culture of video and computer games. It may be premature to speak of a digital aesthetic, but I claim that new computer tools change the nature of the creative process. My observations are based primarily on my own practical experience and that of my collaborators in the field.
I use the term "interactivity" with regard to two phenomena. First, I address "interaction" as a spatial and architectural concept for performance, and secondly I look at "interactivity" in the more narrow sense of collaborative performance with a control system in which the performer-movement, gesture and action are tracked by cameras/sensors and thus used as input to activate or control other component properties from media such a video, audio, MIDI, text, graphics, QuickTime movies, scanned images, etc. In the latter case we speak of an interactive system that allows performers to generate, synthesize and process images, sounds, voice and text within a shared real-time environment. Before examining the behavior of such systems, I want to apply the notion of interaction to the historically evolved understanding of multimedia performance as a multi-faceted and multi-dimensional dynamic process, either based on choreography or improvisation and more open-ended, fluxus-like constellations. Historically, interactivity as an aesthetic category would not derive from classical concepts of composition or choreography; rather, it is indebted to the early 20th century avant-gardes and their experiments with performance as live concatenation of different, sometimes conflicting media (dadaist, futurist, surrealist performances) as well as with performance as a conceptual instrument for the activation and provocation of the audience. Western art history derives its understanding of interactive media arts predominantly from the participatory events of the 1960s (happenings, Fluxus, process art, Situationism, kinetic art, concept art, "art and technology," the John Cage/Robert Rauschenberg collaborations, cybernetic art, closed-circuit video installations, etc) and the progressive "dematerialization of the art object" which implied the active, physical participation of the audience in the event. Since the 1970s, interactivity in art generally refers to multimedia installations and environments that involve electronic or computer-assisted interfaces. Nicholas Negroponte already suggested in 1970 (The Architecture Machine: Toward a More Human Environment) that such interfaces are characterized not only by the points of contact and interaction between a machine and the physical or information environment, but by the artistic strategies used to engage audiences in a dialogue.(5) Compared to interactive installations and digital artworks, sound sculptures, immersive environments, computer games, and more recent internet-based forms of telepresence, interactive dance in the strict sense of computer-assisted design cannot claim such a long and heterogeneous history. Dance makers have largely remained committed to presentational stagings of multimedia works - complete and highly structured works for the consumption and aesthetic contemplation of the audience. Dance installations and interactive online dance which engage active viewer-participants are rare events that require careful attention and analysis, especially since we don't have any established aesthetic or social criteria for the evaluation of a successful interface. Regrettably, the professional and academic dance community has not found many commonalities with the vibrant club and rave cultures either, which to some extent has contributed to a sense of isolation among younger dance artists growing up with computers, music television, techno, hip hop, and the transnational exchanges and cross-overs in music. The problem that I see is the overwhelming emphasis, in academic and concert dance training and its specialized professional practice in the West, on specific techniques (technical training in ballet and modern dance), vocabularies and compositional structures which have limited usefulness for an exploration of participatory processes and the integration of recipient behaviors and feedbacks. Moreover, dance practice as it is commonly understood in Western training has been largely focused on the performer's physical virtuosity and bodily intelligence, shaping and disciplining the body for the execution of choreography, and not for interaction with mediated and unstable environments. Addressing "interaction" as a spatial and architectural concept for performance, therefore, means shifting the emphasis away from the creation of steps, phrases, "combinations" or points on the body that initiate movement, away from the dancer's internal bodily awareness (widely encouraged in today's practices of yoga, somatics, experiential anatomy, body-mind centering and release techniques) unto her environment, to a not-given space but a shifting relational architecture that influences her and that she shapes or that in turn shapes her. Shifting attention to touch, weight and energy transfers in partnering, as it is practiced in contact improvisation, is a good preparation for working with physically motivated sensor interfaces, especially wearable sensors. This re-orientation also implies an initial awareness of how lighting sculpts space, and how lighting color, angle, temperature and intensity are constituents of the dynamic and intermediating plasticity of space that creates opportunities for movement. Moving bodies and changing light, along with the crucial experience of the resonating body within a reverberating sonic environment, are part of the collective consciousness in which we are enveloped and in which we are co-creative participants. This notion of a resonating environment, in my own practice, is indebted to the plastic sculptural process that dancers, visual artists, media artists, programmers and architects have recently explored -- a plastic process of "designing" fluid space that allows for integration of "nervous" or sensitive media presences.(6) In a sense, I see the sculptural process as a contemporary modification of Laban's Space Harmony, of the Bauhaus principles of synaesthetic abstract constructivism, and of Joseph Beuys's and Hélio Oiticica's enactments of "social sculptures." In philosophical terms, I am also suggesting a non-Western and non-Euclidian approach to spatial "science" and geometry: There is a need for a philosophical framework
that enables us to engage harmoniously with the contextual living space
in which we are immersed and from which we are as inseparable as a whirlpool
is from a water flow. Reversing the man-induced ebb of essential harmonies
may come through a philosophy of In other words, a relational performance architecture is participatory, and it does not exclude virtual architectures, as Rafael Lozano-Hemmer has suggested in his writings and artistic projects, for example his highly charged public interface event "Vectorial Elevation" (1999-2000), a transformation of Mexico City's Zócalo Square with enormous light sculptures created by participants on the Internet using a virtual reality program.(8) On the contrary, dance and the changing notions of "site-specificity" in interactive installations need to be discussed with regard to virtual reality environments and such models of immersion that integrate physical and synthetic, 3-D simulated environments, in order to perceive the connections between designs based on representational space and designs generated from algorithms. Current developments in computer science, artificial life research and 3-D design programming (VRML) point to hitherto unimagined combinations and hybrid environments for performance and play which could have a considerable impact on collaborations between choreographers, composers and designers interested in complex, imaginative and dynamic "improvisation technologies," to use the term that William Forsythe applied to his rehearsal operations. I want to give an example of such research to clarify my point. At the 2001 "Subtle Technologies" Conference in Toronto, Maja Kuzmanovic and David Tonnesen showed a computer simulation of the "T-Garden" project they were developing with their FOAM initiative at Starlab (Brussels). Tonnesen emphasized the interdisciplinary nature of the "T-Garden" collaboration and explained its conception: It is a responsive/hybrid play-space where visitors
can 'converse' with sound, 1. How do we develop sustainable, international
collaboration networks 2. How do we allow the project to evolve in
the most open and interactive 3. How do people individually and collectively
make sense of responsive, 4. Can play (in the broadest sense of the word)
become an essential model 5. How can new forms of expression be sustained
by a fusion of media, These questions point to the heart of current experimentations with interactivity, which for dance makers until very recently was largely a dialogue with composers and programmers who designed MIDI-activated sonic environments for non-linear choreography. "T-Garden" suggests an expanded architecture allowing the performers or "gardeners" to experience physical and tactile relationships to a virtual reality that they can actually modify and shape, moving through the projective, computer-generated world. Since this world needs to be projected via surround-sound speakers and LCD projectors, it means that the performer moves through light waves, fields of color and pulsations, floating virtual objects, etc., and her body potentially experiences ruptures of the kinesthetic from the visual senses as all physical body-surfaces gain a multidimensional tactile extensionality. We also note that it is the audience who become the performers and "gardeners" in this interactive installation. Its logic of "interactivity" in play aims at dissolving spectacle and the featured stage performer altogether.(10) Such "movement-through" interactive and generative environments posits a shift in perception that many dance practitioners, used to working in delineated real space - with a forward orientation on the proscenium and toward a passively spectating audience - have been reluctant to engage. The engagement requires new vocabularies informed by interactive design and VRML (Virtual Reality Modelling Language) and involves such notions as parameters, mapping, navigation, tracking systems, Musical Instrument Digital Interface (MIDI), genetic algorithms, modules and patches based on specific programming languages such a MAX/MSP. It also requires at least a basic understanding of the underlying computational processes which generally remain invisible. And it prompts us to reflect on contemporary science-derived concepts of "emergent" or autogenerative systems as they are now being used by diverse artists working with interactive videoinstallations, artificial life architectures, 3D shared spaces, and telepresence. Navigations and Interfaces Technology has decisively challenged bodily boundaries and spatial realities, profoundly affecting the relations between humans and machines. The convergences between dance and technology reflect back on the question of dance and its physical-sensory relationship to space and the world, its immediate, phenomenological embodiedness to lived experience in one place. We are still in one place among other moving bodies when we dance, whether we are in a rehearsal studio or in the street or a discotheque. But if dance indeed takes the lead, among the theatrical arts, in absorbing technology as a creative tool, it needs to revise its rehearsal methods and training facilities. First, interactivity has implied the relocation of the compositional process into a laboratory-like environment. The directors of the ISA at Arizona State University call it the "intelligent stage," where dancing takes place with computer-assisted design and MIDI interfaces in an interactive ambience which allows a different "programming" of physical motion and motion sensing. At ISA the stage is wired for internet access and telematic transmission of streaming video and MIDI signals, while also featuring the "Very Nervous System" design developed by Rokeby. As with other tracking systems such as BigEye, EyeCon, or Isadora, the sensing in the VNS interface is done by cameras and motion detection devices. But the dancers also become "sensors," adopting to a new spatial awareness of a digitally enhanced space or "operating system" which triggers responses and feedback. Dancers appear to be touching invisible partners; they become ghostcatchers.(11) Secondly, the engineering of interface designs moves to the foreground in such labs, and the term "sensing" gains a dimension reaching beyond the physical and organic understanding of bodily anatomy, musculature, and proprioceptive spatial awareness of moving-within-the-kinesphere that dancers trained in modern traditions (after Laban) bring to the studio. The convergence of interface design and movement analysis extends earlier Laban-derived structural explorations of the body's repertoire for movement. In more than one sense, it involves the entire sphere of movement as interaction, encompassing perceptive and receptive processes. If movement is a "continuous current" (Laban), a new understanding of "interspaces" in networked performance is now evolving. The interactivity of sensitive environments is one crucial aspect of it. The notion of real-time flow changes, as the environment also functions as a video studio or soundstage, and cameras, sensors and appropriate lighting need to be continuously calibrated. There will be constant interruptions. If the environment is networked, there will be delays in the uplink/downlink teleoperation; such delays might affect kinesthetic perception. The most significant intervention into movement today is the dis-location, and subsequent re-distribution, of movement as captured and processed image, micro-movement, sampled ghost. Movement, as it is used in interactive and networked performance-installations, is not a continuous current with space itself but continuously crosses between real space , projective space (video/animation) or other virtual contexts (VR, remote sites). ADaPT, multi-site telepresence performance, Space is dematerialized, movement is captured, commuted, transferred and rematerialized elsewhere; we interact with sensory information such as video which projects different three-dimensional kinesthetic perceptions of movement energy, position, and velocity (cf. slow motion, close-ups, different scale, distorted color/pixilation, dis-focus, etc).(12) The programming of interfaces between dancers and the computer implies the creation of an unstable system. "Choreography" more closely resembles the "live mix" we experience in techno culture when DJs create a situation, a sound continuum, and use filter devices to modify the parameters in response to energy that is transferred between dancers and musical stream. The intensities of the event develop a kind of autopoiesis; in current dance experiments with interface designs based on feedback/triggers in real time, the composition process is like an "emergent system": symbiotic improvisation with invisible sensor lines or dynamic fields in space.
Dance, closely associated with visual forms and
rhythms, is fundamentally a multimedia system. We know from photography
and motion studies that performances were staged exclusively for the
camera. Choreographers discovered that videodance is a composite medium
in its own right: choreography is editing of frames. Making dances for
the camera has become not only a cinematographic alternative to live
dance, but motivated choreographers to re-conceive the aesthetic framing
of live dance. The impact is evident in the cinematic quality of many
contemporary works. The Japanese companies Dumb Type (Memorandum) and
OM2 (The Convulsions of Mr K.) literally used no less than six simultaneous
screen projections in recent performances. Such projective video topologies
need to be examined as moving architectures in the environment. Video
projection opens up a screen space for movement Dancers become conscious of the deep structure of computer interfaces, learning to navigate expanded spheres of movement that require a radical reordering of the senses due to an increase in telematic or virtual interaction. We are engaged in a new form of motion studies, and in the analysis of its remote effects. To my knowledge, there are four types of environments currently evolving in dance: (1) interactive environments (based on sensors and motion tracking); (2) derived environments (motion-capture based re-animations of bodily movement or liquid architecture, which can also be networked and reintroduced into live telepresence or telerobotic communications between remote sites); (3) immersive environments (Virtual Reality based, such as the "Cave" or panoramic installations - similar to "T-Garden" - that integrate the body, with stereoscopic devices in front of the eyes, into the polysensual illusion of moving through space); and (4) networked environments (telepresence, videoconferencing and telerobotics, allowing users to experience a dispersed body and to interact with traces of other remote bodies, avatars and prostheses).. The parameters of all of these environment types can be mixed; we can then speak of mixed reality environments. In the following sections I describe these environments and how they affect performers and observers. Interactive Systems The first type of environment translates physical gestures of dance via sensors, motion-tracking cameras and analog/digital converters into a signal representation inside of a computer program. Looking at the MAX/MSP software as an example of interactive systems, we encounter specific design features that organize the relational architecture. MAX is a graphical programming environment that allows the building of controllers for real-time synthesis and signal processing. The gestural data, received via a sensor system, is mapped to the control of a given sound synthesis parameter. The continuous MIDI data - generated by physical gestures of the dancer or musician (playing instruments with sensor devices) - affect the performance parameters of the MAX/MSP patch which runs on the computer and effects the sonic and graphic output. As an instrument the MAX software primarily controls the source materials (the sound and video files stored in the computer or synthesizer), sound parameters, and the dynamics of real-time synthesis. It can harbor considerable complexity since patches might be constructed in the manner of a "nested" design -- enfolded entities that are in a continuously fluctuating state of unfolding to activate the modular parts. What is crucial about the interaction is the range of responsiveness of the controller with regard to the dynamic range of expression of the mover. Beyond simple cause and effect, I should think that the system only becomes challenging and valuable to the dancer once she can play with it and modulate the sonic space and musical parameters around her to allow herself and the audience to perceive the real-time conversions of movement into sound. Given such complexity in the programmed environment,
we must ask how performers and musicians regard the physical relations
between performance and "controlled" parameters, and
From a choreographic point of view, the dancer
within an interactive environment resembles the "player" in
"T-Garden" -- she will need to familiarize herself with the
response behavior of the sound and video parameters, and both player
and composer will strive to create an exponentially more sensitive,
articulate and intuitive system. In a shared environment this could
mean refinements in sensors, filters, and output processors, but also
an attenuation of the performer's spatial-temporal consciousness. How
is the performer-musician-system relationship evolving, emergent? What
can we learn from jazz-improvisational structures, from video game structures,
from different cultural contextualizations of virtual environments? Corpo Aberto (2000) As these examples illustrate, dancers, composers, and designers can interpret the relational architecture of interactive systems in many different ways, depending on a work's emphasis on dance gesture-to-music synthesis, or dance gesture-to-video synthesis. Robert Wechsler (Palindrome Intermedia Performance Group) recently suggested in an internet posting that mapping strategies should address the basic problem common to most intermedia pieces which place dancers in the role of musical performers: namely, how to create an interdisciplinary work that succeeds from all the choreographic, music-compositional, and filmic perspectives. What he implies, of course, is that the dancer is or becomes the "musical performer," or, in Santana's case, plays with being the eye of the camera, which is not the same as inter-acting with a dynamic multisensory sound/video environment which may respond in unexpected and uncontrollable ways. Yet the question remains whether choreographers and composers have different or conflicting goals, or whether there is an aesthetically stringent co-resonance between movement, sound, and video that can transform the entire environment kinesthetically. Let us look at two other interactive works, presented at CROSS FAIR in Germany, which connect the first and second types of environment. Scanned, conceived and directed by Christian Ziegler, is a performance-installation that consists of video projections of a dancer's scanned movements. To create the live work, Ziegler first asks dancer Monica Gomis to perform movement phrases lasting from one to fifteen seconds, which are taped by a video camera. A program wrote for the computer allows him to let a digital video scanner unfold the movement-images over time, controlling direction and speed of the scan as well as resolution and tempo of the scanned material. In performance, the scan projections slowly emerge over a period of time, as if we were watching a painting come into life. The "choreography," according to Ziegler, can be seen by the imagination of the viewer. One could also argue that there is no choreography, but that the interface with the computer creates temporally dilated paintings of human gestures and movements, completely reorganizing the time and space of the dance frames. Scanned (2000) Yours (2000) Yours, a collaboration between the Polish composer Jaroslav Kapuscinski and Frankfurt Ballet dancers Nik Haffner and Antony Rizzi, is performed as a dialogue between a pianist and a video projection of a dancer, accompanied by percussion sounds and a female voice (reciting from Beckett's "Texts for Nothing"). Kapuscinski enters the dark center of the room, with audiences seated on both sides of the Disklavier placed there, with a film screen suspended above the instrument. As he begins to play his composition, a dialogue evolves in real time: every strike of the piano keys manipulates the digital video image on the screen by intervening in the order and speed of the dancer's movements. Rizzi was filmed in the nude, his movements based on Haffner's choreography. The interface here is the piano: Kapuscinski steers the video samples of the dance, as well as additional audio samples, via a computer that "reads" the key strokes and even senses the particular articulations in the playing. The composition is newly interpreted in each live performance, and the piano interface is also open to audience exploration, as Kapuscinski suggests after his 45-minute performance. He invites the audience to "play the dancer." Those of us who tried the piano realized that the interactivity was based on relatively simple MIDI trigger (on/off) signals that allow the pianist to play the video image track backward and forward, freeze-frame the motion or advance it, literally, frame by frame, thus controlling the image of the dancer down to the finest atom.(17) Conceptually, the aesthetic of interactive digital art is necessarily indebted to such "MIDI performances," exploring the potential "pastforward" connections that can be made between instruments and media, as well as directing critical attention to our unstable relationships to sound and image environments that seem to have a life of their own. As in the case of Dumb Type's image machine in Memorandum, the past-forwarding and re-winding of movement images in Yours plays tricks with our memory and optics, and digital artist/programmer Michael Saup went so far as to argue at CROSS FAIR that technologies are not our tools or extensions but autonomous intelligent systems: we ought to be interested in what they do to our psyche. Collaborative work such as Paul Kaiser/Shelley Eshkar/Bill T. Jones's Ghostcatching, which was exhibited as a stand-alone virtual dance, along with Yours, and Scanned already points in this direction of a certain autonomy of the system as installation running processed and derived data. Ghostcatching is based on optical motion capture, a system of multiple infrared cameras, computer hardware and software that enable digital 3-D representation of moving bodies. Recording involves the placement of reflectors in strategic positions on the performer's body; cameras surrounding the performer track these sensors in time and space, feeding the information to the computer for consolidation into a single data file. Mocap data subsequently drives the movement of simulated figures on the computer, where they can be mapped onto other anatomies in an animation program. With the animation tool one can draw out and reconfigure the abstracted motions and trajectories of the dance. What we see is the ghost of the dance or, rather, animated motion pictures and drawings.(18) Captured movement phrases thus become the digital building blocks for virtual dance or interactive performances that explore possible, emerging, and always newly manipulable relationships between live and synthetic presences, forms, images, micro-frames, sounds, and their resonances in our imagination. The promise of motion tracking technology and real-time digital signal processing (also now available in motion capturing) is the simultaneous exploration of a fluid environment in which dance can generate sound and animation, sound can affect video images, and captured images inform new movement and a new form of action painting. Immersed in Invisible Writing / Mapping I will discuss "immersive environments" by mainly addressing them as a new knowledge and training space. Creating interactive virtual reality is a heavily research-oriented endeavor and requires appropriate laboratory conditions. Apart from the CAVE environments that were created at ars electronica and various computer science labs, I am thinking of the groundbreaking Art and Virtual Environment Project conducted at the Banff Center in Canada (1992-94), which featured nine complex VR works including Dancing with the Virtual Dervish by Diane Gromola, Marcus Novak and Yacov Sharir. Osmose, a VR environment created by Char Davies and a team of engineers from the SoftImage company, gained notoriety when it was exhibited at ISEA '95. Virtual reality was hailed as a new medium in the early 90s, but due to the immense computational labor and arrays of equipment involved, the creation of immersive environments understandably has not yet become a common language, even as choreographer Yacov Sharir describes his experience in very provocative terms: What artistic, intellectual, kinesthetic, and emotional issues could be addressed using this technology?… Virtual technologies allow us to manipulate, extend, distort, and deform information as well as experience of the body. They are vehicles that enable us to extend and color work in many ways, some of which may not be possible in the physical realm and/or by traditional means. They offer a way to augment and extend possibilities creatively, experientially, spatially, visually, sonically, and cognitively.(19) The augmented interface he recounts is a convergence of the interactive principles I described above and an immersive experience within a real-time, 3-D graphic and aural environment generated by computers. Sharir refers to it as a "distributed performance environment" which he entered and inhabited with a head-mounted display and dataglove. The three-dimensional world, created by Gromola, projected on an enormous scale the torso (skeletal spine, pelvis, ribs) and inner organs of her body built from X-ray and MRI data. Sharir notes that when moving through the virtual torso he also encountered digitized images of himself dancing, which diffused and multiplied his sense of being inside an other body. Much could be said about the psychological and cognitive effects of navigating immersive environments, and literally moving into someone else's (virtual) body also strikes me as a politically and ethically charged process. While the literal involvement of the performer's or audience-interactor's body and neural system in the image of Gromola's medically scanned body raises many culturally sensitive issues about contemporary (bio)technology, Sharir's experience of the distributed self-as-image refers us back to the motivations for such performance research in the first place. He clearly states a desire to provoke questions about human beings, subjectivity, perceptual systems, and how we re-envision and re-configure ourselves through technology.(20) On the computational and formal level of making work, however, he seems largely content to explore "possibilities" for a new spatiotemporal aesthetics, asking where one can locate his performance. In the real space or the virtual world? In the pursuit of knowledge about the body and its movement possibilities I detect a cybernetic impulse that surely drives the software development. The interactive, multisensory, and reflexive body developed in our artistic training is merged with an informational conception of the body: the "bioapparatus," as Sharir calls it, is tracked, and the data it offers are computed and then extended outward into controlling devices that simulate and regulate the behavior of an organism, a virtual world or any simulation system through feedback. Sharir and other choreographers are fascinated with the transmutational potential of the software to project a body immersed in fluid, non-linear and non-mimetic environments. There is intense spatial ambiguity, since the immersant retains a felt body that is, however, mapped quite differently than the seen bodies that may float through the animate virtual environment. As with movement through synthetically constructed game worlds or "virtual theatres," such floating or flying evokes the anti-gravitational and disembodied vistas of a cyberspace which effectively rules out social interaction with an audience. There are many representational dilemmas when interfacing with dataworlds, but our main question here will be where to locate the dance. Scott deLahunta has pointed out that we cannot avoid to pay more attention to software development for artists by artists. "The process of computation is invisible in the simplest sense that the labor of the software programmer or engineer is largely taken up in the 'writing' of an instruction that tells the computer hardware and connected peripherals how to execute an operation."(21) This writing and rewriting is part of the creative process whereby an interactive virtual environment is made. It is interesting, in this respect, that choreographers have been working with code that was by and large written by and for musicians (BigEye, Image/ine, MAX/MSP, VNS). There has been much debate about dance making and interactive systems, especially with regard to "transparency" and the receptivity of an audience to the aspects of the work that might be invisible. What is being considered "invisible" in this context is the "mapping" from input to various forms of output. "Mapping" is not a spatial representation, it is an operation that assigns continuous MIDI data received from a tracking system to control DSP functions such as filters, low frequency oscillators, distortion algorithms, etc. "The interpreted data provides information about the speed, direction, and location of moving objects in the video image, and that information can be used to provide input control data to music-generating software." (22) This is essentially what is referred to as an interactive system. Mapping, therefore, is at the heart of the creative process as regards these systems. However, deLahunta argues, it is the manifestation of mapping that enters the field of perception of the viewer/listener, not the mapping itself. Once completed, the instructions that comprise the mapping itself are relegated to the invisibility of computation. How this invisible mapping works is of interest primarily to those who are engaged in its construction. Although deLahunta is correct in observing this gap between computation and choreography, he may underestimate the curiosity with which some dance companies have approached the relations between writing operations, algorithmic composition and movement creation. The Frankfurt Ballet, Jo Fabian, Jean-Christoph Maillot's Ballets de Monte-Carlo, Pablo Ventura Dance Company, Yacov Sharir, Isabelle Choinière, among others, have developed rehearsal systems influenced by computational thinking. Reversing this process, physical performance expression could inform and nurture an adaptive code structure which "learns" over time from performance expression that does not follow the rules. What remains to be seen is whether artistic work with interactive systems allows audiences access to all facets of the systems -- input, mapping, and output. Dance performances using interactive systems generally tend to give an audience access only to the output, while interactive installations allow access to input and output. deLahunta proposes to "include exposure to the mapping itself during performance." This corresponds to my early experiences with motion capture. In the initial stages of the capturing process performers/audiences generally cannot see the data that are recorded, nor can they experience in real time how the recording data might be mapped onto a character or figure animation. This becomes possible, however, with real-time magnetic and optical capturing systems that wire the hardware/software to video projectors which display the data processing and mapping immediately to the performers and to audiences. A real-time closed circuit relationship to the mapping affords the performer who practices with these systems a training environment for more complex interactions with them, thus combining input measurement that responds to a higher level of detail and subtlety in performer action with more complex mappings. Confronting proprioceptive and cognitive questions about the immersed body requires intensive practice with interface methods that manipulate the intermingling of self and virtual world, and a more in-depth understanding of the programming language itself. But where in the performing arts do we discuss notions of performer training overlapping with the development of interactive systems? Do we create learning environments in which dancers and musicians could practice in depth with interactive lighting, sound and video projection systems, especially if the latter (midi operated) depend on the fine tuned lighting and calibration of camera sensing systems, as well as on aesthetic lighting design choices (in conjunction with the use of video projections) that are an integral part of a multimedia performance work? Networked Interactivity Networked or distributed environments and real-time processing of performance as multimedia objects for transmission again require new conceptual models that take into account the specific integration of video, communication and network technologies into performance environments and the international co-production and project management of remote and multiple site events with streaming media, web technologies, real-time video production/ editing, and collaborative compositional practice. The "intelligent stage" of the future may not be a theatre but the network itself. Yet in order to transmit movement-images, a dance has to "happen" at some point in real time/real space. Telematic performance thus harbors beautiful paradoxes, as transmittable data have to be produced and processed in synchrony between different locations which may involve different environments.
In a teleperformance created at IDAT, the Australian Company-in-Space staged Escape Velocity as an interactive duet between two dancers, two cameras, and two projectors linked by a direct online connection between the Web Cafe at Arizona State University and a performance space in Melbourne. The live mix effectively merged the two dancers, layering the choreography and the bodies in a spellbinding, transparent symmetry across a vast spatial and temporal gap. It was transparent insofar as we knew that the teleconference had been set up between Arizona and Australia, we could see the audience Down Under. And when Hellen Sky started her dance in front of our eyes, we could see the projected image of her sister dancing the same choreography in Melbourne, and the two camera artists on either end of the performance began to interact with the performers and send their video signals through the line. At various points during the performance we could imagine the dancers being at-one, the sisters becoming a composite dancer floating in a third space created by the overlaid projections which included film footage of several outdoor locations (a forest, a desert). More hauntingly, the apparent symmetry of the dance of course was not precise. Tiny delays in the transmission became part of the choreography and entered into the dialogue between present physical body and technologically mediated body. Ironically, both dancers were simultaneously mediated and transprojected. At the moment when these dispersions become possible, all safe parameters of the body's relationship to space, time and place have shifted. We witnessed a dialogue between spectral dancers mixed onto the pixilated, filtered and manipulated surface of the filmic space created by the projectors, the dance a traveling across time, the body morphing and aging right in front of us.
In'ter, 2000 Two main operational principles can be said to determine the parameters of telematic dance: interactivity and telepresence. Since such work involves real performers who generate digital objects in a real space, it is important to keep the physical dimension of the collaboration in mind, in order to distinguish it from online digital art that gives complete primacy to computer-generated design information or animation over physical space. The sensitive interactive environment in a real studio is a precondition for the creation of movement images transmitted as digital video and sound, since such sensory environments allow the practitioners to experiment with all available interface elements and the aesthetic and psychological aspects of network delay and image/audio degradation. Since live streaming media are contingent on network traffic, the compositional process can regard frame-rate, pixilation, image-disintegration and audio interferences as meaningful conceptual and thematic concerns. Secondly, computer and Internet-based telepresence allows performers to be present at a distance and to act upon others in a real remote physical location via live video images. I emphasize the notion of action or "acting" in order to distinguish telematic dance from telecommunications that exchange data or manipulate objects and affect remote environments via telerobotics, for example. Telepresence is a challenge since we have no existing aesthetic or cultural models for real-time dance interaction with a physically remote location, nor will we know much about the role of our potential Internet audiences unless we invite them into the mix. The bridging of spatial distance via telecommunications, as we operate out of our camera-originated environments, allows us to examine the emerging conventions of "networked dance" as new types of montage, layering, filtering, editing, mixing, and transcoding which are to a certain extent derived from our experience of multimedia performance and interactive art. In this sense, networked dance can be considered an extension of our prior experience with the technical production of videodance. Online performers primarily compose with selections of video/audio data, camera angles or sensor-controlled interfaces, text and chat, and of course with software and the operations embedded in it. In the virtual laboratory of interconnected distances, performance process shifts continuously between physical object (movement, gesture, voice, word, etc) to electronic signal which can be modified in real time by passing it through filters and combining it with other data and interfaces. Starting with In'ter in early 2000, I've named my lab performances "environments." We've transformed our studio into a virtual laboratory that enables us to rehearse new performance operations which are inevitably connected to media and art practices, interface designs, and visual and sonic languages in a number of other areas of culture. As teleconferences, cellular phones and portable media become common practice today, telematic performance is a cultural interface that aims perhaps at specific aesthetic and socially interactive dimensions. It may seem to belong to the representational theatre paradigm, but it is telecommunicational and modeled after the operating systems of computer software applications. We are opening up small windows. This strange paradox deserves careful attention, as we gradually become the (software) designers for our own interactive plays, propelling us out into the world, into new kinds of cultural conversations and exchanges.
1 Cf. Critical Art Ensemble, Digital Resistance: Explorations in Tactical Media (Brooklyn, NY: autonomedia, 2001), p.2. Ricardo Dominguez, in a recent talk on his work with CAE, Electronic Disturbance Theater and Fakeshop, coined the term "hacktivism" when refering to his online art and activism which is focused on the development of electronic civil disobedience in relation to globalization, neoliberalism in the Americas, and the Zapatista struggle. 2 A new motion capture studio was opened at the
Advanced Center for Computing in Art and Design (Ohio State University)
in 2001, and the dancers in my lab, along with Michel Marceau and Australian
performance artist Stelarc, were among the first to be asked to capture
data. My "Environments" Lab, to which I make reference in
this essay, was initiated in 1999 when I was 3 Cunningham first worked with LifeForms in 1990 for the creation of "Trackers." His interest in and use of video dates back to 1974, followed by projects with Charles Atlas, Elliot Kaplan, and other filmmakers. For Ventura's work, see <http://www.ventura-dance.com> 4 Space doesn't permit a fuller technical description of some of the software systems that have been used in performance, but at least I want to mention the developers: VNS (David Rokeby), BigEye and Image/ine (Tom Demeyer, STEIM), EyeCon (Frieder Weiss, Palindrome Inter-media Performance Group), MAX/MSP (David Zicarelli at al., Cycling74.com), Nato (Netochka Nezvanova), ChoreoGraph (Nick Rothwell, Barriedale Operahouse), EyesWeb (Antonio Camurri, Laboratorium für Musik-Informatik, Genua), Isadora (Mark Coniglio, Troika Ranch), and Keystroke (Eric Redlinger, Sher Doruff, WAAG). An overview of software development for performance is offered by Scott deLahunta: <http://huizen.dds.nl/~sdela/transdance/report/>. deLahunta organized the pathbreaking workshop "Software for Dancers" at Sadler's Wells Theatre in London (October 2001). It was followed by "Performance Tools: Dance and Interactive Systems," a think tank I organized at Ohio State University in January 2002; cf. <http://www.dance.ohio-state.edu/workshops/ttreport.html>. The IRCAM website has a bibliography of significant research in interactive systems: <http://www.notam.uio.no/icma/interactivesystems/dance.html> 5 Cf. Nicholas Negroponte, The Architecture Machine: Toward a More Human Environment (Cambridge: MIT Press, 1970), p. 101. For an excellent critical overview of the evolution of interactive art within the context of the visual and media arts, see Söke Dinkla, Pioniere Interaktiver Kunst (Karlsruhe: ZKM/Cantz Verlag, 1997). 6 "Nervous environment" is derived
from the term sound artist David Rokeby uses for his interactive software
"Very Nervous System" (VNS), first created in 1982. VNS uses
video cameras, image processors, computers, synthesizers and a sound
system to create a space in which the movements of one's body create
sound and/or music. In his writing Rokeby has pointed out that VNS is
not a "control system" but an interactive system, by which
he means that neither partner in the system (installation and moving
person) is in control. "Interactive" and "reactive"
are not the same thing, according to Rokeby. "The changing states
of the installation are a result of the collaboration of these two elements.
The work only exists in this state of mutual influence. This relationship
is broken when the interactor attempts to take control, and the results
are unsatisfying." Quoted from "Lecture for 'Info Art', Kwangju
Biennale," 1996 [http://www.interlog.com/~drokeby/install.html].
For an illuminating discussion on lighting and choreographic rehearsal
process, see Senta Driver, " William Forsythe and Jennifer Tipton:
A Conversation about Lighting," Choreography and Dance 5:3 (2000),
41-78. 8 For an extensive documenation and critical discussion of his interactive art project, see Rafael Lozano-Hemmer, ed., Alzado Vectorial/Vectorial Elevantion: Relational Architecture No.4 (Mexico City: Conaculta, 2000). 9 Maja Kuzmanovic and David Tonnesen, "T-Garden," Presentation at "Subtle Technologies" Conference, May 2001, Toronto, Canada. Quoted with permission. For more information on the project, visit <http://f0.am/> and <http://www.subtletechnologies.com/2001/index.html> 10 The "T-Garden" project derives some
of its conceptual terminology from theatre, understood as spatial practice
and social drama (Victor Turner), cultural anthropology, virtual architecture
and current research in hybrid spaces, and the world of 3D game design.
It cannot be used as an example of interactive digital art designed
for the general "user" in clearcut distinction to interactive
dance and interactive music performed by trained professionals. The
sliding between performer and recipient needs to be carefully theorized,
especially as an increasing number of choreographers, composers, visual
artists and programmers collaborate on the design of interactive installations
for audience-play. In the summer of 2002, together with 30 other artists,
I participated in such a design project ("Real-Time and Presence:
Composition of Virtual Environments") at the Trans-Medien Akademie
in Hellerau-Dresden (Germany), and with the exception of the French
musicians Bertrand Merlier and Jean-Marc Duchenne, all other teams created
environments for audience-interaction (http://www.t-m-a.de). The theoretical
debate on interactivity meanwhile is still in its infancy. Cf. Annette
Hünnekens, Der bewegte Betrachter: Theorien der Interaktiven Medienkunst
(Köln: Wienand, 1997); Peter Gendolla, Norbert M. Schmitz, Irmela
Schneider und Peter M. Spangenberg, eds.,Formen interaktiver Medienkunst
(Frankfurt: Suhrkamp, 2001); Roy Ascott, ed. Reframing Consciousness:
Art, Mind, and Technology (Portland: Intellect, 1999); Armando Menicacci
and Emanuele Quinz, eds., La scena digitale: nuovi media per la danza
(Bolzano, Italy: Marsilio, 2001). 12 For a fascination discussion of the use of "dis-focus" in the complex rehearsal operations practiced in William Forsythe's Frankfurt Ballet, see Dana Casperson, " It Starts From Any Point: Bill and the Frankfurt Ballet," Choreography and Dance 5:3 (2000), 27ff. In his pathbreaking new book, Lev Manovic examines digital composition from within the history of the cinematic and postcinematic. Cf. The Language of New Media (Cambridge, MA.: MIT Press, 2001). 13 Cf. Lisa Marie Naugle, "Distributed Choreography," Performing Arts Journal, 71 (2002), 56-62. PAJ 70 and 71 presented special issues on dance and digital media, edited by Johannes Birrringer. 14 See also the documentation/theorization of the telepresence experiments conducted by ADaPT (Asssociation of Dance and Performance Telematics): http://www.dance.ohio-state.edu/workshops/ipstheory.html. Cf. Johannes Birringer, Ellen Bromberg, Naomi Jackson, John Mitchell, Lisa Naugle, and Doug Rosenberg, "Connected Dance: Distributed Performance across Time Zones," in Transmigratory Moves/Dance in Global Circulation, Congress On Research in Dance Conference Proceedings, New York University (October 2001), pp. 51-77. See also Richard Loveless and Lizbeth Goodman," Live and Media Performance: The Next Frontier," Performance Research 4:2 (1999), 74-75. 15 The group group Interface (Bahn, Trueman,
Hahn) presented these two performances in a collaborative concert, "Moving
Voices," at the end of an international workshop on dance and interactive
technology at OSU, June 30, 2001. For more information on their work
visit the websites: <http://www.arts.rpi.edu/crb/interface/interface.htm>
16 Coniglio, a musician and software programmer known internationally for his work with Troika Ranch, a New York City based company he directs with choreographer Dawn Stoppiello, wrote two interactive programs, Interactor and Isadora, which map data input to control a variety of media outputs, e.g. sonic, video, lighting and robotic. Troika Ranch conducts regular Live Interactive workshops to give participants the opportunity to explore the use of interactive computer technology in performance (http://www.troikatronix.com/) 17 For a more detailed discussion of the Cross fair event, see my "The Intelligent Stage," Performance Research 6:2 (2001), 116-22. 18 Cf. Ann Dils, "The Ghost in the Machine," Performing Arts Journal, 70 (2002), 94-104. William Kentridge's work is a brilliant example of the cross-overs between drawing, animation and theatre and presents a different form of digital storytelling concerned with narrative content rather than abstract motion. His transformations of drawings into animated films (e.g. Felix in Exile, 1994; History of the Main Complaint, 1996) struck me as dark expressionist landscapes, CT scans of a history of violent repression and memory, guilt and forgetting. He has also used his "drawings for projection" in collaborations with Johannesburg's Handspring Puppet Company. See Yvette Coetzee, "Visibly Invisible," South African Theatre Journal 12.1-2 (1998), 35-51, and Leah Ollman, "William Kentridge: Ghosts and Erasures," Art in America 87.1 (1999), 70-75; 113. 19 Yacov Sharir, "Dancing with the Virtual Dervish: Virtual Bodies," in Immersed in Technology: Art and Virtual Environments, ed. Mary Ann Moser with Douglas MacLeod (Cambridge, MA.: MIT Press, 1996), pp. 281-85. Virtual reality works don't travel easily because of the complex hardware/software set-up. Sharir was able to perform in a new version of "Virtual Dervish" recreated by the National Museum of Contemporary Art, Athens,Greece, on October 14, 2002. During the run of the exhibition, visitors experienced the work (one by one) by wearing a head mounted set/display and an electronic glove designed to assist them in the navigational process. 20 For the current debate on biotechnologies and genetic engineering, see for example Gregory Stock, Redesigning Humans: Our Inevitable Genetic Future (New York: Houghton Mifflin, 2002); Chris Hables Gray, Cyborg Citizen (New York: Routledge, 2002); Jeremy Rifkin, The Biotech Century (New York: Tarcher/Putnam, 1998), and Mark Dery, The Pyrotechnic Insanitarium (New york: Grove, 1999). There is also a growing body of feminist critiques of biotechnologies inspired, for example, by Donna Haraway's earlier writings on biology and cyborg organisms. See her Simians, Cyborgs, and Women: The Reinvention of Nature (New York: Routledge, 1991). 21 This passage, and some of the subsequent observations, are indebted to conversations and an unpublished manuscript, "Invisibility/Corporeality," which Scott de Lahunta presented in my Environments Lab during his residency at the Interactive Performance Series (April 2001), Ohio State University. deLahunta suggested that "writing is arguably not the best descriptive metaphor for software programming - building is preferred as often coding requires increasingly the reuse or reassemblage of previously written code." [Quoted with permission of the author.] See also deLahunta's analysis of Blast Theory's building of a mixed-realty environment in their installation Desert Rain ["Virtual Reality and Performance," PAJ 70 (2002), 105 -14]. 22 For a useful and straightforward breakdown, see Christopher Dobrian's website "Video motion tracking for musical input": <http://music.arts.uci.edu/dobrian>. See also Todd Winkler, "Motion-Sensing Music: Artistic and Technical Challenges in Two Works for Dance": <http://www.brown.edu/Departments/Music/faculty/winkler/>
Independent choreographer/videomaker
Johannes Birringer is artistic director of AlienNation Co.. He has created
numerous dance-theatre works, video installations and site-specific
performances in collaboration with artists in Europe, North America,
and Latin America. Over the last eight years he has taught several international
workshops on performance technologies in Europe and the U.S. He is the
author of several books, including Theatre, Theory, Postmodernism (1991),
Media and Performance: along the border (1998), and Performance on the
Edge: transformations of culture (2000). After heading the Dance &
Technology Program at The Ohio State University, he founded the Interaktionslabor
Göttelborn in Germany. He is currently Professor of Performance
Technologies at Brunel University, West London.
|