Design And Performance Lab
|
Dance and Interactivity
Johannes Birringer (c) 2003
Split (2000)
New Environments for Dance
DJ Spooky
At the turn of the new century, many interests in related fields -- film, electronic music, digital art, science and technology, design, engineering, robotics, telecommunications -- advance our understanding of the complementary thinking processes that drive new interdisciplinary research and conceptual models influenced by the computer's information processing capabilities and the internet's global reach. Performance incorporated new compositional ideas and instruments such as cameras, video-projectors, microphones, sensors, synthesizers or computer softwares. Like music before it, dance extended its reach; choreography and movement composition now share the language of programming, design, animation and film editing. In the following, I propose to look at ways to describe a new aesthetics of interactivity in dance which recognizes the technical context of programming languages and the artistic challenges we face when working with real-time processing. I also claim that the proscenium and conventional production processes have become inadequate. New dance, involving technologies and interactive designs from the conceptual starting point, requires a different environment for its evolution. I will offer an overview of the terms crucial for an understanding of interactive dance, especially as they relate to new training environments and "performance systems." These terms have gained currency in the emerging field of dance and technology, while being intrinsically connected to music, electronic instrument design, architecture, digital art, and the popular culture of video and computer games. My observations are based primarily on my own practical experience and that of my collaborators in the field.
I use the term "interactivity" with regard to two phenomena. First, I address "interaction" as a spatial and architectural concept for performance, and secondly I look at "interactivity" in the more narrow sense of collaborative performance with a control system in which the performer-movement, gesture and action are tracked by cameras/sensors and thus used as input to activate or control other component properties from media such a video, audio, MIDI, text, graphics, QuickTime movies, scanned images, etc. In the latter case we speak of an interactive system that allows performers to generate, synthesize and process images, sounds, voice and text within a shared real-time environment. Before examining the behavior of such systems, I want to apply the notion of interaction to the historically evolved understanding of multimedia performance as a multi-faceted and multi-dimensional dynamic process, either based on choreography or improvisation and more open-ended, fluxus-like constellations. Historically, interactivity as an aesthetic category would not derive from classical concepts of composition or choreography; rather, it is indebted to the early 20th century avant-gardes and their experiments with performance as live concatenation of different, sometimes conflicting media (dadaist, futurist, surrealist performances) as well as with performance as a conceptual instrument for the activation and provocation of the audience. Art history derives its understanding of interactive media arts predominantly from the participatory events of the 1960s (happenings, Fluxus, process art, Situationism, kinetic art, concept art, "art and technology," the John Cage/Robert Rauschenberg collaborations, cybernetic art, closed-circuit video installations, etc) and the progressive "dematerialization of the art object" which implied the active, physical participation of the audience in the event. Since the 1970s, interactivity in art generally refers to multimedia installations and environments that involve electronic or computer-assisted interfaces. Nicholas Negroponte already suggested in 1970 (The Architecture Machine: Toward a More Human Environment) that such interfaces are characterized not only by the points of contact and interaction between a machine and the physical or information environment, but by the artistic strategies used to engage audiences in a dialogue.(2) Compared to interactive installations and digital artworks, sound sculptures, immersive environments, computer games, and more recent internet-based forms of telepresence, interactive dance in the strict sense of computer-assisted design cannot claim such a long and heterogeneous history. Dance makers have largely remained committed to presentational stagings of multimedia works - complete and highly structured works for the consumption and aesthetic contemplation of the audience. Dance installations and interactive online dance which engage active viewer-participants are rare events that require careful attention and analysis, especially since we don't have any established aesthetic or social criteria for the evaluation of a successful interface. Regrettably, the professional and academic dance community has not found many commonalities with the vibrant club and rave cultures either, which to some extent has contributed to a sense of isolation among younger dance artists growing up with computers, music television, techno, hip hop, and the transnational exchanges and cross-overs in music. The problem that I see is the overwhelming emphasis, in academic and concert dance training and its specialized professional practice in the West, on specific techniques (technical training in ballet and modern dance), vocabularies and compositional structures which have limited usefulness for an exploration of participatory processes and the integration of recipient behaviors and feedbacks. Moreover, dance practice as it is commonly understood in Western training has been largely focused on the performer's physical virtuosity and bodily intelligence, shaping and disciplining the body for the execution of choreography, and not for interaction with mediated and unstable environments. Addressing "interaction" as a spatial and architectural concept for performance, therefore, means shifting the emphasis away from the creation of steps, phrases, "combinations" or points on the body that initiate movement, away from the dancer's internal bodily awareness (widely encouraged in today's practices of yoga, somatics, experiential anatomy, body-mind centering and release techniques) unto her environment, to a not-given space but a shifting relational architecture that influences her and that she shapes or that in turn shapes her. Shifting attention to touch, weight and energy transfers in partnering, as it is practiced in contact improvisation, is a good preparation for working with physically motivated sensor interfaces, especially wearable sensors. This re-orientation also implies an initial awareness of how lighting sculpts space, and how lighting color, angle, temperature and intensity are constituents of the dynamic and intermediating plasticity of space that creates opportunities for movement. Moving bodies and changing light, along with the crucial experience of the resonating body within a reverberating sonic environment, are part of the collective consciousness in which we are enveloped and in which we are co-creative participants. This notion of a resonating environment, in my own practice, is indebted to the plastic sculptural process that dancers, visual artists, media artists, programmers and architects have recently explored -- a plastic process of "designing" fluid space that allows for integration of "nervous" or sensitive media presences.(3) In a sense, I see the sculptural process as a contemporary modification of Laban's Space Harmony, of the Bauhaus principles of synaesthetic abstract constructivism, and of Joseph Beuys's and Hélio Oiticica's enactments of "social sculptures." In philosophical terms, I am also suggesting a non-Western and non-Euclidian approach to spatial "science" and geometry: There is a need for a philosophical framework
that enables us to engage harmoniously with the contextual living space
in which we are immersed and from which we are as inseparable as a whirlpool
is from a water flow. Reversing the man-induced ebb of essential harmonies
may come through a philosophy of 'inclusionality' wherein, as in the
wisdom of indigenous traditions, all things are understood to be dynamic
contextual inclusions that both include and In other words, a relational performance architecture is participatory, and it does not exclude virtual architectures, as Rafael Lozano-Hemmer has suggested in his writings and artistic projects, for example his highly charged public interface event "Vectorial Elevation" (1999-2000), a transformation of Mexico City's Zócalo Square with enormous light sculptures created by participants on the Internet using a virtual reality program.(5) On the contrary, dance and the changing notions of "site-specificity" in interactive installations need to be discussed with regard to virtual reality environments and such models of immersion that integrate physical and synthetic, 3-D simulated environments, in order to perceive the connections between designs based on representational space and designs generated from algorithms. Current developments in computer science, artificial life research and 3-D design programming (VRML) point to hitherto unimagined combinations and hybrid environments for performance and play which could have a considerable impact on collaborations between choreographers, composers and designers interested in complex, imaginative and dynamic "improvisation technologies," to use the term that William Forsythe applied to his rehearsal operations. I want to give an example of such research to clarify my point. At the 2001 "Subtle Technologies" Conference in Toronto, Maja Kuzmanovic and David Tonnesen showed a computer simulation of the "T-Garden" project they were developing with their FOAM initiative at Starlab (Brussels). Tonnesen emphasized the interdisciplinary nature of the "T-Garden" collaboration and explained its conception: It is a responsive/hybrid play-space where visitors
can 'converse' with sound, 1. How do we develop sustainable, international
collaboration networks 2. How do we allow the project to evolve in
the most open and interactive 3. How do people individually and collectively
make sense of responsive, 4. Can play (in the broadest sense of the word)
become an essential model 5. How can new forms of expression be sustained
by a fusion of media, These questions point to the heart of current experimentations with interactivity, which for dance makers until very recently was largely a dialogue with composers and programmers who designed MIDI-activated sonic environments for non-linear choreography. "T-Garden" suggests an expanded architecture allowing the performers or "gardeners" to experience physical and tactile relationships to a virtual reality that they can actually modify and shape, moving through the projective, computer-generated world. Since this world needs to be projected via surround-sound speakers and LCD projectors, it means that the performer moves through light waves, fields of color and pulsations, floating virtual objects, etc., and her body potentially experiences ruptures of the kinesthetic from the visual senses as all physical body-surfaces gain a multidimensional tactile extensionality. We also note that it is the audience who become the performers and "gardeners" in this interactive installation. Its logic of "interactivity" in play aims at dissolving spectacle and the featured stage performer altogether.(7) Such "movement-through" interactive and generative environments posits a shift in perception that many dance practitioners, used to working in delineated real space - with a forward orientation on the proscenium and toward a passively spectating audience - have been reluctant to engage. The engagement requires new vocabularies informed by interactive design and VRML (Virtual Reality Modelling Language) and involves such notions as parameters, mapping, navigation, tracking systems, Musical Instrument Digital Interface (MIDI), genetic algorithms, modules and patches based on specific programming languages such a MAX/MSP. It also requires at least a basic understanding of the underlying computational processes which generally remain invisible. And it prompts us reflect on contemporary science-derived concepts of "emergent" or autogenerative systems as they are now being used by diverse artists working with interactive videoinstallations, artificial life architectures, 3D shared spaces, and telepresence.
Navigations and Interfaces Technology has decisively challenged bodily boundaries and spatial realities, profoundly affecting the relations between humans and machines. The convergences between dance and technology reflect back on the question of dance and its physical-sensory relationship to space and the world, its immediate, phenomenological embodiedness to lived experience in one place. We are still in one place among other moving bodies when we dance, whether we are in a rehearsal studio or in the street or a discotheque. But if dance indeed takes the lead, among the theatrical arts, in absorbing technology as a creative tool, it needs to revise its rehearsal methods and training facilities. First, interactivity has implied the relocation of the compositional process into a laboratory-like environment. The directors of the ISA at Arizona State University call it the "intelligent stage," where dancing takes place with computer-assisted design and MIDI interfaces in an interactive ambience which allows a different "programming" of physical motion and motion sensing. At ISA the stage is wired for internet access and telematic transmission of streaming video and MIDI signals, while also featuring the "Very Nervous System" design developed by Rokeby. As with other tracking systems such as BigEye, EyeCon, or Isadora, the sensing in the VNS interface is done by cameras and motion detection devices. But the dancers also become "sensors," adopting to a new spatial awareness of a digitally enhanced space or "operating system" which triggers responses and feedback. Dancers appear to be touching invisible partners; they become ghostcatchers. (8) Secondly, the engineering of interface designs moves to the foreground in such labs, and the term "sensing" gains a dimension reaching beyond the physical and organic understanding of bodily anatomy, musculature, and proprioceptive spatial awareness of moving-within-the-kinesphere that dancers trained in modern traditions (after Laban) bring to the studio. The convergence of interface design and movement analysis extends earlier Laban-derived structural explorations of the body's repertoire for movement. In more than one sense, it involves the entire sphere of movement as interaction, encompassing perceptive and receptive processes. If movement is a "continuous current" (Laban), a new understanding of "interspaces" in networked performance is now evolving. The interactivity of sensitive environments is one crucial aspect of it. The notion of real-time flow changes, as the environment also functions as a video studio or soundstage, and cameras, sensors and appropriate lighting need to be continuously calibrated. There will be constant interruptions. If the environment is networked, there will be delays in the uplink/downlink teleoperation; such delays might affect kinesthetic perception. The most significant intervention into movement today is the dis-location, and subsequent re-distribution, of movement as captured and processed image, micro-movement, sampled ghost. Movement, as it is used in interactive and networked performance-installations, is not a continuous current with space itself but continuously crosses between real space , projective space (video/animation) or other virtual contexts (VR, remote sites).
ADaPT, multi-site telepresence performance,
Space is dematerialized, movement is captured, commuted, transferred and rematerialized elsewhere; we interact with sensory information such as video which projects different three-dimensional kinesthetic perceptions of movement energy, position, and velocity (cf. slow motion, close-ups, different scale, distorted color/pixilation, dis-focus, etc). (9) The programming of interfaces between dancers and the computer implies the creation of an unstable system. "Choreography" more closely resembles the "live mix" we experience in techno culture when DJs create a situation, a sound continuum, and use filter devices to modify the parameters in response to energy that is transferred between dancers and musical stream. The intensities of the event develop a kind of autopoiesis; in current dance experiments with interface designs based on feedback/triggers in real time, the composition process is like an "emergent system": symbiotic improvisation with invisible sensor lines or dynamic fields in space.
Dance, closely associated with visual forms and
rhythms, is fundamentally a multimedia system. We know from photography
and motion studies that performances were staged exclusively for the
camera. Choreographers discovered that videodance is a composite medium
in its own right: choreography is editing of frames. Making dances for
the camera has become not only a cinematographic alternative to live
dance, but motivated choreographers to re-conceive the aesthetic framing
of live dance. The impact is evident in the cinematic quality of many
contemporary works. The Japanese companies Dumb Type (Memorandum) and
OM2 (The Convulsions of Mr K.) literally used no less than six simultaneous
screen projections in recent performances. Such projective video topologies
need to be examined as moving architectures in the environment. Video
projection opens up a screen space for movement Dancers become conscious of the deep structure of computer interfaces, learning to navigate expanded spheres of movement that require a radical reordering of the senses due to an increase in telematic or virtual interaction. We are engaged in a new form of motion studies, and in the analysis of its remote effects. To my knowledge, there are four types of environments currently evolving in dance: (1) interactive environments (based on sensors and motion tracking); (2) derived environments (motion-capture based re-animations of bodily movement or liquid architecture, which can also be networked and reintroduced into live telepresence or telerobotic operations and communications between remote sites); (3) immersive environments (Virtual Reality based, such as the "Cave" or panoramic installations - similar to "T-Garden" - that integrate the body, with stereoscopic devices in front of the eyes, into the polysensual illusion of moving through space); and (4) networked environments (telepresence, videoconferencing and telerobotics, allowing users to experience a dispersed body and to interact with traces of other remote bodies, avatars and prostheses).. The parameters of all of these environment types can be mixed; we can then speak of mixed reality environments. In the following sections I describe these environments and how they affect performers and observers.
Interactive Systems The first type of environment translates physical gestures of dance via sensors, motion-tracking cameras and analog/digital converters into a signal representation inside of a computer program. Looking at the MAX/MSP software as an example of interactive systems, we encounter specific design features that organize the relational architecture. MAX is a graphical programming environment that allows the building of controllers for real-time synthesis and signal processing. The gestural data, received via a sensor system, is mapped to the control of a given sound synthesis parameter. The continuous MIDI data - generated by physical gestures of the dancer or musician (playing instruments with sensor devices) - affect the performance parameters of the MAX/MSP patch which runs on the computer and effects the sonic and graphic output. As an instrument the MAX software primarily controls the source materials (the sound and video files stored in the computer or synthesizer), sound parameters, and the dynamics of real-time synthesis. It can harbor considerable complexity since patches might be constructed in the manner of a "nested" design -- enfolded entities that are in a continuously fluctuating state of unfolding to activate the modular parts. What is crucial about the interaction is the range of responsiveness of the controller with regard to the dynamic range of expression of the mover. Beyond simple cause and effect, I should think that the system only becomes challenging and valuable to the dancer once she can play with it and modulate the sonic space and musical parameters around her to allow herself and the audience to perceive the real-time conversions of movement into sound. Given such complexity in the programmed environment,
we must ask how performers and musicians regard the physical relations
between performance and "controlled" parameters, and
From a choreographic point of view, the dancer
within an interactive environment resembles the "player" in
"T-Garden" -- she will need to familiarize herself with the
response behavior of the sound and video parameters, and both player
and composer will strive to create an exponentially more sensitive,
articulate and intuitive system. In a shared environment this could
mean refinements in sensors, filters, and output processors, but also
an attenuation of the performer's spatial-temporal consciousness. How
is the performer-musician-system relationship evolving, emergent? What
can we learn from jazz-improvisational structures, from video game structures,
from different cultural contextualizations of virtual environments?
Corpo Aberto (2000) As these examples illustrate, dancers, composers, and designers can interpret the relational architecture of interactive systems in many different ways, depending on a work's emphasis on dance gesture-to-music synthesis, or dance gesture-to-video synthesis. Robert Wechsler (Palindrome Intermedia Performance Group) recently suggested in an internet posting that mapping strategies should address the basic problem common to most intermedia pieces which place dancers in the role of musical performers: namely, how to create an interdisciplinary work that succeeds from all the choreographic, music-compositional, and filmic perspectives. What he implies, of course, is that the dancer is or becomes the "musical performer," or, in Santana's case, plays with being the eye of the camera, which is not the same as inter-acting with a dynamic multisensory sound/video environment which may respond in unexpected and uncontrollable ways. Yet the question remains whether choreographers and composers have different or conflicting goals, or whether there is an aesthetically stringent co-resonance between movement, sound, and video that can transform the entire environment kinesthetically. Let us look at two other interactive works, presented at CROSS FAIR in Germany, which connect the first and second types of environment. Scanned, conceived and directed by Christian Ziegler, is a performance-installation that consists of video projections of a dancer's scanned movements. To create the live work, Ziegler first asks dancer Monica Gomis to perform movement phrases lasting from one to fifteen seconds, which are taped by a video camera. A program wrote for the computer allows him to let a digital video scanner unfold the movement-images over time, controlling direction and speed of the scan as well as resolution and tempo of the scanned material. In performance, the scan projections slowly emerge over a period of time, as if we were watching a painting come into life. The "choreography," according to Ziegler, can be seen by the imagination of the viewer. One could also argue that there is no choreography, but that the interface with the computer creates temporally dilated paintings of human gestures and movements, completely reorganizing the time and space of the dance frames.
Scanned (2000)
Yours (2000)
Yours, a collaboration between the Polish composer Jaroslav Kapuscinski and Frankfurt Ballet dancers Nik Haffner and Antony Rizzi, is performed as a dialogue between a pianist and a video projection of a dancer, accompanied by percussion sounds and a female voice (reciting from Beckett's "Texts for Nothing"). Kapuscinski enters the dark center of the room, with audiences seated on both sides of the Disklavier placed there, with a film screen suspended above the instrument. As he begins to play his composition, a dialogue evolves in real time: every strike of the piano keys manipulates the digital video image on the screen by intervening in the order and speed of the dancer's movements. Rizzi was filmed in the nude, his movements based on Haffner's choreography. The interface here is the piano: Kapuscinski steers the video samples of the dance, as well as additional audio samples, via a computer that "reads" the key strokes and even senses the particular articulations in the playing. The composition is newly interpreted in each live performance, and the piano interface is also open to audience exploration, as Kapuscinski suggests after his 45-minute performance. He invites the audience to "play the dancer." On opening night very few people actually tried it, being aware that Kapuscinski was working from a structured score that allowed him to develop the digital dance in a deliberate, dramatic manner. Those of us who did try the piano realized that the interactivity was based on relatively simple MIDI trigger (on/off) signals that allow the pianist to play the video image track backward and forward, freeze-frame the motion or advance it, literally, frame by frame, thus controlling the image of the dancer down to the finest atom. (14) Conceptually, the aesthetic of interactive digital art is necessarily indebted to such "MIDI performances," exploring the potential "pastforward" connections that can be made between instruments and media, as well as directing critical attention to our unstable relationships to sound and image environments that seem to have a life of their own. As in the case of Dumb Type's image machine in Memorandum, the past-forwarding and re-winding of movement images in Yours plays tricks with our memory and optics, and digital artist/programmer Michael Saup went so far as to argue at CROSS FAIR that technologies are not our tools or extensions but autonomous intelligent systems: we ought to be interested in what they do to our psyche. Collaborative work such as Paul Kaiser/Shelley Eshkar/Bill T. Jones's Ghostcatching, which was exhibited as a stand-alone virtual dance, along with Yours, and Scanned already points in this direction of a certain autonomy of the system that runs processed and derived data. Ghostcatching is based on optical motion capture, a system of multiple infrared cameras, computer hardware and software that enable digital 3-D representation of moving bodies. Recording involves the placement of reflectors in strategic positions on the performer's body; cameras surrounding the performer track these sensors in time and space, feeding the information to the computer for consolidation into a single data file. Mocap data subsequently drives the movement of simulated figures on the computer, where they can be mapped onto other anatomies in an animation program. With the animation tool one can draw out and reconfigure the abstracted motions and trajectories of the dance. What we see is the ghost of the dance or, rather, animated motion pictures and drawings.(15) Captured movement phrases thus become the digital building blocks for virtual dance or interactive performances that explore possible, emerging, and always newly manipulable relationships between live and synthetic presences, forms, images, micro-frames, sounds, and their resonances in our imagination. The promise of motion tracking technology and real-time digital signal processing (also now available in motion capturing) is the simultaneous exploration of a fluid environment in which dance can generate sound and animation, sound can affect video images, and captured images inform new movement and a new form of action painting.
Immersed in Invisible Writing / Mapping I will discuss "immersive environments" by mainly addressing them as a new knowledge and training space. Creating interactive virtual reality is a heavily research-oriented endeavor and requires appropriate laboratory conditions. Apart from the CAVE environments that were created at ars electronica and various computer science labs, I am thinking of the groundbreaking Art and Virtual Environment Project conducted at the Banff Center in Canada (1992-94), which featured nine complex VR works including Dancing with the Virtual Dervish by Diane Gromola, Marcus Novak and Yacov Sharir. Osmose, a VR environment created by Char Davies and a team of engineers from the SoftImage company, gained notoriety when it was exhibited at ISEA '95. Virtual reality was hailed as a new medium in the early 90s, but due to the immense computational labor and arrays of equipment involved, the creation of immersive environments understandably has not yet become a common language, even as choreographer Yacov Sharir describes his experience in very provocative terms: What artistic, intellectual, kinesthetic, and emotional issues could be addressed using this technology?… Virtual technologies allow us to manipulate, extend, distort, and deform information as well as experience of the body. They are vehicles that enable us to extend and color work in many ways, some of which may not be possible in the physical realm and/or by traditional means. They offer a way to augment and extend possibilities creatively, experientially, spatially, visually, sonically, and cognitively.(16) The augmented interface he recounts is a convergence of the interactive principles I described above and an immersive experience within a real-time, 3-D graphic and aural environment generated by computers. Sharir refers to it as a "distributed performance environment" which he entered and inhabited with a head-mounted display and dataglove. The three-dimensional world, created by Gromola, projected on an enormous scale the torso (skeletal spine, pelvis, ribs) and inner organs of her body built from X-ray and MRI data. Sharir notes that when moving through the virtual torso he also encountered digitized images of himself dancing, which diffused and multiplied his sense of being inside an other body. Much could be said about the psychological and cognitive effects of navigating immersive environments, and literally moving into someone else's (virtual) body also strikes me as a politically and ethically charged process. While the literal involvement of the performer's or audience-interactor's body and neural system in the image of Gromola's medically scanned body raises many culturally sensitive issues about contemporary (bio)technology, Sharir's experience of the distributed self-as-image refers us back to the motivations for such performance research in the first place. He is clearly stating a desire to provoke questions about human beings, subjectivity, perceptual systems, and how we re-envision and re-configure ourselves through technology. On the computational and formal level of making work, however, he seems largely content to explore "possibilities" for a new spatiotemporal aesthetics, asking where one can locate his performance. In the real space or the virtual world? In the pursuit of knowledge about the body and its movement possibilities I detect a cybernetic impulse that surely drives the software development. The interactive, multisensory, and reflexive body developed in our artistic training is merged with an informational conception of the body: the "bioapparatus," as Sharir calls it, is tracked, and the data it offers are computed and then extended outward into controlling devices that simulate and regulate the behavior of an organism, a virtual world or any simulation system through feedback. Sharir and other choreographers are fascinated with the transmutational potential of the software to project a body immersed in fluid, non-linear and non-mimetic environments. There is intense spatial ambiguity, since the immersant retains a felt body that is, however, mapped quite differently than the seen bodies that may float through the animate virtual environment. There are many representational dilemmas when interfacing with dataworlds, but our main question here will be where to locate the dance. Scott deLahunta has pointed out that we cannot avoid to pay more attention to software development for artists by artists. "The process of computation is invisible in the simplest sense that the labor of the software programmer or engineer is largely taken up in the 'writing' of an instruction that tells the computer hardware and connected peripherals how to execute an operation."(17) This writing and rewriting is part of the creative process whereby an interactive virtual environment is made. It is interesting, in this respect, that choreographers have been working with code that was by and large written by and for musicians (BigEye, Image/ine, MAX/MSP, VNS). There has been much debate about dance making and interactive systems, especially with regard to "transparency" and the receptivity of an audience to the aspects of the work that might be invisible. What is being considered "invisible" in this context is the "mapping" from input to various forms of output. "Mapping" is not a spatial representation, it is an operation that assigns continuous MIDI data received from a tracking system to control DSP functions such as filters, low frequency oscillators, distortion algorithms, etc. "The interpreted data provides information about the speed, direction, and location of moving objects in the video image, and that information can be used to provide input control data to music-generating software." (18) This is essentially what is referred to as an interactive system. Mapping, therefore, is at the heart of the creative process as regards these systems. However, deLahunta argues, it is the manifestation of mapping that enters the field of perception of the viewer/listener, not the mapping itself. Once completed, the instructions that comprise the mapping itself are relegated to the invisibility of computation. How this invisible mapping works is of interest primarily to those who are engaged in its construction. Although deLahunta is correct in observing this gap between computation and choreography, he may underestimate the curiosity with which some dance companies have approached the relations between writing operations, algorithmic composition and movement creation. The Frankfurt Ballet, Jo Fabian, Jean-Christoph Maillot's Ballets de Monte-Carlo, Pablo Ventura Dance Company, Yacov Sharir, Isabelle Choinière, among others, have developed rehearsal systems influenced by computational thinking. What remains to be seen is whether artistic work with interactive systems allows audiences access to all facets of the systems -- input, mapping, and output. It is true that dance performances using interactive systems tend to give an audience access only to the output, while interactive installations allow access to the input and the output. deLahunta proposes to "include exposure to the mapping itself during performance." This corresponds to my early experiences with motion capture technology. In the initial stages of the capturing process performers generally cannot see the data that are recorded, nor can they experience in real time, while they perform, how the recording data might be mapped onto a character or figure animation. This becomes possible, however, with real-time magnetic and optical capturing systems that wire the hardware/software to video projectors which display the data processing and mapping immediately to the performers and to audiences. A real-time closed circuit relationship to the mapping could afford the performer who practices with these systems a training environment for more complex interactions with them, thus combining input measurement that responds to a higher level of detail and subtlety in performer action with more complex mappings. Confronting proprioceptive and cognitive questions about the immersed body requires intensive practice with interface methods that manipulate the intermingling of self and virtual world, and a more in-depth understanding of the programming language itself. This reference to learning begs the question: where in the dance field do we discuss notions of dance training overlapping with the development of interactive systems? Where in the dance field do we create learning environments in which dancers and musicians could practice in depth with interactive lighting, sound and video projection systems, especially if the latter (midi operated) depend on the fine tuned lighting and calibration of camera sensing systems, as well as on aesthetic lighting design choices (in conjunction with the use of single or multiple video projections) that are an integral part of a multimedia performance work? deLahunta argues that there is a small number of practitioners whose efforts over years are accumulating richness and depth through personal determination. However, their interactive performances are focused on artistic output, not training. In addressing the invisibility of computation in relationship to physical performance and performer training in interactive systems, deLahunta also cautions us about "the long-term outcome of creative activity that is proportionately shifting its center of labor from the physical spaces and composition to the virtual spaces and mapping configurations (e.g. in MAX/MSP environments). As he rightly argues, any dance artist working with interactive systems knows that the amount of work involved in "getting the technology to work" is immense and seems disproportionate to the amount of work done in the studio, perspiring and flexing. A shift away from the physical is by consequence in aesthetic terms a shift away from formal expression to the virtual, the conceptual. deLahunta therefore wonders whether we will see future audiences develop a taste for mapping and for complex yet transparent interactive architectures, coming better prepared and interested in watching or contemplating choreographic choices for dancing in interactive systems. Once dancers begin to inhabit and play with multidimensional mapping environments, the invisibility of computation is displaced by experiential play and the physical consciousness of new behaviors in mixed reality environments where "interactive systems" are infiltrated increasingly by sweating/flexing bodies who spend more time in them sweating, moving, and creating new expressions and stories that may only be possible within such interactive worlds.
Networked Interactivity Networked or distributed environments and real-time processing of performance as multimedia objects for transmission again require new conceptual models that take into account the specific integration of video, communication and network technologies into performance environments and the international co-production and project management of remote and multiple site events with streaming media, web technologies, real-time video production/ editing, and collaborative compositional practice. The "intelligent stage" of the future may not be a theatre but the network itself. Yet in order to transmit movement-images, a dance has to "happen" at some point in real time/real space. Telematic performance thus harbors beautiful paradoxes, as transmittable data have to be produced and processed in synchrony between different locations which may involve different environments.
In a teleperformance created at IDAT, the Australian Company-in-Space staged Escape Velocity as an interactive duet between two dancers, two cameras, and two projectors linked by a direct online connection between the Web Cafe at Arizona State University and a performance space in Melbourne. The live mix effectively merged the two dancers, layering the choreography and the bodies in a spellbinding, transparent symmetry across a vast spatial and temporal gap. It was transparent insofar as we knew that the teleconference had been set up between Arizona and Australia, we could see the audience Down Under, and when Hellen Sky started her dance in front of our eyes, we could see the projected image of her sister dancing the same choreography in Melbourne, and the two camera artists on either end of the performance began to interact with the performers and send their video signals through the line. At various points during the performance we could imagine the dancers being at-one, the sisters becoming a composite dancer floating in a third space created by the overlaid projections which included film footage of several outdoor locations (a forest, a desert). More hauntingly, the apparent symmetry of the dance of course was not precise. Tiny delays in the transmission became part of the choreography and entered into the dialogue between present physical body and technologically mediated body. Ironically, both dancers were simultaneously mediated and transprojected. At the moment when these dispersions become possible, all safe parameters of the body's relationship to space, time and place have shifted. We witnessed a dialogue between spectral dancers mixed onto the pixilated, filtered and manipulated surface of the filmic space created by the projectors, the dance a traveling across time, the body morphing and aging right in front of us.
In'ter, 2000
Two main operational principles can be said to determine the parameters of telematic dance: interactivity and telepresence. Since such work involves real performers who generate digital objects in a real space, it is important to keep the physical dimension of the collaboration in mind, in order to distinguish it from online digital art that gives complete primacy to computer-generated design information or animation over physical space. The sensitive interactive environment in a real studio is a precondition for the creation of movement images transmitted as digital video and sound, since such sensory environments allow the practitioners to experiment with all available interface elements and the aesthetic and psychological aspects of network delay and image degradation. Secondly, computer- and Internet-based telepresence allows performers to be present at a distance and to act upon others in a real remote physical location via live video images. I emphasize the notion of action or "acting" in order to distinguish telematic dance from telecommunications that exchange data or manipulate objects and affect remote environments via telerobotics, for example. Telepresence is a challenge for dance since we have no existing aesthetic or cultural models for real-time dance interaction with a physically remote location, nor will we know much about the role of our potential Internet audiences unless we invite them into the mix. The bridging of spatial distance via telecommunications, as we operate out of our camera-originated environments, allows us to examine the emerging conventions of "networked dance" as new types of montage, layering, filtering, editing, mixing, and transcoding which are to a certain extent derived from our experience of multimedia performance and interactive art. In this sense, networked dance can be considered an extension of our prior experience with the technical production of videodance. Online performers primarily compose with selections of video/audio data, camera angles or sensor-controlled interfaces, and of course with software and the operations embedded in it. The notion of "distributed choreography" is at best metaphorical, since in the virtual laboratory of interconnected distances, performance process shifts continuously between physical object (movement, gesture, voice, etc) to electronic signal which can be modified in real time by passing it through filters and combining it with other data and interfaces. One of our main tasks is to transform our studios into virtual laboratories that enable us to rehearse new performance operations which will inevitably be connected to media and art practices, interface designs, and visual and sonic languages in a number of other areas of culture. As teleconferences, cellular phones and portable media become common practice today, telematic performance is a cultural interface that aims perhaps at specific aesthetic and socially interactive dimensions. It may seem to belong to the representational theatre paradigm, but it is telecommunicational and modeled after the operating systems of computer software applications. This strange paradox deserves careful attention. In the future, we may have to become the (software) designers for our movement interactions. We discover new processes of composition that are cognizant of new coordinates of "placedness" and the spatial ambiguities of immersion. Technique classes include "virtual techniques" in telematically linked studios, movement-with-camera, and movement-with-sensors. Composition/choreography will mean, inevitably, that performance is understood, in the sensitive environments I described, as a multimedia process of design, programming, interactive architecture, capturing, editing, transposition and conversion of movement possibilities, some of which may not even be anticipated by us in the rehearsal. This process will be conducted by teams with artists and engineers from different disciplines; interactive dance in the future might originate not only in dance departments or dance companies, but in collaborative ventures such as the "Future Physical" project launched by shinkansen in London. Distance dancing may become part of the alternatives, as the Internet provides an extended studio for creative production propelling us out into the world, into new kinds of cultural conversations and exchanges. If educational institutions want to participate in this development, certain changes are advisable: 1) new spaces for new dance (integrated studios that combine training and performance with media and technology tools/softwares for experimentation); 2) a complete restructuring of the existing model of dominant ballet/modern dance education, opening out to new performance conventions/co-authoring processes that are team-based and no longer hierarchical; 3) destructuring of existing curricula and the exploration of dynamic/interactive learning and research environments that integrate arts and sciences; 4) a stronger emphasis on interdisciplinary and cross-cultural research and development in telecommunications designs. Finally, the boundaries that separate the professional dance world from club cultures, the music and art worlds, and the Net communities, could be crossed more consistently.
1 Cunningham first worked with LifeForms in 1990 for the creation of "Trackers." His interest in and use of video dates back to 1974, followed by projects with Charles Atlas, Elliot Kaplan, and other filmmakers. 2 Cf. Nicholas Negroponte, The Architecture Machine: Toward a More Human Environment (Cambridge: MIT Press, 1970), p. 101. For an excellent critical overview of the evolution of interactive art within the context of the visual and media arts, see Söke Dinkla, Pioniere interaktiver Kunst (Ostfildern: Cantz Verlag, 1997). 3 The term "nervous environment" is
derived from the term sound artist David Rokeby uses for his interactive
software "Very Nervous System" (VNS), first created in 1982.
VNS uses video cameras, image processors, computers, synthesizers and
a sound system to create a space in which the movements of one's body
create sound and/or music. In his writing Rokeby has pointed out that
VNS is not a "control system" but an interactive system, by
which he means that neither partner in the system (installation and
moving person) is in control. "Interactive" and "reactive"
are not the same thing, according to Rokeby. "The changing states
of the installation are a result of the collaboration of these two elements.
The work only exists in this state of mutual influence. This relationship
is broken when the interactor attempts to take control, and the results
are unsatisfying." Quoted from "Lecture for 'Info Art', Kwangju
Biennale," 1996 [http://www.interlog.com/~drokeby/install.html].
For a very illuminating discussion on lighting and choreographic rehearsal
process, see Senta Driver, " William Forsythe and Jennifer Tipton:
A Conversation about Lighting," Choreography and Dance 5:3 (2000),
41-78. 5 For an extensive documentation and critical discussion of his interactive art project, see Rafael Lozano-Hemmer, ed., Alzado Vectorial/Vectorial Elevantion: Relational Architecture No.4 (Mexico City: Conaculta, 2000). 6 Maja Kuzmanovic and David Tonnesen, "T-Garden," Presentation at "Subtle Technologies" Conference, May 2001, Toronto, Canada. Quoted with permission. For more information on the project, visit <http://f0.am/> and <http://www.subtletechnologies.com/2001/index.html > 7 The "T-Garden" project derives some
of its conceptual terminology from theatre, understood as spatial practice
and social drama (Victor Turner), cultural anthropology, virtual architecture
and current research in hybrid spaces, and the world of 3D game design.
It cannot be used as an example of interactive digital art designed
for the general "user" in clearcut distinction to interactive
dance and interactive music performed by trained professionals. The
sliding between performer and recipient needs to be carefully theorized,
especially as an increasing number of choreographers, composers, visual
artists and programmers collaborate on the design of interactive installations
for audience-play. In the summer of 2002, together with 30 other artists,
I participated in such a design project ("Real-Time and Presence:
Composition of Virtual Environments") at the Trans-Medien Akademie
in Hellerau-Dresden (Germany), and with the exception of the French
musicians Bertrand Merlier and Jean-Marc Duchenne, all other teams created
environments for audience-interaction (http://www.t-m-a.de). The theoretical
debate on interactivity meanwhile is still in its infancy. Cf. Annette
Hünnekens, Der bewegte Betrachter: Theorien der Interaktiven Medienkunst
(Köln: Wienand, 1997); Peter Gendolla, Norbert M. Schmitz, Irmela
Schneider und Peter M. Spangenberg, eds.,Formen interaktiver Medienkunst
(Frankfurt: Suhrkamp, 2001); Roy Ascott, ed. Reframing Consciousness:
Art, Mind, and Technology (Portland: Intellect, 1999); Armando Menicacci
and Emanuele Quinz, eds., La scena digitale: nuovi media per la danza
(Bolzano, Italy: Marsilio, 2001). An emerging bibliography is at <http://www.notam02.no/icma/interactivesystems/dance.html> 9 For a fascination discussion of the use of "dis-focus" in the complex rehearsal operations practiced in William Forsythe's Frankfurt Ballet, see Dana Casperson, " It Starts From Any Point: Bill and the Frankfurt Ballet," Choreography and Dance 5:3 (2000), 27ff. In his pathbreaking new book, Lev Manovic examines digital composition from within the history of the cinematic and postcinematic. Cf. The Language of New Media, Cambridge, MA.: MIT Press, 2001. 10 Cf. Lisa Marie Naugle, "Distributed Choreography," Performing Arts Journal, 71 (2002), 56-62. PAJ 70 and 71 presented special issues on dance and digital media, edited by Johannes Birringer. 11 See also the documentation/theorization of the telepresence experiments conducted by ADaPT (Asssociation of Dance and Performance Telematics): http://www.dance.ohio-state.edu/workshops/ipstheory.html. Cf. Johannes Birringer, Ellen Bromberg, Naomi Jackson, John Mitchell, Lisa Naugle, and Doug Rosenberg, "Connected Dance: Distributed Performance across Time Zones," in Transmigratory Moves/Dance in Global Circulation, Congress On Research in Dance Conference Proceedings, New York University (October 2001), pp. 51-77. See also Richard Loveless and Lizbeth Goodman," Live and Media Performance: The Next Frontier," Performance Research 4:2 (1999), 74-75. 12 The group group Interface (Bahn, Trueman,
Hahn) presented these two performances in a collaborative concert, "Moving
Voices," at the end of an international workshop on dance and interactive
technology at OSU, June 30, 2001. For more information on their work
visit the websites: <http://www.arts.rpi.edu/crb/interface/interface.htm>
13 Coniglio, a musician and software programmer known internationally for his work with Troika Ranch, a New York City based company he directs with choreographer Dawn Stoppiello, wrote two interactive programs, Interactor and Isadora, which map data input to control a variety of media outputs, e.g. sonic, video, lighting and robotic. Troika Ranch conducts regular Live Interactive workshops to give participants the opportunity to explore the use of interactive computer technology in performance (http://www.troikatronix.com/) 14 For a more detailed discussion of the Cross
Fair event, see my "The Intelligent Stage," 15 Cf. Ann Dils, "The Ghost in the Machine," Performing Arts Journal, 70 (2002), 94-104. 16 Yacov Sharir, "Dancing with the Virtual Dervish: Virtual Bodies," in Immersed in Technology: Art and Virtual Environments, ed. Mary Ann Moser with Douglas MacLeod (Cambridge, MA.: MIT Press, 1996), pp. 281-85. Virtual reality works don't travel easily because of the complex hardware/software set-up. Sharir was able to perform in a new version of "Virtual Dervish" recreated by the National Museum of Contemporary Art, Athens,Greece, on October 14, 2002. During the run of the exhibition, visitors experienced the work (one by one) by wearing a head mounted set/display and an electronic glove designed to assist them in the navigational process. 17 This passage, and some of the subsequent observations, are indebted to conversations and an unpublished manuscript, "Invisibility/Corporeality," which Scott de Lahunta presented in my Environments Lab during his residency at the Interactive Performance Series (April 2001), Ohio State University. deLahunta suggested that "writing is arguably not the best descriptive metaphor for software programming - building is preferred as often coding requires increasingly the reuse or reassemblage of previously written code." [Quoted with permission of the author.] See also deLahunta's analysis of Blast Theory's building of a mixed-realty environment in their installation Desert Rain ["Virtual Reality and Performance," PAJ 70 (2002), 105 -14]. 18 For a useful and straightforward breakdown,
see Christopher Dobrian's website "Video motion tracking for musical
input": <http://music.arts.uci.edu/dobrian> See also Todd
Winkler, "Motion-Sensing Music: Artistic and Technical Challenges
in Two Works for Dance," Proceedings of the 1998 International
Computer Music Conference.
Johannes Birringer is a choreographer and artistic director of AlienNation Co., a multimedia ensemble based in Houston (www.aliennationcompany.com). He has created numerous dance-theatre works, digital media installations and site-specific performances in collaboration with artists in Europe, North America, and Latin America. He is the author of several books, including Media and Performance: along the border (1998), and Performance on the Edge: transformations of culture (2000). He completed a new book on dance and interactive technology (forthcoming). After designing the new Dance and Technology Program at Ohio State University, he founded the Interaktionslabor Göttelborn in Germany (www.interaktionslabor.iks-saar.net), and was Principal Research Fellow in Live Art-Digital Research at Nottingham Trent University. In 2006 he was appointed Professor of Drama and Performance at Brunel University. |
