Design And Performance Lab
|
Performing Arts, Performing Science: Interactive Environments and Digital Perception
Johannes Birringer (c) 2004
Rather than introducing a wider context of interactive art and various forms of interactive design, I proceed from a few concrete examples.(1) My emphasis will be on the ways in which digital environments affect our senses, and how performance can be understood as transmedial, implying cognitive and affective notions of perception beyond the representational apparatus of theatre or cinema (perspectival vision), and thus beyond the "image" or visuality of movement. The future of performance not only promises new models of transcultural collaboration and exchange, it also implies more complex distributed systems of sentience, communication and interaction.(2)
1. Transmediality The notion of "interactivity" - connecting bodies to digital interfaces - gains its meaning if we read it in the context of design processes which build an extended, transindividual nervous system which is intrinsically changeable and malleable. One could also call it a condition or state, and as such it harbors certain dynamic behaviors. Practitioners in interactive art, dance and music have referred to their networked and computer-rigged laboratories as "intelligent stages," but I consider the theatrical metaphor misleading: responsive and interactive environments can be created in many diverse configurations, bounded (installations, site-specific work, theatre stages) and unbounded (urban space, networked space). What concerns us here is the perceptional process within such "systems." The discourse on the body in live art, dance and representational theatre relied on notions of subjectivity, identity, reference, physicality, nature, technical artifice, etc., which had to be refocused in recent performance and media theory, especially in regard to the properties of digital media and pervasive computing that no longer maintain materially mimetic qualities (still prevailing in analog film and photography). Information theory, concerned with abstract, disembodied and decontextualized transmissions and simulations, construes a posthuman cybernetic world(3), whereas digital performance recovers embodiment and agency in the particular physical, affective, proprioceptive and tactile dimensions of experience with which interaction constitutes its space through computer processing and internal bodily processing. Transmediality, at the same time, points away from the anthropocentric view of the performer-body in real space to the interspatiality of virtual environments and complex human-machine communications that involve programming and various media applications. The technical simulation of vision and other sensory perceptions in turn affects our knowledge and experience of human perception. As a mode of technical mediation within such an infrastructure, interactivity points to a new understanding of environments of relations, a relational aesthetics based on physical interaction as well as a new technological kinesthetic. Designing digital interfaces in dance means organizing a sensory and intelligent space for communicative acts that are inherently dynamic and unpredictable. The space is not "set" for a fixed choreography, but programmed for potential interactions and movements in which partners behave within a network of relays and responses, and in which technologies and media generate perceivable realities. Interaction thus involves the whole environment, and it maps its "world" through the continuous biofeedback it receives via direct sensory stimuli which are also technically mediated (sound, image projection, tactile sensors, wearable computing built into textiles, etc). Ephemerality is no longer a concern. Digital objects can be collected and stored in databases, and performance thrives on re-organizing, using, interfering with the information space, adapting to it in ever open, improvisational ways. In terms of composition, interactive dance is processual, and as a transmedia form it can be embodied in different ways and shapes. But as an aesthetic event it implies a sensory and sense-making body-brain activity. Interactive movement and play involve behaviors that I describe in human and social terms rather than machinic and functional ones, even though the programming of an environment implies assigning functions and values of recording/recorded data, as the software uses the input from the tools of connection and manipulates, mixes, and remixes the input, which in the case of dance includes bodily movements, gestures, sensations. We use a mixed language if we speak of "sampling sensations" or "framing affects," and the sensorial and technical levels of interactivity often get confused. But it is apparent that aesthetic emphasis has shifted from the object of representation (choreography) to the emergent situation, the performative interplay, and the materialization of technology, itself. We see dance and "digital objects" in the environment (projections, 3D virtual worlds, artifacts that serve as interface, etc), but real time computing in interactive installations generally shifts the "process" to the physical involvement of the user, and thus alters conventional distinctions between "artwork" and "observer." This is not the case in interactive performances staged for a spectating audience. Interactivity, in general, offers and assigns roles to the users when interacting becomes an essential component in the condition of the situation, its actualization and reception.
2. back to return Willi Dorner Company's back to return, a continuous three-hour exhibition of dance, film and sound installations, was performed in and around the entire Lakeside Arts Center during the recent 2004 NOTT Dance festival (Nottingham). The audience was handed a schedule and location plan, invited to roam freely and explore the various aspects of the installation, which comprised nine solos, a duo, a trio as well as a number of audio and video pieces which, in some cases, functioned like a solo (in one a voice spoke a monologue in three different tempi, in another the recorded footsteps of a dance were audible). Although this environmental installation did not use any direct real-time interactive processing, the conceptual treatment of the "dance material" (both live and recorded) as database or as samples indicated a digital approach to a locational and temporal structuring and dispersion of information which required each visitor to form her own synaesthetic picture of the whole. The relationship between microelements and macrostructure is transmedial, and since individual components of the installation are performed repeatedly, and thus can be viewed more than once, from different angles and in an accumulative manner, the process of reception is both nonlinear and gradual, without a center of gravity and perspective. The perspective distortion of the multi-layered (frame)work is deliberate.
[Illustration]
Solo 3 was performed in the Green Room for one visitor at a time; I had to sign up and wait my turn, and was then asked to close my eyes upon entering the room where the dancer was waiting for me. I was led to a chair and told to keep my eyes closed, and as the dance unfolded, I found myself in the challenging situation to accept this sense deprivation (the conventional visual perception of movement) and rely on my other senses and my imagination. It was an extraordinary experience, however simple the parameter for this interaction, but I remember this solo to be the most interactive piece of the entire installation. I heard the dancer move, in close proximity, sometimes further away, I followed her breath and energy expenditure, I sensed movement and began to form "mental pictures" of what this movement might be like based on my calculation of its speed, energy, strength and subtlety. My sense apparatus began to perform complex operations, activating my body while my fantasy drifted into other areas of association and interpretation. I could not see this dance but I heard soft and hard motion, my temptation to look decreased as I imagine myself in a darkroom, necessary to develop film, in this case I developed a space in which my attention shifted to breath and circumference, I sensed movement all around me and I became enveloped. The intimacy was exhilarating and erotic; instead of clear pictures I formed a tactile apprehension of the world of felt movement in which I was submerged. Such an interactive performance animates the audience to reciprocate with a kind of "movement processing": it is not a virtual dance but the visitor's bodily perception is itself virtualized.
3. World Modeling I emphasize the phenomenological weight of this sensory processing to distinguish it from fashionable but misleading notions that digital performance today is overwhelmingly predicated on "virtuality" and on the "body" being digitized, its movements and enunciations captured and coded for algorithmic processing (Carver and Beardon 2004: 167-171). In my example, there is a shared real-time environment in which "images" (perceptions) are extravisual, generative, synaesthetic -- contingent on sensorimotoric, haptic and auditory apprehensions. Dorner's playing with the "blind" viewer only underlines that in most cases of contemporary digital dance the interactive relationship involves the dancer's or participant's intervention into the world of projected media (video, 3D virtual reality, animation) which is apprehended through the whole body. If back to return were a model of immersive performance, it indicates how dance can activate bodily modalities apart from sight.(4) The process (proprioception, tactility, affectivity) through which human perception constructs images does not depend on the visual mode. My other examples of interactive dance point to a similar expansion of the visual image and our optical experience. Dancing with virtual worlds opens up a perspectival flexibility which transforms the photographic and cinematic projection of the real. It challenges choreographers to think of the relations between dance and projected image in different ways, especially regarding the familiar and redundant back-projections of video on stage. Some choreographers (Pablo Ventura, Klaus Obermaier, Vim Vandekeybus, Frédéric Flamand, etc.) have experimented with projections directly onto the bodies of the dancers or used video as a lighting source. At the 2004 Monaco Dance Forum , the Multimedia Workshops and TechLab ("Extending Perception") were specifically dedicated to the haptic effects of video-sound projection and to motion-sensing technologies and their kinesthetic impact. The processing speed of digital technologies has reached a point at which motion sensing systems can now transform data in real time. Programmers and digital architects working with dancers have started to use "movement information" for the construction of synthetic image worlds and modelization which enable the dancer to act upon a "living" data environment that is as fluidly evolving, elastic, and changing as physical movement, but the projected images are no longer representational. The virtual world, which Hellen Sky and John McCormick (Company in Space) in their recent collaboration with Ruth Gibson and Bruno Martelli (Igloo) called Sentient Space, affords a different mental experience of the image-space.(5) Their experiments with telepresence and motion capture took a complex conceptual turn towards real-time dynamic systems in which the image-environment is generated by the crossing of several networked morphogenetic processes. Sky and McCormick explored telematic performance based on the idea of visual world-modeling. Movement data, captured in real time from Sky and Gibson (in exoskeletal motion sensing suits), were transmitted and processed to create "soft bodies," i.e. the data were not mapped onto animated figures but stretched, folded and manipulated to create topological models -- images that act like the congealment of the space in-between the two dancers performing in remote locations.
[Illustration]:
Dancing in responsive environments that use telepresence and 3D VR interactivity means participating in virtual or distributed space: to enter into image-space and kinesthetically experience the body somewhere else (and as something else) requires different kinds of intuition. The term "sensing" gains a dimension beyond the habitual physical and organic understanding of bodily anatomy, musculature, cellular consciousness, and proprioceptive spatial awareness of moving-within-the-kinesphere. The convergence of interface design and movement perception here extends (Laban-derived) structural explorations of the body's kinespheric repertoire for movement with regard to space, shape, effort, dynamics, rhythm, and expressive qualities. In the digital laboratory, these qualities are intuited in or between data-space, projected space (video/animation) or other virtual topologies (VR, nongeometric virtual space). The performer learns to "see" with the whole body, projecting herself into the entire space that surrounds her.
Finally, the transformation of site into multi-sitedness in streaming media performance can be illuminated through the phenomenal experience of synaesthetic perception. "Dancing across" here means that digital technologies can connect and fuse different spheres of activity, but delays and interferences in internet transmission also play tricks on our perception. This encourages us to play with synchronicity and recursion, and to invent dramaturgies that use lags, loops, discrepancies and peripheral perceptions aesthetically. At the first interactive laboratory in the former coal mine Göttelborn (2003), I created a 6-minute film of a solo by Koala Yip on the roof of the immense sinking pond rotunda. Entitled Oracle, the edited film was not shown at the concluding exhibition. Rather, I asked Koala to question the "oracle" again in her native language. Her voice (via wireless microphone) then animated sections of the film in real time, processed in a Max/Msp software patch created by Marlon Barrios Solano. The algorithmic process translates Koala's voice into new shapes of light -- distorted, pulsating, curved, exploding and shrinking picture-animations that bear little relation to the original dancefilm, even though we still recognize Koala's movement and her mirror reflection in the watery vortex of the sinking pond. The film became a virtual environment that lit the imaginary world, echoes of Koala's vocal intonations pulling, stretching, and scratching the digital emulsion of the filmstrip.(http://interaktionslabor.de) In the telematic dancework Self Talking (2005), collaboratively created with Kelly Gottesman and his team, the live stream connects his movement (in Detroit) with my voice and sound (generated in Nottingham) across the distance. I am the voice in his head which he hears as he dances, a voice remembering and anticipating the movements I feel as I receive them in my incoming stream. Telepresence creates an interactive environment allowing the real-time synthesis of various media acting upon each other in a shared virtual reality (the internet) which needs to be spatialized through projection. I see a projected dance, I move with it, and there are other systems of reference in play: memory, physical consciousness and orientation, sensing of color, light, shapes and sound, one sense modulating another as I speak through the movement of Kelly's body and begin to play the piano keyboard on my end. In our studios the emphasis is on the dancer's actions, how he or she incorporates the projected light and sound of the stream into an extended sense of the world, an extended body. One aesthetic challenge in telepresence is the conscious incorporation of the camera interface into the performance, with dancer and cameraperson working very closely together in a restricted area under subtly diffused light. The choreographic relationship to the streaming environment, and to the frame compositions of the virtual images created by the cameras, suggests that the dancer insert herself into a moving architecture and move "through" the filmspaces.(6) It is perhaps also appropriate to say that she "frames" herself. The other challenge is to orient yourself in a media-rich space by focusing on the expressive quality of the proprioceptive system, on the potential variations and associations you make with sounds, shapes, and rhythms, on implicit form which cannot really be measured and quantified by computers and tracking devices. Space fills with voices and resonances, performance emotions, changing and transforming from one medium to an other, spilling fantasies.
[illustration]
As an emergent system, the telepresence event cannot be controlled. It is an "adaptive system": we re-adapt existing media and bodily techniques to the new interface, playing with the real-time synthesis of various media forms in an evolving space of emotions in which we perceive our own vitality. We are separate but appear together in a shared world where we need to intuit the other presences. The emphasis is on our actions, not on avatars who journey in pure data environments like the synthetic worlds of games. Digital perception however implies that the dancer inserts herself into the moving architecture. I would venture to suggest that the dancer in this case has to "wear" the digital environment, the image environment, the sensual presences she feels. Thus the perceptional experience is closer to the kind of sensation we have when we wear certain clothes that heighten our awareness of our surroundings, the effect we have on our surroundings or the eyes we feel on our own bodies. Digital environments, in this way, can be worn or experienced as a second skin. For the dancer herself this means that she develops a (virtual) proprioceptive and tactile vision that goes beyond vision, since she may not recognize herself in the stream nor all the sonic and visual information that continuously flows through it. To a large extent, this dance is an imaginary activity, a fantasy of self and other, a perceptional "modeling" in the way in which wear ourselves for the environment which clothes or undresses us.
[illustration]
In the future, we may develop new digital "scripts," devise interplay methods to incorporate audiences as active participants who want to join the interface, enter the story. I imagine that such performance-scripts involve interactive storytelling, distributed narratives and game-like structures for the audience's direct collaboration in the fiction or in spatialized play.(7) In a recent online performance with partners in Detroit and Tempe, we tested such a "game engine." Saira Virous explores a collaborative process which involving both programming (Max/Msp and Isadora patches creating a game-landscape with different levels for movements and behaviors in a surreal game) and open participatory performance by multiple players. Players enter unprepared into a game-scenario in which the networked environment acts as a fantasy space (any other social space of interaction could also serve as a model). Strange tasks are engaged by the players who interact with each other and the streams which are filtered through the patches. While my earlier work with the ADaPT collective, similar to that of Company in Space and other telematic choreographers (Lisa Naugle, Sita Popat, Laura Knott, Wayne McGregor, Paul Sermon, Susan Kozel, etc), was focused on camera-generated telepresence, I now prefer a more sensuous engagement with the digital plasticity and modular interactional possibilities of the medium itself, treating tactile and spatial movement through the images in the manner in which textile and fashion designers work with transformable fabrics on the body. (8) I envision the live web-streams to be playful, erotic and fantastical digital processes we sense on the skin and through the pores. In such dancing we absorb the digital environment: the virtual and the fantastic are not strange to us when we incorporate them as physical sensations. Movement-images, even when they are abstracted and filtered through visual and sonic processing, continuously merge with our consciousness. They affect us to create a reaction, and even if the complexity and density of the continual flow overwhelms us, blinds us, we also discover through such interactive art how our neurobiological bodies adopt to information-rich environments and organize their creativity beyond our perceptual habits.
Notes 1 For an extended discussion, see "Dance and Interactivity" (Birringer 2004) and "Dancing with Technologies" and "Impossible Anatomies" (Birringer 1998: 27-144). For a laboratory report on the workshop on interactive tools and systems, see http://www.dance.ohio-state.edu/dance_and_technology/ttreport.html. A bibliography on interactive dance is at: http://www.notam02.no/icma/interactivesystems/dance.html 2 My reflections on a new digital phenomenology are inspired by Mark B.N. Hansen's theorization of the "digital image" (extending Henri Bergson's theory of perception) and his approach to interactive information environments which become a bodily process of filtering and composing images (Hansen 2004: 93-124). For an examination of transcultural media performance, see Birringer 2004b. 3 Cf. Hayles 1999 and Whitelaw 2004. 4 Cf. Grau (2003: 343-44). For important insights into affective perception, see Massumi, (2002.); for the cybernetic aspects of interactive art, see Ascott (2003). A pragmatic study of rehearsal methods for digital dance is available in Dinkla and Leeker (2003), and for the history and theory of interactive art see Dinkla 1997, Hünnekens 1997, and Manovic 2001. 5 Sentient Space was created as an experiment at the Digilounge Workshop in Chelmsford (UK) in February 2004. The workshop was convened by Scott deLahunta and produced by Essexdance and the British Arts Council. I am grateful to deLahunta for inviting me to the workshop, and to the artists for sharing their findings. Another experiment at the workshop was conducted by Carol Brown and her dancers. In their collaboration, Brown and digital architect Mette Ramsgard Thomsen devised an interactive environment (SPAWN) to explore how kinetic information informs the behavior of virtual architectures. The dancers created duets or trios with the projected light of the architectures computed by the software Thomsen had written. Their bodily contours were tracked by the camera and processed by the computer program, and they affected the morphological shapes of the image-light and image-color as their movement dynamics became embodied architecture. They literally enacted the virtualization of/in their bodies. 6 The telematic performances described here are produced by the Association of Dance and Performance Telematics (ADaPT). Partner sites since 2001 include Columbus (Ohio), Tempe (Arizona), Salt Lake City (Utah), Madison (Wisconsin), Detroit (Michigan), Irvine (California), Brasilia and São Paulo (Brazil), Nottingham (UK) and Tokyo (Japan). The ADaPT online performances are documented and archived at: http://www.dance.ohio-state.edu/dance_and_technology/ips3.html, : http://dance.asu.edu/adapt/, and http://www.dance.wayne.edu/DanceTechnology.html. 7 New theories on gaming and interaction design (Copier and Raessens 2003; Newman 2004) address the overlap between the media, as do festivals on computer games such as "Screenplay" (held every February at Nottingham's Broadway Cinema). Apart from its various games contests and exhibitions, the festival features seminars, webcasts and performances exploring interactive digital technology and its impact on culture today. See http://www.broadway.org.uk. 8 For example Alexander McQueen, Hussein Chalayan, Lucy Orta, or Vexed Generation (Quinn 2002: 117-39). In this context, it is also interesting to look at the work of textile/performance artist Regina Frank who has explored the intersections between textile/text, fabric and internet communications in her durational exhibitions in which she stitches email messages or text files into the garments she wears. See www.regina-frank.de/. The relationship between costume design and wearable computing is also explored in the choreography of Yacov Sharir and other practitioners in the dance and technology field, and by design artists such as Jane Harris.
Birringer, J. (1998) Media and Performance: along the border, Baltimore: Johns Hopkins University Press. Birringer, J. (2004) 'Dance and Interactivity'. Dance Research Journal 35(2)/36(1): 88-111. Birringer, J. (2004) 'Der transmediale Tanz', in Krassimira Kruschkova, Nele Lipp, eds., Tanz Anders Wo: Tanz intra- und interkulturell. Jahrbuch der Gesellschaft für Tanzforschung, Hamburg: LIT Verlag, 23-56. Brouwer, J., Mulder, A., Charlton, S. , eds. (2003) Information is Alive: Art and Theory on Archiving and Retrieving Data, Rotterdam: V2_ Publishing/NAI Publishers. Brouwer, J., Mulder, A. , Charlton, S., eds. (2003) Making Art of Databases. Rotterdam: V2_ Publishing/NAI Publishers. Carver, G. and Beardon, C. , eds. (2004), New Visions in Performance: The Impact of Digital Technologies, Lisse: Svets & Zeitlinger. Copier, M. , Raessens, J., eds. (2003) Level
Up: Digital Games Research Conference, Utrecht: Diagra/University of
Utrecht. Dinkla D., Leeker, M., eds. (2003) Dance and
Technology/Tanz und Technologie: Moving towards Media Productions -
Auf dem Weg zu medialen Inszenierungen, Berlin: Alexander Verlag. Hayles, K.N. (1999) How We Became Posthuman: Virtual Bodies in Cybernetics, Literature, and Informatics, Chicago: University of Chicago Press. Manovich, L. (2001) The Language of New Media. Cambridge, MA.: MIT Press.
Whitelaw, M. (2004) Metacreation: Art and Artificial Life, Cambridge: MIT Press.
Johannes Birringer is a choreographer and artistic director of AlienNation Co., a multimedia ensemble based in Houston (www.aliennationcompany.com). He has created numerous dance-theatre works, digital media installations and site-specific performances in collaboration with artists in Europe, North America, Latin America, and China. He is the author of several books, including Media and Performance: along the border (1998), Performance on the Edge: transformations of culture (2000), and Dance Technologies: Digital Performance in the 21st Century (forthcoming). After creating the dance and technology program at The Ohio State University, he now directs the Interaktionslabor Göttelborn in Germany (http://interaktionslabor.de) and the Telematics Dap Lab at Brunel Universsity.
|