Slipstreamkonza Semiotics: Towards a Telemimetic Sublime in the Data Landscape ABSTRACT Slipstreamkonza is an art/science research project that imagines carbon flux and climate change as a semiotic aesthetic of the sublime. Concerning the sense of place and landscape, this work in progress paper thinks through problems of semiotic installation design and ‘big data.’ 1. INTRODUCTION “The design of such intimate technology is an aesthetic issue as much as an engineering one. We must recognize this if we are to understand and choose what we become as a result of what we have made.” --Myron Krueger, Responsive Environments, 1977 As a visual artist, one may turn a gaze to what cannot be ‘seen’. Here we move into a zone of the sublime. Sublimity refers to that which is below, beyond or immanent relative to an ontological or cognitive threshold. I assume that there is a way of expressing this indeterminate zone, or invisible condition, in both the realms of the physical and cultural landscape and in the interior, “behind the screen” topology of the electronic sublime. Slipstreamkonza addresses aesthetics of digital data expression of land as a breathing ecosystem. The time based data stream of carbon flux is interpreted as rhythmic, virtual expression of sound and image in net based and spatial installation. 2. SITE On and near Konza Prairie, in eastern Kansas, since 1997, diurnal and annual data are collected as "eddy correlation" or "eddy covariant" flux measurements. From two of the sites, a located on the Rannels Ranch next to the Konza field station, wireless net carries the live data online for collection and analysis. Jay Ham, PhD, agronomist and climatologist, conducts research into carbon flux dynamics relative to models of climate change, at Kansas State University. He is the scientific partner for the present project. Konza is the Osage term for “south wind.” Like breath on a mirror, the metaphor of photosynthesis as konza suggests, to this artist at least, the evanescent imprint of an invisible and inaudible (at least on the human scale) dynamic. How to generate a cybernetic process-space that progressively and recursively self reveals, or ‘voices’ itself? Slipstreamkonza exists at a distance from, and following behind, and layering into, the semiotic landscape of konza itself, that is, the dynamic, time-based measurement and interpretation of the phenomenon of carbon respiration. 2. SLIPSTREAM In the tallgrass prairie region, solitude opens up many hours of contemplation of invisible realities – the layers of time, memory, human and geophysical dynamics in a single place, such that time is not, to the contemplative mind, a linear vector; rather, a looping suite of simultaneous spatial layerings. This is a kind of ‘slipstreaming’ -- the artist working at the margins of large phenomena, catching the wind, as it were, from behind a massive, too-big phenomenon much like a sports car can catch the air wave behind a large lorry or truck on the freeway. The sublimity to which gaze and ears and mind obsessively deflect, or defect, is the sense of landscape as huge, transpersonal, and mostly invisible and inaudible dynamic. Things might be going on just out of sight and earshot. You want to catch the waveforms of that dynamic, to surf the stream. There will, too, arise a sense that if you pay close attention to this dynamic (just to be able to stay on the crest of the wave), you will, at least for a moment, have a sense of the deep structure of a place –its phenomenological essence, if you will, or ‘inscape’, as Gerard Manley Hopkins once called it. Immersion in ubiquitous computing and the electronic space now turns the imagination towards intimations of a paralleling digital/real landscape, or ‘slipstreaming inscape,’ which involves some kind of time slippage, or transport-- between an ultimately unrealizable and untouchable Real -- and expressions of that reality in terms of electronic representation, or remediation. The nonlinear and nonfinite phenomenology of digital media, the simultaneous endlessness of reiteration, copy, and reproduction; and the continuous decay, loss, and disappearance of reiteration, copy and reproduction couples with this sense of the invisible dynamic of an unrealizable ‘real’ in the data landscape. In the slippage, or slipstreaming, between the condition of endless iteration in digital media and the huge volume of dynamic data measurement in scientific exploration of landscape, there exists a semiotic, or transitive zone. 3. SLIPPAGE Thereby hangs the tale of a ‘semiotic’ data landscape, if “real” and “sign”, never fixed, make a dynamic, Mobius loop between observation, measurement, and representation. You can never really ‘be’ either ‘in’ in the electronic space, nor participate without observational distance in the physical landscape. In the slippage between these two conceptual points is a place for the sublime in a ‘data landscape. How the data is interpolated into that space becomes an issue of semiotics: the electronic space conceptually is analogous to a space of language, because the data must be interpreted and transformed via arbitrary aesthetic rules through a pipeline of code. 4. KONZA In 2001, Jay Ham invited me to use the large volume of data associated with his ongoing global climate change studies on the tallgrass prairie. Jay participates as a partner in a global longitudinal study of carbon levels in the atmosphere relative to global climate change. Photosynthesis, during the daylight hours, takes carbon from the atmosphere, and at night, the prairie respires carbon from the surface into the atmosphere. Carbon respiration data is delivered via remote LAN into servers at the research site, and from there, may be transmitted and interpreted at other remote sites, including installation and exhibition locations elsewhere in North America, Europe or wherever sufficient server capacity exists. The research question that drives this climatologic research also stimulates my search for tools and methods to create a work of art that refers to and embodies an aesthetic of the sublime in the data landscape. This question has to do with a mysterious shortfall, or absence, in the mathematical models we currently use to describe and predict large-scale climate change. Global warming, implicates the increasing atmospheric level of carbon as a primary agent. Nonetheless, the total worldwide carbon budget, which takes into account all known petrochemical usage on an annual basis, shows that terrestrial systems must be absorbing more carbon than we realize. Carbon flux patterns of selected microsystems worldwide, like the tallgrass prairie, may reveal conditions under which more carbon is been absorbed than is being released. From my point of view as a conceptual artist and designer, this discrepancy gives rise to an aesthetic of the sublime, e. g. the representation of something in excess, or outside of a system that cannot be accounted for in that system. With respect to semiotics of representation, the sublime refers to that which is below, beyond or immanent relative to a cognitive threshold. At Jay’s research installations on the prairie, the movement of CO2 between the prairie and the atmosphere is measured using a method called eddy covariance. This technique requires two instruments: a sonic anemometer and an open-path CO2 analyzer that operate continuously throughout the year. The sonic anemometer measures the velocity of air in all three Cartesian coordinates by measuring the speed of sound between paired transceivers. Data are collected very rapidly (ten times per second). These data are coupled with results from the gas analyzer (also collecting data ten times per second) that show fluctuations in CO2, water vapor concentration and fluctuations in air temperature, to calculate the number of CO2 molecules moving vertically above the surface (towards the surface or away from the surface). Slippage, or slipstreaming, between the present continuous volume of six million data per day coming from the scientific installations on site, and the remote, or “telemimetic” transport of those data into a sonic and figurative language space is a crucial design problem for Slipstreamkonza as installation art. It is because you are not ‘there’ that, paradoxically, you can be telematically present to the data or it to us via a semiotic looping in sonic and visual forms. To date, Slipstreamkonza has only involved the creation of digital prints and video that include visual and sound abstractions generated from interpolations of code based on saved samples of data, together with photographic documentation of the technological installations on the prairie. Recent exhibitions of the Slipstreamkonza project have occurred at San Francisco, St. Louis and Berkeley in 2003 and 2004; and online in the magazine SCALE published by the University of California San Diego (2004). Honored and surprised by the intensity of the positive international response to this work in progress, I am driven towards a more thoughtful problematization of the design of the proof of concept for Slipstreamkonza, in anticipation of its further elaboration as installation art. The design questions that arise in the context beg for significant feedback from the computer semiotic community of scholars and artists at COSIGN. 5. SONIC GAIA As a place of continuous ruin and simultaneous regeneration, the networked space of electronic communications is re-presenting, itself. A semiotic model may offer us the net as a subjective topology, a synaptic process-space. Semiotically, it ‘voices’ itself. A model of the net as a live voice finds some echo in analogy to the Gaia hypothesis on the nature of the physical landscape. As life, Gaia persistently self-represents, or emits information about herself [1]. This is an old idea in new dress. “Day by day pours forth speech,” declares the Psalmist. The problem is that such a voice doesn’t necessarily make sense, becoming “music of the spheres” or of the land. The stochastic or noise aspect of the sonic expression is important because it emphasizes the inaccessibility of meaning, of what is ‘really’ happening on the prairie. I have begun to design data driven sonic topologies that loop reflexively into audio and video installation, exclusive of overt interactivity. Slipstreamkonza resists a participatory or interactive art installation because its identity, or ground of being, is in the prairie landscape itself rather than in the installation space. And yet, this prairie landscape, in my view, is unknowable, despite its limited self-expression to the five senses and to the statistical labyrinths of data collection. This anterior reference, to something beyond or behind or below the level of perception, that is motivating a mysterious expression in audio and video conditions, extends an obsession in my lifelong artwork, with ephemerality, absence and memory from photography and painting. 6. SUBLIME There aren’t any claims here for a pure 'nature'. Once we are in the realm of electronic emulation of data we can no longer claim to be creating a situation to which nature 'directly' responds. This is true even a work of art that appears to model direct encounter with nature, such as The Lightning Field (1977), by the American sculptor Walter De Maria in remote southwestern New Mexico. I see the installation of Slipstreamkonza in a human scaled, installation space as analogous to a situation of interpretation, or secondary manipulation, of The Lightning Fields. To pursue this analogy: imagine you might put a web cam on at The Lightning Fields, and wait for the electrical storm footage from when the lightening fields actually work (extremely rarely). Then live video could stream into a remote site for installation. This is a three-step interval, lightening storm, web cam recording, streaming packets to installation site. Slipstreamkonza could be the kind of installation like Lightening Fields wherein very little occurs except in these incredibly rare intervals, maybe the action occurs for a 1/10 sec on Sunday morning at 3 am. As soon as one decides to design for the human condition, however, rather than for the vision machine, one has to address visual and auditory style, timing, delivery -- in a sense, cinema and architecture, and ultimately, the semiotics of data as art. Thus we arrive at a human design problem. How can one reveal the condition of artificiality as an aesthetic premise in itself, in the installation? How to bring big data into a human scale so that it is visual, sonic and in a scale that is interesting to the primate level of reality? I think too that if you take the data too seriously, as if you can somehow ‘represent’ the reality of the prairie in the installation, that you are at risk of simplifying the content of the prairie into pretty packets of sound and image. Pretty soon it’s just kitsch new media. So what? So the design program must concern itself with how to critically interpolate, rather than represent; to remediate, rather than to show, a remote physical phenomenon, that of the carbon flux on the tallgrass prairie. I like a kind of weird collaboration with the Gaia hypothesis rather than in an attempt to show or demonstrate the supposed truth of such an hypothesis. I have chosen to look at the data conceptually as a flawed or entropic formal array, for reasons that honor the artificiality of the installation situation and the incontrovertible aspect of the sublime, i. e. that it cannot be accessed. 7. GLITCH The data has a number of flaws in it, instances of ‘flat affect,’ such as values like 9999999 or 0000000. These flaws can be appropriated arbitrarily as a part of the aesthetic of the sublime, because the flaws are integral to the data landscape, and because the consciousness of the artist and the tools of the artist are in a condition of indiscriminate immersion in the data. It is indiscriminate because it is impossible to ascertain what preconditions of meaning may be assessed in a purely aesthetic semiotics of the data landscape. Remember that we deliberately discard any attempt at scientific visualization (or sonification as the case may be). Our only recourse is to remember that like the ecosystem of the prairie from which it derives, the data landscape itself may be described as continually subject to entropy, following the second law of thermodynamics. Life itself may be thought of arising, like a phoenix from ashes, as an articulate resistance to entropy. A continuous dialectic between entropy and the architectural self-structuring process of life means that homeostasis is predicated on breakdown, or ruin. Data stream is not always continuous. Scientific instrumentation for measurement and transmission of physical data may fail. Anomalies of landscape data are not always explicable based on known models. Humans struggle with the limitations of their bodies, including, fatigue, inattention, illness and mortality. A telemimetic aesthetic of the sense of place in the data landscape accommodates breakdown of the ‘language’ of information streams. It is mimetic insofar as it represents itself relative to a precessive content (landscape data) and does so at a distance from itself. Telematic art is asynchronous communication: between the in and out of data feed and interpretation, there is an alteration in time and space. At the point of rupture, in the place between, is to be found the ‘sense of place’ in virtual topologies. Thus the Platonic view of an anterior, or precedent Form, which comes into consciousness only through a physical expression, is undermined by the feedback loop into digital media installation. Even though it is tempting to assign transcendent values to a digital media ‘expression’ of data, I have come to resist such thinking. I would rather play with the traces of data within artificial structures of semiotic meaning, such as paradox. I am pessimistic regarding the possibility of creating an aesthetic expression of the data that responds on any level to an anterior reality that the data is supposed to be reflecting. Layers and layers of time and meaning conspire both to create a vivid and sonically exciting array and at the same time, relentlessly resist assignation of cognitive significance. I think it is very important to include the data glitches, the mistakes and miscues of corrupted data as much as the supposedly ‘accurate’ data, without resort to any kind of precursive truth or reference to the Real, or to an ultimate Platonic form. At the same time, as I am free of any obligation to representation as in scientific visualization, I have been happy to throw out what appears to be corrupted or excessively noisy patterns. I have done this with Java driven images and am working on the sound now. If Gaia has speech, it is an inflected, provisional, medium-specific speech. No claims for representation of reality hold up in the end: even the data is a manufactured event or infinite series of events, and when it is fed back into a feedback loop of audio and video expression, it continues to represent only itself. The installation as autodidactic and autopoetic -- in this perhaps I wonder if it analogizes the carbon data landscape itself. 7. TROPE A first attempt at visualizing the data in an arbitrary abstraction was undertaken in spring 2003 using Java scripting to convert arrays of data. In this instance, only RGB values were assigned to median arrays of values. Extremely low and high values were dropped because they would not yield visual content. This choice was arbitrary and driven by artistic taste without regard for scientific visualization. The results were cast into Final Cut Pro as a video of animated stills layered with flashes of the digital print suite. This experience suggested to me a middle path between two kinds of data landscape constructs: one in which the data is assigned abstract numerical values (as in Java) and one in which live photographically and videographically acquired imagery and live sound are acquired on site at the remote data collection installations. In the video, sound tracks and digital prints of Slipstreamkonza so far, combination of the two tropes. A trope is, in linguistics, the figurative use of an expression. The two tropes or modes of data, are, one, a kind of ‘accumulation and assignation’ and two, a kind of ‘illumination and acoustic exploration’. These couple or slipstream past and into each other. At the point of slippage is the deep architecture, or design program, for the installation itself. The conceptual precursors of the project itself remain intact, but the aesthetic expression becomes one of arbitrary and ephemeral character, a work or works of art. The looping between these tropes, offers up a sense of place that is neither entirely of the world of generative code nor of the world of documentary photographic and localized sound capture. This sense of place is at the border between two or more incommensurate conditions. Therefore it becomes a third trope, a paradox. A paradox is a proposition that is or appears to be contradictory but expresses some measure of truth. The tension between these sets up clashes as well as harmonics, and, I hope, a baroque range of effects between extreme darkness and light, between articulation and blur, between noise and tonal wave. 8. DESIGN Three parameters of the aesthetics of this data interpolation design can be addressed in the balance of this paper. One has to do with the scale of the data and its accessible aspects, on immediate (‘live’), 30 minute, 24 hour, and annual data flows in compressed timescales. The second has to do with how we might develop sonic and video conditions from the data source. In the second topic, we are using one 24-hour data set from 2002 as a prototype for thinking through possibilities of sound expression as waveforms corresponding to the breathing of the prairie. The third issue has to do with software. When looking at the data access problematic, I have been thinking in terms of using Macromedia Director to handle the data transmission to web and installation spaces. But, I have also been attracted to the possibility of using Pure Data or MAX/MSP/Jitter for the conversion on the fly from data to sonic and visual dynamics. Two designers have recently collaborated with Jay and me to address issues of data access and interpolation, each from a perspective of his own discipline. Will Bauer (Edmonton, Alberta) is an interactive designer who has most recently designed remote data installations for Raphael Lozano-Hammer. Henry Warwick (San Francisco, California) is a code based musician and performance cinematographer who produced the San Francisco Performance Cinema Symposium in 2003. 9. ACCESS Real-time data flows of 7 or more parameters per site measured at a rate of 10 measurements per second (10 Hz) may be available "live" but we have to be careful not to overwhelm the site's wireless connectivity bandwidth. Additionally, without post-processing of this data, the data make fairly chaotic or turbulent structures. Since it is hard to find patterns because of all the extra "noise" that is filtered out by the post processing, Sipstreamkonza as installation will probably use summaries from the field. Summaries of this real-time data are post-processed by laptops in the field and are stored as files. These thirty-minute data summaries contain the post-processed 10 Hz data plus a number of other, slower changing, parameters - perhaps 60 to 70 parameters in total are calculated/collected every 30 minutes. These thirty-minute summaries do show clear "breathing" patterns over a diurnal cycle. We can download these files as they are produced (i.e. have a new one every 30 minutes) rather than wait for the 24-hour summary files that are currently produced. Historical data are available, going back to 1997 on the site. This makes for the interesting possibility of also visualizing annual data flows on a compressed timescale (sort-of like time-lapse photography) as part of the interactive environment we create. There are interesting annual and seasonal patterns as well as just daily ones that are perceivable in these data sets. We are in the process of capturing sound clips and a web-cam image from at least one of the field sites. Due to the bandwidth limitations of the wireless data telemetry equipment, these clips and images may update slowly (e.g. perhaps also every thirty minutes) but they would still provide an interesting reference point with some interesting compositional possibilities arising from use of the sound clips (which Jay says will be mostly "wind" sounds). It is possible to make the data files available on a university-based server. Given the size of the data files involved (21 Kbytes for each daily aggregate of the 30 minute data files) the relatively small size of the data files also predicts a net based version via a projector in Macromedia Director to access the web links through FTP or streaming protocols or from a remote computer acting as a reflector (to avoid loading the remote site computers with multiple file requests from many people wanting to view the piece). 10. REMEDIATION Data can be manipulated to generate information in a number of ways. One way is through scaling or distributing the data over time. Using a twenty-four hour sample, it is possible to take specific data points over (x) time and use it to describe a waveform. Making a waveform is one thing - making one that is sonically useful is quite another. For example, a column of data might have a nice sinusoidal waveform: but this makes for a very boring sound - pretty much like a flute or making an "oooooooo" sound. Sometimes the sine wave found may not be very strong: And this would simply be a very quiet "oooooo". The other waveform that was immediately found was a sawtooth: This makes a buzzier sound than a sine wave. When combined with a triangle wave, it has a quasi-violin line sound. With a sine wave it makes for more of a "mmmmmmm" kind of sound. These various waveforms can be set to modulate each other in synthesis. The less rhythmic or wave-like data that forms chaotic or stochastic wave forms can be used to control the modulation between these other waveforms, or can be modulated upon. This can be done using MaxMSP, where data would be loaded into a field that is managing the output or control of a given MaxMSP module and its effect on another module. Each of these can be given data from the dataset. If one has two out of phase waveforms modulating each other (using FM synthesis or even simply filtering one another) and the sawtooth sweeping another filter range, an entirely different range of sound can be modulated using the more stochastic waveforms, making the sound modulate ( pitch / volume / filter ) chaotically. The rate of this stochastic change can be further modulated by other data, stochastic or waveform. The waveform can also be slowed down in this way. Take the above wave form which (we'll pretend for the sake of argument) the following values: 5, 9, 8, 2, 1, 9, 8, 2, 9, 4, 6, 3, 5, 9, 8, 1, 6 and if these values were controller values for a MIDI pitch modulation in the key of C, it would be: G, D^, C^, D, C, D^, C^, D, C, D, D^, F, A, E, D^, C^, C, A If we then assign time points to each and have it modulate itself over time, it would come up as: (^ indicating an octave up). GGGGG,D^D^D^D^D^D^D^D^D^,C^C^C^C^C^C^C^C^,C, D^D^D^D^D^D^D^D^D^, C^C^C^C^C^C^C^C^, DD, C, DD, D^D^D^D^D^D^D^D^D^, FFFF, AAAAAA, EEEEE, D^D^D^D^D^D^D^D^D^, C^C^C^C^C^C^C^C^, C, AAAAAA Furthermore, the pitch needn't be so strict and note based: it can be quite fluid as a portmanteau can be assigned - and controlled by another data set. 11. IMMERSION Should all data be used, or should only data that is not corrupt (i.e. isn't pegged at 99999 or stuck at zero)? My feeling is that the anomalous data does not need to be retained, since we are not trying to represent an accurate measurement of carbon flux. Alternatively, the problem becomes, then what level of data filtering is appropriate? Does one keep as much as possible and use as much as possible, or does one pick as choose the data points from the dataset according to an arbitrary sensibility? In the case of the video work I have done so far based on a combination of the java-interpreted code visualizations and photography, that an apparently ‘arbitrary’ or chaotic choice function is preferable. I like to think that this approach allows for the possibility that the carbon flow datascape can influence me formally on unconscious levels of creative work, as when the mind is preoccupied with a work of art while dreaming. A possibility is that human synaptic pathway performs as a layer of dynamic connotation, whether or not I as artist am fully conscious of the same. Or, to put it another way, the landscape represents itself in and as layers of time and human presence. Recursion and flow, between natural data and human/machine is an interpolated topology [2] As we develop stochastic and waveform values from the data, these can be paired with the live sound clips from the site. The resultant coupled content creates the dissonant slippage or slipstreaming that both allows for and counters the development of a musical motif or ‘song’. Even if each day's data set could be a logical "song" length, given the massive scale of the data, we doubt that the data will vary much from day to day, except for "glitches" in the data. These "glitches” could form components of interest and analysis: when something suddenly goes "quiet" or pins itself at 99999 or (-99999) or gives some vastly anomalous reading, the results could be used for their own meaning/non-meaning slippage. CONCLUSION Slipstreamkonza desires the extremes of boundary conditions and interlocking means, all in service towards a description of something indescribable, a remote perfection that appears chaotic at our level of resolution but might make formal sense from the perspective of a viewer on the furthest edge of the Milky Way. Slipstreaming, we are immersed in the inexplicable: to all observers, why do the carbon flow data show an incommensurate condition? Where does the atmospheric carbon go but into the slipstreamed topology of pixels and waveforms—the topology of semiotic imagination. ©Christina McPhee 2004 ACKNOWLEDGEMENTS Portions of this paper first appeared in an earlier essay, “Sense of Place and Sonic Topologies: Towards a Telemimetic Sublime in the Data Landscape,” published in issue 6/7, SCALE, online at http://scale.ucsd.edu, July-August 2004, Brett Stalbaum, article editor, and Michael Podolak, issue editor, University of California-San Diego; and YLEM Journal, vol. 24, no. 6, May 2004, print edition, San Francisco, California. My sincere thanks to Jay Ham, PhD, Will Bauer, Henry Warwick, Brett Stalbaum, Molly McPhee and Terry Hargrave for their continued critical interest in the development of this work in progress. All images are ©Christina McPhee 2003-2004 from Slipstreamkonza and Sonictopos, suites of digital prints from data and photographs photographed by the artist, edited and printed in limited edition C prints on lightjet Fujiflex; and screen shots from the data driven video, Slipjavaone. See www.christinamcphee.net REFERENCES [1] Geri Wittig has looked at the Gaia hypothesis relative to the discourse on landscape data, holism and science, and includes a bibliography on this topic, at <http://www.c5corp.com/research/complexsystem.shtml>. [2] Brett Stalbaum asserts that “data's role in the instantiation of the actual may be a matter of virtual informatic interrelations (or external relations between data sets), forming their own consensual domains that heretofore have not yet been observed as such, but which potentially inflect the operation of actual systems via informational transfer between neighboring systems of interrelations.” (http://www.noemalab.com/sections/ideas/ideas_articles/ stalbaum_landscape_art.html)

Published in: First published at COSIGN-2004, 14 – 16 September 2004, University of Split (Croatia)