Artist's Statement Beyond Code

Beyond Code

Writing software has been the foundation of my work for the last five years. During this time, I've come to think more generally about software, discussing it in term of processes and systems rather than computers and programming languages. There’s a long and fascinating history of artists focusing on process and systems, but little acknowledged precedent for artists working with computer software despite the natural confluence. This essay focuses on elucidating some of the properties and potential for working with software within the context of the arts. It presents my current thoughts on process, systems, and interpretation as they can relate to the software medium. It explores one process across different media and elaborates on the process of developing software. I focus first on a collection of works entitled MicroImage to show how the same process is expressed in multiple software interpretations, prints, and animation. The evolution of the work Process 4 is then shown from its initial conception to its refined outcome.


The open Work

In 1962 a young Umberto Eco wrote Opera Aperta (later translated into English as The Open Work). He described the concept of a work of art that is defined as structural relationships between elements that can be modulated to make a series of distinct works. John Cage, Alexander Calder, and Yaacov Agam are examples of artists working in this manner contemporary to Eco's text. While all artworks are interpreted by the individual, he distinguished the interpretation involved in this approach to making art as fundamentally different from the interpretation of a musician playing from a score or a person looking at a painting. An open work presents a field of possibilities where the material form as well as the semantic content is open.
The software I've been writing the past four years extends this idea and explores the themes of instability and plurality. These works are continually in flux, perpetually changing the relationships between elements and never settling into stasis. Each moment in the performance of the work (by the artist or another individual) further explains its process, but the variations are never exhausted. The structure is not imposed or predefined, but through the continual exchange of information, unexpected visual form emerges. Through directly engaging the software and changing the logical environment in which it operates, new behavior is determined and additional channels of interpretation are opened.

This software is the basis for my explorations into print, animation, installation, and responsive works. I work in printed media to expose the density of my processes and to provide a precise image of the state of system at one moment in time. I work in animation to precisely choreograph the development of the process. I create installations to explore the potential of relating my processes to the human body and architectural space. I build interfaces and objects that allow people to directly engage with the software to enable an understanding of the relation between the elements. It's through these different perspectives that a more complete understanding of the process emerges.


Perspectives on a Process

MicroImage is a series of software, prints, and animations developed from 2001-2003. The concept for MicroImage originated with the book Vehicles, Experiments in Synthetic Psychology by the neuroanatomist Valentino Braitenberg. As Braitenberg spent his career studying the nervous systems of insects and mammals, he distilled generalizations about how nervous systems work. He wrote, "I have been dealing for many years with certain structures within animal brains that seemed to be interpretable as pieces of computing machinery because of their simplicity and/or regularity [Braitenberg p.1]." He uses the term synthetic psychology to define the construction of personality/behavior through building up structures of computing machinery. In Vehicles, Braitenberg defines a series of 13 conceptual constructions by gradually building more complex behavior with the addition of more machinery.

The software processes in MicroImage are analogous to Braitenberg’s Vehicles 2 and 3. Simple layers of code combine to create the deceptively complicated behavior of these software machines. Each machine has two simulated sensors to detect the environment and two simulated actuators to propel itself. The relationships between the sensors and actuators determine the specific behavior for each machine. In the thousands of software machines simultaneously running in MicroImage, there are four distinct types, each specified with a color. The architectures of the MicroImage machines are shown in Figure 1. The semicircles at the front (top) represent the sensors and each rectangle at the bottom represents a motor. Each machine has a variable speed, direction, and position, but each shares the same size, turning rate, and maximum speed.

I first approached visualizing Braitenberg’s machines through software. Transferring their behavior from diagrams to computer code required making many decisions about the precise distance between each sensor, the speed of each vehicle, if there should be any perturbations on their simulated surface. After making preliminary studies, I decided to represent each vehicle as a thin graphic line connecting each vehicle's most recent locations (coordinates). Each segment of the line connects a previous location of the vehicle a more recent location to visualize its speed and degrees of curvature. The motion of each vehicle type is controlled by changes to the simulated environment. People may affect the software by positioning the series of three points that define the environment. They experience an awkward sense control as their actions have a direct, but imprecise effect on the software. It is not possible to directly control the software machines, but their behavior can be encouraged and intuitively understood. Small changes in the positions of the stimuli create large global changes in the movements of the vehicles. If there are no stimuli, each vehicle moves in a straight line, but as each new point is added, the vehicles respond according to their simulated wiring. The software offers a near infinite set of images, each dependent on the configuration of points that comprise the environment.

Because the software implementation of MicroImage is continually in motion, it is impossible to look at the precise forms that emerge from the interactions between the vehicles and their environment. In addition, because the resolution of computer monitors are so low (72 pixels per inch), it's difficult to precisely see how the lines are positioned in space. I began making static images from the MicroImage software to have a different perspective on the system. To make these images at a higher resolution than the default screen, I began exporting each line as a series of vector line segments, which allowed them to be printed at resolutions up to 2880 dots per inch. Over time, I modified the software by changing the parameters of the vehicles’ behavior, modifying the line length, and changing the number of vehicles. I challenged some of the assumptions in the original software by changing the way the sensors were related to the motors by testing a series of different nonlinear relations. Over time, five distinct software variations were produced to show a range of ways to interpret Braitenberg’s original text. Through working with the software over a period of months, I exhausted the range of potential images produced through each software variation and made eight prints from each software to show the potential variation in the system. Each print was made while working actively with the software by changing the environment and then exporting specific frames. Looking at these prints displayed on a wall gives a distinctly different view of the software than can be gleaned from working with the interactive version.

The type of images created with live software like MicroImage is limited by the amount of information a computer can process at thirty frames per second. To move beyond this restriction, I began exploring MicroImage through rendered animation. Because of the possibility to increase the time for calculating and drawing each frame, creating the animation allowed the numbers of vehicles displayed to increase from two thousand to five hundred thousand. This significantly changed the way the MicroImage process is perceived by the viewer. This change shifted the focus away from line and motion and place the emphasis on mass and depth. The complexity of the environment was also increased so up to eight stimuli now interacted with the vehicles. Over the course of a month, I produced a twelve minute animation.

Working on the animation led to the idea for a different software interpretation that would simulate the density of the animation. Rather than clearing the screen and re-drawing each frame, I began accumulating each frame on top of the previous. This allows the visual data to continually build over time, thus revealing the history of the vehicles’ motion as well as their current position. The hues used for each type of vehicle were modified so layers of dark and light would continually interact.

These various interpretations of the MicroImage process led me to thinking more about representation. As I claim the core of my work is about creating processes and not images, I need to think carefully about how the process are represented . While I'm principally interested in the construction of the process, they must be made palpable to be experienced. I made some assumptions in the initial software, print, and animated interpretations and I started to consider these assumptions. Principally, I had chosen to work only with the minimal representation of the line to hopefully expose the process without imposing extraneous visual adornment. There are obviously many different and valid ways to represent these processes and I began to think about them. Some examples are the text representation of the algorithms (computer code written in C++), and the data that is produced as the program executes. In addition, there are many different ways of displaying the vehicles. It's possible, for example to make their sensors and bodies more explicit.


Evolving a Process

While many people have a basic understanding of the steps involved in creating a painting or novel, very few have any idea what is involved in creating a work within the software medium. This section discusses the little understood process of writing software. I follow the evolution of one process, entitled Process 4, from initial conception through many stages of development. Rather than focusing on the different ways of interpreting a process, I reveal the modifications and changes within the period of refinement.
Over the past three years, I've been collaborating with Ben Fry in developing a software application called Processing (2), a programming language and environment for artists and designers. On one level, Processing is a software sketchbook, a tool for getting ideas for software out of your head and into code as quickly as possible. The application environment is minimal, containing only a text editor and toolbar to enable the code to be run, stopped, saved, or exported. We developed Processing because we thought we could develop a better better tool for creating our research and could simultaneously develop a better environment for teaching concepts of software and interaction within design and art schools. I have been using it extensively for making sketches for more complex works in software. I enjoy taking time to sit for a few hours and play with code.

Process 4 emerged from a simple idea, to give software elements the behavior of maintaining their own “personal” space. When they overlap with another shape, they move away so they are no longer touching. I consider this a basic pattern of living organisms and was curious to see how it translated into a simplified, dynamic simulation. I began by writing code where I could move the software elements around the screen and when I deposited one on top of another, they both moved away. The next step was to give them constant motion, bringing the screen into perpetual motion. No two elements overlap because they turn away as soon as they touch another. I was not able to properly see their behavior because the information related to their turning was not shown, so I made their direction visible as a yellow dot. I then began drawing a line between the centers of two elements as they are touching and I stopped drawing the circumference of the circles. Every visual element was then removed except the the lines and the software was modified so they accumulate onto the screen. The final stage of Process 4 within Processing used three colors to represent short, medium, and long lines and began with the elements in different configurations to encourage the pattern to develop in different directions. This was the extent to which I could explore the idea in Processing and I moved to the C++ language to get a different view.

Each programming language is like a different physical material, each with its own properties. Processing's material properties make it quick to write programs, but they display at a lower frame rate. The properties of C++ make it more complicated to write, but there is potential for more resolution in time and space (the software can run faster and cover more pixels). A direct port of the Process 4 code from Processing to C++, therefore caused it to be experienced differently. The increased speed and resolution revealed additional properties about the system that were not visible before and opened up a new avenue of exploration. After future refinement, the software was brought to a point where it was exhibited at the DAM GALLERY in Berlin in February 2004. This new version of the process was used as to build the TI installation at the Telic Gallery in Los Angeles in Spring 2004. This piece consisted of 24 disks ranging in size from 12 inches to 36 inches and raised one inch above the floor. Live software visuals were projected onto their surface. The audience was free to roam in and around the images, exploring a hybrid digital/physical space that evoked a tension between the organic and synthetic.

During the entire evolution of Process 4 from its first implementation to the TI installation, the core algorithm remained principally the same, but the method of presenting it slowly evolved over a period of months and through both technical advancements and purely aesthetic decisions. All modifications were made directly through editing the computer code. The following code was written for the display of Process 4 in an early Processing version:

stroke(r, g, b);
line(x, y, others[i].x, others[i].y);

An evolved method of displaying Process 4 in C++ code follows:
if (tempd < 40) {
  glBegin(GL_LINES);
  glColor4f(r/255.0, g/255.0, b/255.0, 0.0);
  glVertex2f(x, y);
  glColor4f(r/255.0, g/255.0, b/255.0, 0.1);
  glVertex2f(others[i]->x, others[i]->y);
  glEnd();
}
By looking at these code fragments, it's difficult to know precisely what the change will be. It's obvious there are differences, but exactly how they will be perceived is not possible to know until the program is run. Figures 10a and 11 show the obvious differences between these two pieces of code. Each visual decision was first made intuitively and qualitatively and was then by necessity made technically and precisely in the computer code before I was able to perceive the alteration. For artists and designers used to working directly and visually with their materials, the strict constraints of programming notation can be a barrier to the flow of ideas. This barrier begins to erode with experience, yet always remains.

Beyond Code
The technologies I work with are continually becoming obsolete. As computers become increasingly faster and operating systems evolve, the software of the present quickly seems dated and possibly may not run directly within a few years time. I've recently been pursuing a way of working with processes and software that exists outside of the current technological framework — outside of the constraints of Java, C++, Mac OS X, Microsoft Windows — to better focus on the concepts behind the work. This technique was inspired by the wall drawings of the artist Sol LeWitt, in which he would specify an image to be drawn on a wall through a few terse and sometimes ambiguous sentences. In LeWitt's work, the program is the English text and is executed/interpreted by a skilled draftsperson. In my former work the program was computer code and was executed by a machine and in my new way of working, the program is English text, is interpreted by a skilled programmer, and then executed by a machine. Putting the emphasis on the English text rather than the computer code creates the opportunity for multiple interpretations of the work and gives the flexibility to not feel constrained by the current state of computer technology. As an example, Process 4 is stated in English as follows:
A rectangular surface filled with varying sizes of Element 1. Draw a line from the centers of two Elements when they are touching. Set the value of the shortest line to black and the longest to white, with varying grays between.

Element 1 = Form 1 + Behavior 1 + Behavior 2 + Behavior 3
Form 1 = Circle
Behavior 1 = Constant linear motion
Behavior 2 = Constrain to surface
Behavior 3 = When touching another, change direction

Unique software interpretations of this text are shown in Figures 14a and 14b. I feel that the essence of the process can only be understood through comparing and contrasting these multiple interpretations and through watching the process unfold in time. Because the English text is the primary representation of the process, as technology changes, the process can simply be interpreted and executed to run on the current machines. The work is open for many interpretations. It is an open system from which multiple unique software objects may emerge.
During the last year, Process 4 has been been joined by Processes 5 – 10. The system of elements has slowly grown with the addition of Element 2 and Behavior 4. Each Process is a rigid starting point that fosters loose interpretations as performances, installations, prints, and projections. The code written for each Element is re-usable, thus making programming similar to collage. This modular system encourages rapid experimentation and serves as a broad foundation for future work.


Footnotes

(1) Braitenberg, Valentino. Vehicles: Experiments in Synthetic Psychology. Cambridge, Mass.: MIT Press, 1984
(2) http://processing.org