Bennett C. Brecht

(The rough, unedited submission) copyright 1995 This article appeared co-written and edited by Jim Aiken in a different form in the June '95 issue of Keyboard Magazine under the title "A Lot of Nerve"

Alter-Ego

It's 1995 and our time has finally come. Only twenty-five years after the first Buchla synthesizer was invented, the academic critic's worst fears have become reality. We have reached the day when electronics and computer chips have become a regular part of our musical lives. How has this changed music? Is it true that music has become less human and more mechanical? Is life now lived to the soundtrack of Nintendo? Have we traded our souls for the M1 and the TR-808? We can now safely say with relief, "No!"

... So now that chips are a regular part of the gig and no one's complaining too much, what's next? Will electro-prejudice hinder us from exploring beyond the electronic music horizon? Before you answer, you may just want to take a deep breath, open your mind, and think about it. The next giant is sneaking up on us. It has the power of a freight train and is bigger than anything you've seen or heard so far.

I'm talking about the new developments in soft computing and in particular, neural networks. It seems that the "scientists in white lab coats" have come up with a new form of computing modeled after human brain cell structure. Yes, it's now possible for computer, software or a musical instrument to learn information and retain it by training just like you and I do. In essence, we can teach our machines some very specific tasks by merely exposing them to the right information. Then we can watch them become smarter, faster and more efficient and intuitive about their jobs. Neural networks offer a new view of computing that often simplifies and extends the normal capability of your machine, and they're not just being used in the lab. I've managed to discover some fascinating net projects emerging from the music world. .

HISTORY: ... The best way for me to describe nets is that they're little modules composed of artificial brain cells that are interconnected by threads which carry information. Just as in the human synapse, these threads either strengthen or weaken the communication bond between adjacent cells. They train the module to understand or recognize a task, a gesture or a calculation. ...

... This type of recognition and prediction make neural networks an extremely powerful tool for music. All that is required to create a good net is that you train it well. Teach it a few possibilities and outcomes and it figures all the rest out by itself.

THE PROJECTS:

Enough of the theory though, lets look at some real projects which are great examples of neural programming in music. Keep in mind that the strengths of nets are: 1) interpolation or morphing between two states; 2) recognizing patterns and variations thereof which produce any kind of original interactive output, and; 3) prediction of future events on-the-fly. Since all three of these concepts involve time as a contributing dimension, they are especially suited to many musical applications.

The most impressive functioning project I've seen is the "Global Drummer" developed by David Brubeck (not the jazz musician), Christine Clements and Nate McNamara; all graduate students at U.C. Berkeley's Computer Science Department and the Center for New Music and Technology (CNMAT). This is among the most sophisticated drum machines ever developed. On the control panel, we see choices of ten styles of drumming, ranging from African polyrhythms to hard rock to Cuban conga ensembles to funk. First you start up the machine and select the degree to which you want to mix the styles. This is accomplished by moving various sliders. The nets morph a cross style which is wholly original every measure and outputs the drumming in time, on-the-fly. You can change the mix of these styles as it plays, which is fascinating to hear. What you wind up with is an incredibly flexible multi-national drummer which is almost indistinguishable from a human. I've met very few humans who could play so many styles, so well, together. The most exciting thing about the overall sound is the originality it offers. Yes, it's a consistent cross sound every time, but the fine subtle details of rhythmic and drum head choices are different over time. This takes drum machines into a whole new dimension by offering a constant thinking and evaluation process to every beat of a composition.

This is a great showpiece for nets because it shows the adaptability of output based on unpredictable input commands. The nice thing about this style of programming is that when inputs change drastically, the program doesn't crash or sound unmusical because it may not understand. It merely does the best it can based on how it was trained and accomplishes it's task in a smooth manner. For instance, if we have all the style sliders up to 10 and we suddenly slide eight of them down to 0, the nets merely estimate what the new sound pattern will be, and get from the old state to the new state smoothly by morphing in a highly musical transition. The similarities to human musicianship are too powerful to be overlooked. So look out drummers. Start practicing your chops!

Another project developed out of the computer music programming class at CNMAT by Fred Hillerman is a "listening neuro-drummer." This program is also a synthetic drummer which can listen to any beat stream and estimate both the meter and the tempo of the music that's playing by estimating stress patterns and their relationship to each other. This, again, is gesture recognition. It then takes that information and uses it to program itself to the appropriate drumming rhythm and tempo output. Thus, we have a fully functioning, stand alone drummer capable of handling a live performance outside of the studio. This constant estimating allows for an extremely flexible and musical give-and-take necessary in all musical contexts. It would be fascinating if this project was linked together with the "Global Drummer."

Net developer Michael Mozer at the University of Colorado at Boulder (Computer Science and Cognitive Science Departments, following the work of Peter Todd) has come up with an interesting contribution. He has created a set of neural nets trained on some examples of J.S. Bach and other pre-classical repertoire. As it operates, it acts as a stand alone composer, creating musical and pleasing original melodies based on the rules it has picked up by listening to previous composers. ...

I developed a project, in which a net has been taught to recognize how a conductor moves the baton while conducting. This system is then plugged into a program developed by Guy Garnett (also at Berkeley's CNMAT) which can play any score by reading these recognized gestures. The result is an orchestra of virtual instrumentalists which follow a conductor's gestures precisely. The nets, by recognizing, are able to predict when the next downbeat will happen. It is done visually as in the baseball example above. This allows changes in tempo to be addressed ahead of time in a smooth manner. The result is an ensemble of synthetic performers that act in concert with real human performers or by themselves, perfectly in sync with the conductor's intentions. Again, all that was needed to train this net were a few examples of real conducting and examples of what the output should be. It then infers, later on as it watches the conductor, how to deal with any tempo and any tempo change on-the-fly.

Another project which has existed for years is Dr. Jamshed Bharucha's (of Dartmouth College) "listening" net which is able to identify chords. This research involves training a net to understand how to separate simultaneous notes by intuitively understanding the harmonic structure of sounds in the first place. ...

In line with tonality, there has been another interesting invention by David Wessel, Georg Hajdu and Seth Ober at CNMAT that actually morphs between key areas. By driving in a virtual space with a computer mouse, you are able to hear melodies derived from tone hierarchies that exist in every key anyway. These melodies are smoothly transposed as you travel across and in between different keys. This could be a tool that would allow a composer to experiment with modulation as a primary compositional device. The sound of this experience is truly unique. The transitions between keys are so smooth, it's hard to tell where one key begins and another ends. In effect, you are creating melodies in virtual key areas. This greatly expands one's thinking about typical Western tonal structures and modulation. ... --Bennett Brecht is a composer and researcher / graduate of the University of California at Berkeley at the Center for New Music and Audio Technology (CNMAT) He currently resides in New Orleans where he continues his research and musical endeavors as well as sculpting and painting.