Internet Audio Experiments

by Rob Wright

"We have also sound-houses, where we practice and demonstrate all sounds, and their generation…. We have also means to convey sounds in pipes, in strange lines and distances".

- Francis Bacon, in his essay "New Atlantis" (1627)

Context

Thomas Edison was developing a device that could record, then replay at high speed, telegraph signals when he made the first recording of the human voice (Phonograph, 1877). Elisha Gray, who came a close second to Alexander Graham Bell for the title of inventor of the telephone, instead is known as the inventor of the first of the early electronic musical instruments (Musical Telegraph, 1875). Thaddeus Cahill built the precursor to the synthesizer (Telharmonium, 1906), a remarkable machine capable of not only generating but also broadcasting music to public and private residences using telephone technology.

Despite these close beginnings surrounding the technologies of music and those of telephone communication, the World Wide Web remains largely a quiet ‘place’, often more akin to a public library than an integrated multimedia experience. This is not to say that the internet is mute, far from it, but the relatively slow speed of data transfer typically encountered has precluded the use of audio files from many websites which would otherwise greatly benefit from musical support. Most of us are paying by the minute for our time spent online and as a consequence internet-music is often seen as an expensive and expendable luxury, typically one of the first elements cast aside if a Web Designer hopes for many and regular visitors or ‘hits’ to a website.

The Web Designer who is determined to include a musical ingredient is generally forced into tough decisions; a balance of efficiency against quality. One possibility for example might be to make use of the MIDI standard, which involves very small file sizes and as such does not greatly affect the time a website would take to download. Initially this would appear to be a neat solution, as rather than sending digitally encoded sound, we send instructions (rather like a musical score) which are interpreted at the receivers end. However, without knowledge of the exact specification of the end user’s equipment one can never be completely sure how the result will be heard, if at all. Another option might be to utilize short looped segments of compressed digital audio which, although might suit certain applications, will eventually tire even the most tolerant ears. Using longer and more varied segments of digital audio would in turn affect the all important download time. There are other options available such as MODs, which allow for the efficiency of MIDI in conjunction with small personalized audio samples, or an open connection could be established through which a continuous flow of audio data could be fed (streaming - QuickTime, RealAudio).

However for this method to function effectively quite often only a reduced range of frequencies (or narrow bandwidth) can be sent.

This said, as technology continues to progress, and there is certainly a desire within the Music Industry to utilize the internet as a means of music delivery (along with other forms of entertainment), we should expect changes and technological advances to cope with the current obstacles. We have already seen some impressive steps forward, for example the introduction of MP3 (a low-loss audio compression which greatly reduces the size of an audio file whilst maintaining high fidelity). In 1997 MP3.com was founded and offered approximately 3000 compressed songs available for free download. In 1998 it became the leading music site on the Internet and received over three million visitors per month, this of course did not pass unnoticed by the commercial eyes of the Music Industry.

More recently we have witnessed the merging of already giant companies from each end of the Music / Internet Service spectrum, along with the forced demise of ‘Napster’ (an unofficial program which allowed users to search and download music files from other users hard drives). We have seen the introduction of new legislation concerning the copyright of digital media and ‘Webcasting’ of music. We are also beginning to see a wide-scale take up of ‘Broadband Internet’ by home users. ‘Broadband’ promises much improved transmission speeds and almost certainly brings us one step closer to interactive television and music broadcasting via the internet.

In the year 2000, the CA*net3 fiber optic network in Canada became the fastest computer network in the world capable of transmitting all nine symphonies of Beethoven in 0.065 seconds.

Internet Audio Experiments

I find the idea of sharing new musical works, installations, experiments and instruments which have been specially devised for fast internet delivery very interesting, yet one that has not really been widely taken up. Interesting, in that here is a medium without commercial obstacles, one that easily allows material which may appeal only to a local minority, to find a larger global audience.

I suppose that it is partly because of this freedom, because of the inherent challenge of creating new works for a new medium which are efficient yet interesting (greater than the sum of the parts), and partly because if the ‘Web’ is to become a more interesting ‘place’, music technologists and composers should perhaps begin to claim a much larger stake. For all of all of these reasons and for personal development, I have decided to begin tentative research into the current possibilities, and limitations of transmitting a variety of music related applications across computer networks. I also hope that along the way I might gain an insight into the work that has already been completed and the possible future directions to which this exciting field might lead.

Still very much in its early stage, my research into the potential of the internet as a vehicle to deliver musical applications (or perhaps more correctly the suitability of this area to the internet), has started with the completion of a few simple experiments, some of which at the time of writing can be found here. Initially this research is solely concerned with available options when small file sizes are of real importance. It is not my intention to investigate methods in which existing musical works could be delivered via the internet as this is quite straight forward, rather to explore the associated technology and options in a creative way and to invent strategies for composition, installation or interactive instrument construction that suit this new medium.

Work of this nature seems to have been given the label - ‘Soundtoys’, and this seems to encompass many new electronic art forms, interactive environments / games / educational resources, generative musical experiments and pretty much anything else which unites the audio and visual aspects of new technologies through the internet.
To those interested in the technical aspects or to anyone encountering problems attempting to run these files, all of these preliminary experiments were constructed using Macromedia’s FlashMX software, and as such require the latest version of the Flash Player plug-in in order to function correctly (if necessary this can be downloaded here.

Before I go on to describe my work in a little more detail I feel it necessary to stress that I am aware, now more than ever, of the existing and much more advanced research currently being conducted in this field. On a personal level my primary areas of interest lie firmly in the composing of new music and in education, hardly qualified as a programmer I am content to scratch the surface of this new subject, whilst keeping an interested eye on the true innovations which will help to shape the future of a more sonically interesting internet.

The Experiments

WCM: Wind Chime Marimba
File Size - 276k
The aim of this work was to develop an infinite 'composition-engine' small enough in file size to be viable for use on the internet, one that would produce evolving musical material based upon collisions between visually represented objects. Once the file has been fully downloaded, an infinite composition begins, and no further interaction by the user is necessary. Musical output is determined by the following rules :-

i. Notes (samples) are triggered when any Note-Object collides with the larger Trigger-Object, there are twelve Note-Objects representing the pitches from one chromatic octave.

ii. Dynamic stereo position and loudness are derived from the respective X and Y coordinates of each collision. Therefore a collision occurring in the lower-right corner of the screen would result in a pianissimo sample heard in the right speaker only.

iii. When a collision occurs within the Roll-Area, represented by a moving blue square, a roll articulation is activated, rather than a single hit.

iv. The ‘Repeat-Object’ is toggled upon contact with the Trigger-Object and creates the effect of an echo.

v. Objects ‘Accel.’ and ‘Rit.’, alter the velocity of the Trigger-Object upon contact and can dramatically affect the density of the music.

I believe that there are many possible applications and future directions for this work. For example, the rules outlined above could be extended or further timbres could be introduced to create much more varied and less aleatoric results. It would also be a simple step to dispense with the visual content altogether yet still employ the engine as a generator for ever changing background music for a website. However, personally I quite like the idea of allowing people to view the ‘mechanics’ at work, perhaps it gives extra purpose to the music when one can predict what is likely to happen by visually tracking the Trigger-Object?

CLARA: Internet Instrument
File Size - 52k
This project is a virtual instrument based upon the Theremin (an early electronic instrument developed by Leon Theremin in 1917). Named after Clara Rockmore, (Theremin virtuoso), this instrument has again been designed for fast internet delivery and can be played using mouse movements alone. Similar to the original instrument, pitch is controlled by moving the (on-screen) right hand along the x-axis and loudness is controlled by moving the left along the y-axis. Unlike the original instrument which had an approximate four octave compass, this instrument has a reduced three octave range but allows for recording and playback of the hand movements.
There is a small paragraph of text on the page alongside the instrument which can be read whilst experimenting with the instrument. Although in this case the information provided is hardly exhaustive, it at least shows the potential of projects such as this to be employed as hands on educational resources.

VDS: Virtual Drum Skin
File Size - 32k
The aim of this work was to develop a fun environment to explore rhythmic patterns. The resulting work allows for a succession of ‘grains’ to be dropped upon virtual drums of different tunings. The user has direct control over the following parameters which can affect the output (along with several randomizers, that I have termed ‘stability’) :-

i. Initial grain and drum position.
ii. Number of grains and drum size.
iii. Initial upward velocity.
iv. Gravity.
v. Drum skin tension.
vi. Grain release angle.

I view this work as a combination of the two aforementioned experiments as here the user can interact with the environment, exploring the effect upon the musical output.

FPP: Flash Player Piano
File Size - 272k
This work is very similar to WCM (above), however in this experiment a three dimensional representation has been attempted (acknowledgment - adapted 3D code by Pavel Kaluzhny, ‘Flash Math Creativity’ - ISBN 1-903450-50-0). An infinite aleatoric music composition for the internet again employing a computer based composition-engine. Each of the inner sides of the cube represent the different pitches from a six note piano chord. Piano samples (or occasional random bass samples) are triggered by the moving spheres which are contained within the 3D space. A leading sample is played and the chord alternates if a sphere collides with another sphere. Amplitude and pan are derived from the X and Y position of a collision. The arrow buttons in the top right corner rotate the cube, whilst continuing to generate the composition in real time. The introduction of the third dimension (z) would allow for dynamic control of another audible property in addition to pan and volume, however this has not been implemented in this work at this time.

PATTERN CHAIN: Internet Instrument
File Size - 136k
A playable and fun instrument which generates musical patterns based upon the physical movements of a chain of objects which together simulate the properties of either spring or elastic. The user can initiate a pattern by dragging and releasing the on screen ‘beaters’ which are free to perpetually oscillate over the keys of a virtual xylophone. The user has direct control over the following parameters which can affect the output :-

i. Material – spring or elastic.
ii. Lock / unlock the X-axis movement.
iii. Up/down stroke or down only sample triggering.
iv. Tension (friction).
v. Number of beaters (1 to 12).

The instrument has a 3 octave compass and the resulting patterns can be notated in real time.

Future Directions

"The internet is at once a world-wide broadcasting capability, a mechanism for information dissemination, and medium for collaboration and interaction between individuals and their computers without regard for geographical location"
- www.isoc.org/internet/ history (2001)

As previously stated the single common factor that makes these experiments suited to the internet is that they are very small in file size. Although I feel that these experiments as they stand do have potential to be developed further and to be utilized in many different areas, at this stage I have made use of the internet merely as a means of delivering files and instructions. In short for all of the accompanying experiments, the user first downloads all the necessary components (mostly audio samples) and then their computer (sometimes in conjunction with input from the user) is given instructions as to how and when the component parts are to be played. Therefore nonlinear musical works or instruments can be realised from building blocks and a set of rules. I acknowledge that at this point none of these experiments truly explore the wide range of possibilities the internet has to offer, there is actually no limiting reason why these experiments could not be delivered by floppy disk for example. However the next step that I hope to take is to introduce two-way communication into subsequent projects, this may be either to and from a server or possibly from user to user. I feel that there is great potential in this area and by taking this step I would certainly ensure that my further research is truly internet specific and dependent. An example of this might be an internet multi-user environment that could allow many people to simultaneously collaborate on, discuss, listen to and interact with a new work as it is being generated.

I hope that people enjoy these initial experiments, and that this accompanying paper might spark some discussion regarding this extremely interesting area. I will continue to post my research as it grows to the spnm website, and it would be great to see a few more experiments by others appearing there too.

Rob Wright, spnm short listed composer

Rob Wright MSc. BEd.

Rob Wright is a lecturer, composer and music technologist. He currently teaches at the University of Hertfordshire where he oversees undergraduate Music and Sound Design Technology and the delivery of a Postgraduate Certificate, also in Music Technology, which was devised to target the specific needs of the local education community.

In 1997 he gained a Masters degree with distinction studying composition with Javier Alvarez. Specializing in electro acoustic music for instrument and tape, his compositional output has gained artistic awards at international competitions and receives regular performance. Most recently, ‘Arco’ a work for ‘cello and tape was awarded a mention and prize at the 28th Concours International de Musique Electroacoustic et Art Sonore, (Bourges 2001), ‘Wires’ for piano and tape received performance at The Barbican Center as part of the spnm Stockhausen Electronique Programme, (London, Oct 2001 ) and ‘Vendetta Kinda’ Mood’ guitar and tape has been selected by spnm for their 2002 short list.