TIMELINE

1822 : French mathematician and physicist Joseph Fourier (1768 - 1830) theorized that any complex sound can be described by a series of simpler sinusoidal components, giving rise to modern methods of analyzing and depicting sound waves. (Source: Wikipedia, Livio Zucca)


Fourier Harmonics

1860's : German physicist Hermann Ludwig Ferdinand von Helmholtz (1821-1894) used physical objects ("Helmholtz resonators") to demonstrate that air cavities have sonic resonance. This further helped scientists (and later on, electronic musicians) to describe sound as a physical and visual phenomenon. (Sources: Wikipedia, Max Planck Institute, physics.kenyon.edu)

 
Helmholtz Resonators

1862 : Rudolph Koenig (1832-1901) developed the manometric flame apparatus, which was used into the first decade of the twentieth century to examine the wave-shapes of sounds. The heart of the apparatus is the manometric flame capsule. Sound enters the capsule via a funnel and a length of rubber hose, and impinges on a rubber membrane placed between the two halves of the capsule. Illuminating gas enters at the bottom of the shaft and burns in a small flame. The oscillations of the membrane modulate the gas supply, and the height of the gas flame varies accordingly. The oscillating gas flame is viewed in the rotating mirror, which supplies the necessary time base to make the waveshape visible. (Sources: physics.kenyon.edu, Wikipedia, Max Planck Institute)

 
Koenig's Manometer and the waveshape images made using the device

1907 : French-born, London-based Eugene Lauste (1857-1935) - who had worked at Edison's lab between 1886 and 1892 - was awarded the first patent for sound-on-film technology, involving the transformation of sound into light waves that are photographically recorded direct onto celluloid. (Sources: Wikipedia)

1916 : The Optophonic Piano was created by the Russian Futurist painter Vladimir Baranoff Rossiné (1888-1944). The Optophonic Piano generated sounds and projected revolving patterns onto a wall or ceiling by directing a bright light through a series revolving painted glass disks (painted by Rossiné), filters, mirrors and lenses. The keyboard controlled the combination of the various filters and disks. The variations in opacity of the painted disk and filters were picked up by a photo-electric cell controlling the pitch of a single oscillator. The instrument produced a continuous varying tone which--accompanied by the rotating kaleidascopic projections--was used by Vladimir Rossiné at exhibitions and public events. (Sources: Keyboard Museum, Historie du synthe)

 
Rossiné's Optophonic Piano and one painted disk

1919 : American inventor Lee De Forest (1873-1961) was awarded several patents that would lead to the first sound-on-film technology with commercial application. In De Forest's system, the sound track was photographically recorded on to the side of the strip of motion picture film to create a composite, or "married," print. If proper synchronization of sound and picture was achieved in recording, it could be absolutely counted on in playback. (Sources: Wikipedia)

   
Producing optical soundtracks and two examples of sound-on-film

1921 : The Photokinema sound-on-disc system was developed by Orlando Kellum, and was employed to add synchronized sound sequences to D. W. Griffith's failed silent film "Dream Street". In sound-on-disc technology from the era, a phonograph turntable is connected by a mechanical interlock to a specially modified film projector, allowing for synchronization. The sound discs themselves were 16-inch, optically printed phonographic records, such as used in the Vitaphone. While this system failed to win wide support in the film industry, it did pave the way for sound-on-disc music instruments, such as the Lichttonorgel and the ANS. (Sources: Wikipedia, Wikipedia)

1927 : The cellulophone wass invented by the French engineer Pierre Toulon aided by Krugg Bass, in France. The Cellulophone ("Cellule Photo-électrique") was an electro-optical tone generator based instrument resembling an electronic organ. The machine had two eight octave keyboards and a foot pedal board. The sound was generated by rotating discs in which a ring of equidistant slits were cut (54 slits for the lowest note), different shaped masks were used for different timbres. The disks masked a light beam that flashed through the slits and on to a photoelectric cell, the speed of the rotating disk determining the frequency of the output signal, provided by a vacuum tube oscillator. (Sources: Keyboard Museum)

1929 : Abram Room directed the first sound movie created in the Soviet Union: Piatiletka. Plan velikih rabot (Plan of Great Works). A number of the future developers of graphical sound were involved in its production.

"When in October 1929 the first film-roll was developed, the crew members were amazed with the view of the first sound track they ever saw. It was Mikhail Tsekhanovsky who had voiced the idea: what if we take some Egyptian or ancient Greek ornaments as a sound track, perhaps we will hear some unknown archaic music? (Avraamov 1939). Each crew member immediately recognized in the new optical film sound process a means to effectively realize their long-standing ideas: Arseny Avraamov - to develop further his concept of ultrachromatic Welttonsystem and to explore the sonic qualities of new ornamental sound, Evgeny Scholpo - to develop his performer-less musical tools. The next day they were already working furiously on experiments in what they referred to variously as ornamental, drawn, paper, graphical, artificial or synthetic sound..." (Sources: Andrei Smirnov "Sound out of Paper", Wikipedia)

1930 : Soviet artist Arseny Avraamov produced the first hand-drawn motion picture soundtracks. This was achieved by means of shooting still images of drawn sound waves on an animation stand. (Sources: Keyboard Museum, Wikipedia, Theremin Center, Theremin Center)


Avraamov's sound drawings

1930 : Danish-born visual artist Thomas Wilfred designed the Clavilux color organ, which used rotating glass disks with hand-painted color patterns. Its inspiration came from a group of Theosophists who wanted to demonstrate spiritual principles through the use of light and color. Wilfred named the artform of color-music projections "Lumia." He stressed polymorphous, fluid streams of color slowly metamorphosing. He established an Art Institute of Light in New York, and toured giving Lumia concerts in the United States and Europe (at the famous Art Deco exhibition in Paris). He also built "lumia boxes", self-contained units that looked rather like television sets, which could play for days or months without repeating the same imagery.

 
Thomas Wilfred with his Clavilux Junior and several hand-painted disks

Although this is one of many examples from the "Visual Music" tradition, rather than a sound-producing instrument itself, Although this is one of many examples from the "Visual Music" tradition, rather than a sound-producing instrument itself, the relationship of the spinning disks to graphical sound techniques is significant. (Sources: Artists On Line, "The Dream of Color Music, and the Machines That Made It Possible" by William Moritz, Visual Music)

1930 : The Variophone was developed by Evgeny Scholpo in Leningrad in 1930 at Lenfilm Studio Productions, together with composer Georgy Rimsky-Korsakov. It was the most long living project: the Scholpo laboratory existed until 1951. The first practical result was achived in 1931. The technological basis of his invention was the method of photo-optic sound recording used in cinematography, which made it possible to obtain a visible image of a sound wave, as well as to realize the opposite goal - synthesizing a sound from an artificially drawn sound wave. "The method of Scholpo gives easier access to varieties of timbres. He doesn't shoot still images of sounds on animation stand, instead using paper disks with a circular images of combs with appropriate shapes of cogs, rotating synchronously with a moving filmstrip. Exclusive benefits of Variophon are in flexible pitch control and vibrato" (Solev 1935). (Sources: Keyboard Museum, Wikipedia, Theremin Center, forthcoming article "Graphical Sound in Russia" by Andrei Smirnov for the Computer Music Journal)

    
Scholpo working with the Variophone, several examples of the sound disks and a section of printed film

1930 : In Munich, Rudolf Pfenninger made his first films using his tönende Handschrift (Sounding Handwriting) method of artificial sound synthesis. This method involved photographing strips of paper with sound-curves drawn on them to represent each note of the soundtrack graphically. (Sources: Thomas Y. Levin, "Tones from out of Nowhere": Rudolph Pfenninger and the Archaeology of Synthetic Sound)

1931 : The Saraga-Generator was developed by Wolja Saraga at the Heinrich-Hertz Institut Für Schwingungsforschung in Berlin, Germany. The Saraga Generator was an unusual photoelectrically controlled vacuum tube instrument. The instrument consisted of a photoelectric cell mounted on the white painted inside surface of a box with a small slit cut on one face. A low voltage neon lamp was placed at some distance from the box and the performers movements interrupting the light beam caused variations in pitch. Envelope and timbre were affected by manipulating a hand held switch device, the overall volume being controlled by a foot pedal. The instrument had a tonal range of four octaves. (Sources: Keyboard Museum)

1931 : The "Radio Organ of a Trillion Tones" was created and developed by A. Lesti and F. Sammis in the USA. The Radio Organ used a similar technique as the Celluphone and variants - rotating photo-electric disks interrupting a light beam at different frequencies produced varied pitches an timbres from a vacuum tube oscillator; the principle was improved in the "Polytone". (Sources: Keyboard Museum)

1931 : British physicist E.A. Humphries created the first artificial speech synthesis when he is called upon to "correct" the optical soundtrack master to one of the first synchronized-sound films. The problem was that the villian in the film happened to share the same name as that of an aristocratic family, who threatened a lawsuit unless it was removed. Since the lead actress could not be recalled to the studio to re-record the spoken parts, Humphries was forced to paint each instance of the character's new name into the soundtrack by hand! (Sources: Thomas Y. Levin, "Tones from out of Nowhere": Rudolph Pfenninger and the Archaeology of Synthetic Sound)

1931 : Nikolai Voinov invented the "Nivotone" in the Soviet Union. This instrument optically read strips of paper hand-cut by Voinov as sound information. (Sources: Theremin Center, Theremin Center)



Voinov cutting paper strips, a paper soundtrack and the Nivotone

1932 : German abstract animator Oskar Fischinger (1900-1967) published his Sounding Ornaments article, detailing his own experiments with the synthesis of animated abstract, "absolute" images and optical sound. Unlike Pfenninger and many of the various Soviet artists who had an understanding of acoustics and could predict which shapes might produce specififc sounds, Fischinger approached the problem visually and conceptually by positing that there might be cultural isomorphism between a sound and a graphical shape which represents it. (Sources: Wikipedia, Rhythm in Motion, Visual Music, Thomas Y. Levin, "Tones from out of Nowhere": Rudolph Pfenninger and the Archaeology of Synthetic Sound)


Fischinger and his "sound scrolls"

"Between ornament and music persist direct connections, which means that Ornaments are Music. If you look at a strip of film from my experiments with synthetic sound, you will see along one edge a thin stripe of jagged ornamental patterns. These ornaments are drawn music - they are sound: when run through a projector, these graphic sounds broadcast tones of a hitherto unheard of purity, and thus, quite obviously, fantastic possibilities open up for the composition of music in the future". (Oscar Fischinger, "Sounding Ornaments" Wikipedia)

1932-1936 : Boris Yankovsky of the USSR invented the Vibroexponator optical sound instrument based on the "painted sound" techniques of Soviet film soundtrack composers. (Sources: Theremin Center)


Yankovsky's Vibroexponator soundtracks

1932-1939 : Boris Yankovsky develops his system of "syntones" and spectral mutations. (Sources: Theremin Center, Theremin Center, Theremin Center)

   
Yankovsky's syntones and spectral mutations

1934 : A. Lesti and F. Sammis's development of the Radio Organ of a Trillion Tones was christened the 'The Polytone Organ', this instrument was a three keyboard manual organ using the same sound production system as the 'Radio Organ' - rotating photo-electrical tone-wheel sound generation. The instrument was completed in 1934 and was one of the first multi-timbral instruments. (Sources: Keyboard Museum)

1934-1935 : The "Syntronic Organ" (1934) & The "Photona" (1935) were developed. Syntronic Organ was an electro-optical tone generator based instrument engineered by Ivan Eremeef and L. Stokowski and was able to produce "one-hour of continuous variation" created by an optically generated tone using films of tone-wheels. Ivan Eremeef later created the "Photona" a 12 electro-optical tone generator based system, developed at WCAU radio, Philadelphia, USA. (Sources: Keyboard Museum)

1936 : The Welte Light-Tone, designed by Edwin Emil Welte (1876-1958) in Germany, was an electronic instrument using electro-optically controlled tone generators. A glass disk was printed with 18 different waveforms giving 3 different timbres for all the octave registers of each single note. The glass tone wheel rotated over a series of photoelectric cells, filtering a light beam that contolled the sound timbre and pitch. (Sources: Keyboard Museum, Wikipedia)


Welte Light-tone disk

1936 : F. Sammis invented the "singing Keyboard", a precursor of modern samplers, the instrument played electro-optical recordings of audio waves stored on strips of 35mm film which were triggered and pitched when the player pressed a key. More recent instruments such as the Mellotron and Chamberlin use a similar technology of triggered and pitched magnetic tape recordings. (Sources: Keyboard Museum)

1938 : Russian engineer Evgeny Murzin (1914-1970) invented a design for composers based on synthesizing complex musical sounds from a limited number of pure tones; this proposed system was to perform music without musicians or musical instruments. (Sources: Wikipedia, Theremin Center, Theremin Center)

1939 : The book "Theory and Practice of Graphic Sound" was written (but unpublished) in the Soviet Union, focusing on the technique of "painted sound" and including Boris Yankovsky's important essay "Acoustic Synthesis of Musical Colours". The manuscript is kept at the Theremin Center archive. In part, Yankovsky's essay was published in Kinovedcheskie Zapisky #53, 2001. (Sources: Kinozapinski.ru, Andre Smirnov)


Painted soundtrack

1940's-1960's : Canadian filmmaker Norman McLaren (1914-1987), sometimes working with New Zealand kinetic sculptor and filmmaker Len Lye, created films whose sounds were created by drawing or printing various patterns (e.g. shapes such as triangles, circles, etc.) along the optical soundtrack area of the film. (Sources: Keyboard Museum, Wikipedia, Wikipedia, Wikipedia)


McLaren's squarewave cards

1947 : Lev Theremin (1896-1993) developed the "Buran" light-monitoring eavesdropping system while working as an imprisoned scientist for the KGB in the USSR. The system worked by receiving the reflected microwaves from a surface which could be modulated by sound inside a room (a glass window, for example), and converting that back to audible sound. These principles would later be incorporated into commonly-used laser "bugging" systems. (Sources: Theremin Center, Wikipedia)

1958 : Evgeny Murzin finished the first version of the ANS Synthesizer. In this instrument, sine waves are printed onto four glass discs. Each disc has 144 individual tracks printed onto it, producing a total of 576 microtones (discreet pitches) available to the user (a later version was completed in 1964 using 5 disks for a total of 720 individual tracks). These disks are arranged vertically from low frequencies at the bottom to high frequencies at the top. The convolved light is then projected onto the back of the synthesizer's interface.

 
The ANS synthesizer and a graphical score

The interface consists of a glass plate covered in opaque black "mastic" which constitutes a drawing surface upon which the user makes marks by scratching through the mastic, and therefore allowing light to pass through at that point. In front of the glass plate sits a vertical bank of photocells which send signals to band-pass amplifiers, each with dB trim switches.


The mechanical workings of the ANS

The glass plate can then be scanned left or right in front of the photocell bank in order to transcribe the drawing directly into pitches. In other words, it plays what you draw. The ANS is completely polyphonic and will generate up to all 720 of its pitches simultaneously if required. (Sources: Wikipedia, Theremin Center, Theremin Center)

1959 : The technique of Oramics was developed by pioneering British composer and electronic musician Daphne Oram (1925-2003) at the BBC Radiophonic Workshop, in England. It consisted of drawing onto a set of ten sprocketed synchronised strips of 35mm film which covered a series of photo-electric cells that in turn generated an electrical charge to control the sound fequency, timbre, amplitude and duration. This technique was similar to the work of Evgeny Scholpo's "Variophone" some years earlier in Leningrad and in some ways to the punch-roll system of the RCA Synthesiser. The output from the instrument was only monophonic relying on multitrack tape recording to build up polyphonic textures.

 
Oramics recordings by Daphne Oram

The attraction of this technique was a direct relation of a graphic image to the audio signal and even though the system was monophonic the flexibility of control over the nuances of sound production was unmatched in all but the most sophisticated analogue voltage controlled synthesisers. (Sources: Keyboard Museum, Wikipedia)





The Oramics machine as it appears today, in a London storage space awaiting refurbishment by Goldsmiths University (photos: Derek Holzer)

1971 : The Optigan was released by the Optigan Corporation, a subsidiary of toy manufacturer Mattel. The unusual feature of the Optigan was its method of sound synthesis; the Optigan optically read graphic representations of waveforms from a series of 12" celluloid LP sized discs, hence the name Optigan - 'Optical-Organ'. The Optigan read the discs by passing a light beam through the transparent discs, the beam was interrupted or reduced by the shape of the printed waveform and picked up by a photoelectric cell causing a variable voltage which was in turn amplified and passed to the speakers.


Optigan disk with organ in the background


Optigan disk close up (photo courtesy of Simon Jenkins)

The Optigan was essentially an optical sampler, the disks contained 57 loops of sounds which were recordings of real instruments, 37 of the loops were reserved for keyboard sounds ( with individual loops for each key) the other 20 loops being sound effects, rhythms etc. The celluloid discs were sold as a collection for Optigan owners and were mainly sustained organ sounds, as the continually spinning loops had no beginning or end it was impossible to create an attack or decay. (Sources: Keyboard Museum, Wikipedia)

1972 : French composer and instrument builder Jacques Dudon began his work with photosonic synthesis. His system, developed more extensively between the years 1984-2002, used stationary, spinning disks printed with optical patterns drawn in just-intonation scales, plus a series of moveable light sources and optical "comb filters" to produce microtonal music which Dudon associated with the Chakras of Indian spirituality. (Sources: Jacques Dudon, Dudon, Jacques. "The Photosonic Disk." : Experimental Musical Instruments, Nicasio CA 14 : 4, 1999.)


Dudon's photosonic synthesizer (from artwork of his "Lumieres Audibles" LP, 1996 Mondes Harmoniques)


Diagram of Dudon's system, where "L" is the light source, "D" the transparent disk with sound patterns, "F" the optical filter and "C" the photoelectric receiver


One of Dudon's photosonic disks

1977 : Greek-born composer Iannis Xenakis completed his UPIC (Unité Polyagogique Informatique du CEMAMu) system at the Centre d'Etudes de Mathématique et Automatique Musicales (CEMAMu) in Paris. Although a computer-based system, the UPIC interface was essentially the same as in Murzin's ANS: the composer drew lines on the screen with a light-pen representing pitches which changed over time depending on the direction of the line. The compositions could be played back at almost any speed, and could also be subjected to various transformations by graphically inverting, transposing, reproducing or otherwise manipulating the lines on the screen. The waveforms used by the UPIC as well as the volume envelopes were similarly drawn by hand at the start of a session. The UPIC realized Xenakis' dream of composing in a purely graphical manner, and can be seen as an extension of his mathematically-produced "visual" scores.(Sources: Wikipedia)


Block diagram of the UPIC system, with the drawing tablet in the center


Xenakis imagined his UPIC system would open up the possibilties of composition to those with no formal musical training, allowing them to create music in a direct, intuitive way


A section of Xenakis' UPIC score for "Mycènes Alpha" (1978)

1970's : Fiber-optic communication developed. This is a method of transmitting information from one place to another by sending light through an optical fiber. The light forms an electromagnetic carrier wave that is modulated to carry information. This system revolutionized the telecommunications industry and played a major role in the advent of the Information Age. The information age finally catches up with the artistic and musical use of modulated light, and light-based signal systems finally go digital. (Sources: Wikipedia)

ACKNOWLEDGEMENTS

I am deeply indebted to the research of Andrei Smirnov of the Theremin Center, Moscow. Without Smirnov's careful work in preserving and archiving many important documents from the 1920's to the 1960's, most of the information on the pioneering Soviet electronic music instruments, film sound and writings (as well as several surviving instruments) would be lost forever. Smirnov offers several extremely interesting lectures on the "Nonlinear History" of Soviet electronic music instruments, Lev Theremin and the interrelationship of espionage techniques and art. I can highly recommend these lectures as well as his workshops to academies, universities, symposiums or festivals. An introduction to Smirnov's work on graphical sound can be found here.

I would also like to thank the owners of the Keyboard Museum website for their exhaustive collection and documentation of vintage electronic music instruments, especially those invented before the 1960's. I consider their page, along with Andrei Smirnov's, to be one of the best resources for starting an investigation into the history of optical synthesis technology. UPDATE: as of 11 Sept 2009 I am sad to report that the Keyboard Museum website appears to be offline. This is quite unfortunate, however since there is no place to which I can redirect any of the broken links here, they must remain as they are...

This history was compiled for the TONEWHEELS project, as part of a residency at the Tesla media arts laboratory, in Berlin, Germany over the months of Oct-Dec 2007, and at STEIM during the last two weeks of February 2008. I am grateful for their support of this project as well.

Further research has been added over time between 2007-2010, and this "paper" has formed the basis for a lecture on the history of optical sound and graphical composition which I have given at numerous universities, festivals and arts centers since then.

Derek Holzer
Berlin, February 2010

CONTACT

derek ___AT THE DOMAIN___ umatic.nl