Remembering the Good Days at Bell Labs

I thank the organizers of this meeting for their kind invitation to Maui. Living in Germany, Maui is halfway around the world for me. But I was more than glad to come. And I am particularly grateful to my Japanese friends, Hiroya, Sadaoki, and Fumitada, to have accepted, even welcomed, my somewhat unorthodox topic: Remembering the Good Days at Bell Labs.

When I talk about the "good days at Bell" I am not implying that people at other times and other places cannot enjoy their work as much as we did. But there is a general perception that Bell Labs was something special - a "national resource". This did not come about by accident: Bell management decided early on that freedom to pursue one's own ideas was the best well-spring of innovation. In giving this talk I also have a faint hope that industrial research at some future time will see the light again and not limit research to short-range goals and immediate profit. Some of the greatest advances in the past have come from taking the long-range view. - Of course, it helped that before 1984 Bell Labs figured as a (government-sanctioned) expense in the AT&T budget.

How did I get to Bell Labs from distant Germany in the first place 50 years ago? Let me explain: I was studying at the University of Göttingen when a noted linguist from Bonn, Meyer-Eppler, visited us around Christmas in 1951 to give a talk at the General Physics Colloquium. I don't remember the details, but he talked about Shannon and Information Theory. Perhaps this was the first time I had heard the name Shannon and I was immediately fascinated. This sounded much more interesting than the nuts-and-bolts of experimental physics. But after the talk, during the Christmas party, people professed not to have "understood a word" of the talk. The chairman of the colloquium even seemed a little miffed when I dared pronounce the lecture "interesting". How dare I! How could I find a field that was devoid of relativity and the quantum to be interesting? But undeterred, I approached my professor, Erwin Meyer, whether he could recommend me for a job at Bell Labs.  Unfortunately, the answer was "no" because he remembered one of his students before the war who wanted to emigrate but Bell said they didn't hire any foreigners. End of dream to join Bell! Or so it seemed.

Two years later I learned that William Shockley, co-inventor of the transistor, was coming to Göttingen actually looking for bright students for Bell. Well, I flew back to my professor telling him "they do take foreigners... they are even looking for them". Fine he said, I am corresponding with one the research directors of Bell and I will put in a few lines for you. (Later his secretary told me that he had "put in" a whole page singing my praise.)

Two weeks later, I received an invitation for an employment interview by said research director. I was to meet him in the lobby of the Dorchester Hotel in London, no less, on April 25, 1954, at 2 p.m. Well, the interview (I have described the details in my book Computer Speech) apparently went well, and after six weeks I received an offer of permanent employment, by Bell Labs for $640 a week - a princely sum then, especially for a perennially impecunious student. One alternative would have been for me to join Siemens in Munich for the equivalent of $125. Well, I accepted the Bell offer - in fact I would have worked at Bell for nothing, but I didn't mind getting paid for what was for me a dream come true.

So on September 30, 1954, I arrived in New York on the Italian liner "Andrea Doria" - still afloat then. I was met at the pier by my future supervisor, Ralph Miller, and the director, Winston Kock, with a long black chauffeur-driven limousine. But our first step was not the Labs but a fancy restaurant, with a Menu in French that I couldn't understand, except for the word Bratwurst.

The following three days were perhaps the most memorable in my entire life: My first regular job, a new country and a new language (I used to pronounce "steak" steek then) and, yes, a new regular companion: on my third day in the U.S. I travelled to New York and met a young woman from Bulgaria, Anny Menschik, who was working for Radio Free Europe and who soon became my wife.

At the Labs, Win Kock encouraged me to continue my thesis work which concerned concert hall acoustics, but I figured - this being the telephone company - I better do something more germane to the telephone business. So I picked "speech" as my new research field - and speech it was for the next 50 years. Nobody objected to my total ignorance of this field. Electronic Engineering, especially, was an enigma to me, but I learned a lot by "osmosis" you might say. Bell Labs was an ideal place not only for doing research but for learning. The doors were wide open and people friendly and communicative. Nobody was guarding any "trade secrets".

Just one tiny but typical example: one day I was reading a technical paper bemoaning "the metallic twang" of existing artificial reverberators for electronic music. I thought what these audio types need is a reverberator that - when seen as an electrical filter - has an all-pass frequency response. So I swivelled around in my chair and asked my office mate, electrical engineer and country musician Ben ("Tex") Logan, "are there any all-pass filters with an exponentially decaying impulse response?" Ben's answer was "yes" and colorless artificial reverberation was born. As far as I know, Bell Labs didn't make a penny on the patents. But you can now find colorless reverberators in practically all electronic music instruments, Yamaha and the rest.

The Bachelor

Besides having to adjust to new technical fields, the social adjustments also were not always easy. I was 28 when I joined Bell and still unmarried. This was somewhat unusual at the time and my Executive Director, Bill Doherty, introduced me to one and all with "This is Dr. Schroeder. He just joined us from Germany - and he's a bachelor." Or, as my old math professor from Göttingen (Wilhelm Magnus, then at New York University) said: "This is not a country for bachelors. Either you will return to Europe within the year or you will get married." Well, he was right, I did get married 17 months after my arrival. My Best Man was Ed David, who taught me how to grill steak properly (among many other American essentials), and later was ensconced in the White House as a presidential Science Advisor.

Another thing that suprised me was that people were not only very accessible but, I thought, quite humble like S.O. Rice of (mathematical) noise fame, one of my heroes even before joining Bell Labs. His office was at the old New York headquarters at 463 West Street. I had always imagined that such a famous mathematician would reside in a palatial office with lesser people scurrying around him. But no, Steve was actually sharing an office with several other people who treated him like an ordinary human being. And then, a little later, they all went out together, on foot, for lunch in a little restaurant in Greenwich Village. I thought someone as famous as Rice would be carried by a company limousine... . Well this was just one of my European prejudices that took some time to subside. (Incidentally, after lunch, when I returned to my car, which was parked illegally under the West Side Highway, I saw a cop writing a ticket. When he saw me, he apologized profusely but since he had already filled in my plate number he couldn't issue it to someone else. But he urged me to leave the car in the illegal spot for the whole day - the ticket was good for 24 hours! He was visibly disappointed when I told him I had to return to New Jersey right away.)

As auspicious as my beginning at Bell may sound, it was, in retrospect, rather pedestrian. My first speech job at the Labs was inverse filtering: you spoke a vowel sound into a microphone displayed it on an oscilloscope and adjusted a tuneable filter while the formant "wiggles" disappeared and a relatively smooth waveform remained, which was a pretty good approximation of the glottal waveform. Neat! As I learned much later, as I was picking up more Electrical Engineering expertise, I was putting "zeroes" on top of the "poles" in the complex frequency plane. Of course, if the speech signal is the least bit noisy, inverse filtering will only lower the signal-to-noise ratio. In order to increase the SNR, you have to use matched filters, just the opposite of inverse filters, piling poles on poles.
So inverse filtering is no good for pitch tracking for low-quality telephone signals, which was one thing we were trying to do.

My next project was a 6-channel vocoder. Even though a complete novice, I felt that 10 or 16 channels for a vocoder was more than was really required, I thought that 6 channels would be enough. Indeed, I thought single-tuned filters would be selective enough, if I added some "lateral inhibition" to my filter bank: subtracting from each filter the outputs of its two neighbors. It took me many months to build this contraption because I had to wind my own coils. Imagine a Ph.D. in physics, winding coils and spending his days soldering sickly circuits - the technical assistance that I had been promised did not materialize until a year later.

Be that as it may, after all the manual labor and mental gymnastics I had invested in my vocoder project I was convinced that it produced not only intelligible speech, but high-quality, human-sounding speech. Thus, I alerted my supervisor, who alerted our director, who - if you can believe it - alerted the president of Bell Labs (Mervin Kelly). They all filed into my lab and the demonstration began... In retrospect, I am convinced they hardly understood a word of what my vocoder said. But everybody was very polite and all left with best wishes for the future of my project.

Years later, a cooperative student from MIT, Tom Crystal, played me his speech compressor in a noisy, fan-infested, lab. He thought it sounded perfect and so did I. So I suggested that we listen in a very quiet location and I put earphones on - and it sounded horrible. Recently, Fumitada Itakura reminded me that when I first visited him in Japan in 1968, I followed the same procedure. But Fumitada's "maximum-likelihood" compression still sounded pretty good, though not as good as at the International Congress on Acoustics in Tokyo, where our two papers, Fumitata's paper and mine (on Adaptive Predictive Coding) were presented back to back. (Little did we realize then that our two methods were actually identical. Yes, Linear Prediction and Maximum-Likelihood (or Maximum-Entropy) exploit exactly the same redundancies in a speech signal!)

Another horrible contraption I built was an analogue Fourier-synthesizer capable of generating periodic waveforms with 31 harmonic frequencies. The gadget comprised almost 1000 little trimmer pots and a large number of highly unreliable transistors (this was around 1956!). Nevertheless, I got some nice waveforms that helped me solve the problem of minimizing the peak-factor of periodic waveforms by choosing the proper ("Schroeder") phases.

On a more fundamental level, I was intrigued by the relationship between vocal tract shape and its resonances. Of course the problem is ambiguous: different geometric shapes, as the ventriloquist knows well, can have the same acoustic responses. But by measuring the impedance of the vocal tract as seen from the lips unique area functions from acoustic data can be obtained.

Incidentally, the loathsome analogue-circuit building days were soon over. Around 1955 Max Mathews appeared on the scene coming from MIT and he taught us that every circuit we were building could be seen as performing some mathematical operation and as such, if it could be digitized, could be done by computer. In fact, John Kelly, Carol Lochbaum and Vic Vyssotsky wrote a block-diagram compiler, called BLODI-Compiler, that could be used by computer dummies like myself to simulate almost any electronic circuit.

Although I had been raised in Germany under Hamming's motto "Don't compute - think!" I was soon one of the heaviest users of digital simulation including, with Ben Logan, a "Harmonic Compressor" that comprised 400 narrow bandpass-filters.

Of course, the simulation didn't run in real time (this was in 1961) and some eyebrows were raised over my simulations that took several hundred times real time. Even earlier, in the late 1950s, I had started simulating frequency responses of concert halls: a complex Gaussian process in the frequency domain. To accumulate enough statistical data on some parameters that couldn't be calculated analytically, I let the computer run entire weekends. At $600 an hour, the cost was of course astronomical. But the only financial consequence to me was that my batch-processing budget was raised. Talk about "the good days" at Bell labs! I never had to write a research proposal in all my years at Bell - just do it and the money would be provided (unless it was "additions to the plant", which were strictly budgeted.)

After the frequency-domain simulations, I began concert-hall simulations in the time domain, in other words: convolving the music signal with the hall's impulse response. The final output was of course reverberated music. When we first did this we had a little amplifier with a loudspeaker standing behind the digital-to-analog converter to monitor the computer output. When the people who ran the converters heard the music coming from their machines they thought some prankster was trying to fool them. But no, the music did come out of the computer - something entirely new in the early 1960s.

Later I began simulating 16-channel vocoders, again in very "unreal" running times. Some people thought I was crazy. They had envisioned simulations for simple waveform coders, such as delta modulation or differential pulse-code modulation. But that was Bell Labs research in the years gone by: you could do "crazy" things if you so felt (and didn't mind a few raised eyebrows). A good case in point is Peter Denes's successful advocacy of dedicated laboratory computers for speech research - in defiance of considerable doubts by higher management which was still wedded to "computing centers" with big machines at the time (ca. 1964).

By 1966, I had become totally disenchanted by the speech quality of vocoder speech. I thought that instead of the "rigid" encoding of the speech signal practised by vocoders, we should look for methods of speech coding that left "room for error". At this point I remembered the work of my friends in the picture-coding community, especially an engineer named Ernie Kretschmar with whom I had always maintained cordial relations. The method of choice in picture coding was of course prediction: point by point, line by line and frame by frame. I thought  something like that should also work for speech signals, except that the predictor parameters should change with the speech sound: adapted to the different spectra of different speech sounds. That is why Bishnu Atal and I called our first papers Adaptive Predictive Coding, later called linear predictive coding (LPC) and CELP (for code-excited linear prediction). Apart from the technical breakthrough that LPC represented, its genesis is a beautiful example of the cross-fertilisation that was possible, even encouraged, at Bell.

A beautiful illustration of the freedom we enjoyed in the Labs and the good communication between researchers is the computer art work by Mike Noll and Leon Harmon. He and Ken Knowlton had the idea to use the computer cum plotter to generate graphic output that no human could ever hope to draw by hand. Another idea was to produce graphic art that looked different from different viewing distances.  I soon joined the fray, peopled by Noll, Bela Julesz*, Harmon and others. At one of the monthly research directors meetings, instead of presenting the latest progress in speech coding, I showed my new computer graphics - and nobody complained. In 1964 there was a first exhibition of Computer Art at the Howard Wise Gallery in New York City. Computer Art had arrived! I even earned some money with my computer graphics - enough for a taxi ride and dinner for two in Zagreb. Here are some examples of our computer creations, programmed by Sue Hanauer.

Unbeknownst to me, computation of this image from number theory required shutting down the entire Bell computer center to marshal the necessary random access memory.

I also used computer graphics for purely scientific endeavors. With Mohan: Sondhi I constructed an "Acoustic Camera", which was capable of reconstructing a physical object from its blurred acoustic shadow,:

With Mohan, I also did a study of computer ray tracing in the deep ocean:

This was in connection with what I called "Volume Focusing" (suggested by self-steering array antenna arrays for communication satellites). Volume focusing was capable of focusing on a specified volume in the ocean.

Philharmonic Hall, New York

In September 1962 Philharmonic Hall at Lincoln Center for the Performing Arts (located on Upper Broadway) had opened to great fanfare with a gala concert under the baton of Leonard Bernstein in the presence of the First Lady of the Realm: Jacqueline Kennedy. But audiences were less than enthusiastic about the new hall's acoustics. In its predicament, Lincoln Center turned to a trusted friend on Lower Broadway: AT&T, which in turn turned to Bell Labs where, being in charge of acoustics research at the time, it ended up in my lap.

A Committee of Four was formed under the chairmanship of Vern Knudsen, Chancellor of UCLA and one of the founders of the Acoustical Society of America. - Of course, much of the acoustic trouble could have been avoided if Leo Beranek's original designs for the hall had been strictly adhered to!

Under the so-called Consent Decree between the US-Government and AT&T, Bell Labs was actually prohibited from acoustic consulting. So it was decided that I would limit myself to making the necessary measurements to analyze the hall. And analyze we did, using computer-generated Hamming-window tone-bursts as an excitation signal and matched filtering on the acoustic output from the hall. I was assisted in this work by Gerhard Sessler, Jim West, and Bishnu Atal.

Before starting the measurements, I asked the ushers, students of the Julliard School of Music, whether there was one good seat in the hall and they pointed to Seat A15 on the second balcony as really good. So I decided to include A15 in our measurements. Figure 6 shows the received acoustic power along the center aisle for two different frequencies, confirming the main subjective complaint namely the poor low-frequency response of the hall.

In the center of the orchestra floor, there is a difference of almost 30 dB between middle notes (750 Hz) and low notes (125 Hz). By contrast, the difference is only 4 dB at Seat A 15!

What is the cause of this lack of low frequencies (which made the celli in tutti passages nearly inaudible)? By time-gating the responses on the computer, we were able to isolate different reflections and as Figure 12 shows, the culprit were the overhead reflecting panels.

In addition to the poor base response the main problem with Philharmonic Hall was a feeling of "detachment" from the music. To get at this fundamental difficulty, I was able to persuade the German Science Foundation (DFG) to underwrite a large-scale study of concert hall quality. My collaborators (D. Gottlob and K. F. Siebrasse) made recordings with a specially designed "dummy-head" in 22 major halls, which were subsequently reproduced in an large anechoic space by means of an electronic filtering method illustrated in Figure 13.

The main finding of this study, involving thousands of paired comparison tests, was that for good acoustics, all else being equal, there should be strong early reflections -  a difficult goal, given that most modern halls are wide and have a low ceiling which favors sound arriving in the median plane of a forward-facing listeners head. To overcome this problem I proposed diffusing surfaces based on number-theoretic principles to be incorporated in a hall's design. Figure 14 illustrates the most widely used reflection phase grating based on quadratic residues.

The accurate measurement of reverberation time also benefited from our work on Philharmonic Hall.

Ideally, concert hall measurements should be made with music as an excitation signal. This can be done by measuring the modulation transfer functions, both on the stage and in the audience area and forming their ratio.

With Atal I also studied sound decays by means of computer ray simulation. The results showed the inaccuracies of existing reverberation time formulas:

No memories of the good days at Bell would be complete without remembering some of the great people we were privileged to work with like Jim Flanagan who joined us around 1958. Of all the people in the acoustics department at Bell Jim was the only bona-fide speech researcher before coming aboard. After fathering innumerable advances in speech and acoustics, Jim went on to the highest international honors, especially the Marconi Medal, bestowed by the Spanish king, and the National Medal of Science, a unique honor in our field, presented to Jim by President Clinton in a White House ceremony.

John Pierce

One of the greatest leaders at Bell was the unforgettable John Pierce. In fact, Pierce was the most inspiring boss and mentor I had between 1955 and 1971 when he left Bell Laboratories to join his old alma mater, the California Institute of Technology. By his writing he inspired my own writing - although when I once told him I might call my next book "Number Theory for Almost Everyone" he suggested instead, in his usual acerbic mode, "Number Theory for Almost No One". But while often blunt, John could be quite sweet too, as when he called my Voice Excited Vocoder "The first speaking machine that sounds human."

While John and I were watching the unfolding debacle of the avant-garde spectacle "9 Evenings: Experiments in Art and Technology" at the 69^th Regiment Armory in New York City. I was concerned about the bad press friend Billy Klüver (1927-2004) and EAT might get. But John reassured me "They will be written up in the New York Times and that's the main thing - never mind what they actually write".

John always impressed me by his honesty. Every time I had something unpleasant to report, he would immediately call his boss, Vice President Baker, to relay the bad tidings. I was impressed because the culture in which I had been raised, honesty - volunteering the truth - could be deadly. Lack of forthrightness, I believe, was in fact one of the root causes for the failures of Kaiser Wilhelm's Germany and Hitler's regime.

When John was mistaken for the inventor of the Traveling Wave Tube, which happened not infrequently, he would reply "Rudy Kompfner invented it - I only discovered it". Meaning that when John went to England on an wartime mission to search for ideas for a new, more powerful microwave generator, he found Austrian refugee architect Rudolph Kompfner instead, working on a microwave amplifier that unfortunately had a tendency to become unstable and oscillate. John exclaimed: "that's just what we are looking for, an oscillator!". This also led to the low-noise traveling-wave amplifier without which John's idea for an ocean-spanning satellite communication system would have been impractical. (Pierce was of course the inventor of the reflex klystron which was ideal as a local oscillator in microwave receivers. - In my Göttingen Ph.D. thesis I used two such klystrons to investigate the resolved resonances of metallic cavities modeled after concert halls, something that could not be done with sound.

An example of freedom at Bell, one where it backfired, occurred with a Swedish linguist we wanted to hire in the mid-60s. His future department head, Peter Denes, who was to take him out for dinner, asked me what he should answer if the candidate enquired about "freedom" at the Labs. I told Denes to reassure the candidate that he had total freedom - he could do whatever he wanted to do. The next thing we knew his office and the corridor nearby were adorned with posters of Mao Zedong, Ho Chi Minh and Che Guevara - just to test what we meant when we guaranteed him complete freedom. As you might imagine - this was the height of the Vietnam War - there was quite a commotion. People came to see me saying they couldn't continue with their classified, secret work. I had to reassure them that posters couldn't actually see. Then plant security got wind of the situation. They called me and I told them I would take charge, everything was "under control" - which of course it wasn't. But at least I got the security people off my back. Then I called my boss, John Pierce, and explained the whole thing to him. When I was through, he asked me what I was going to do and I said "Nothing, John". He replied with a single word: "Right". Of course, in less than 48 hours the whole thing had blown by.

Bill Baker

One person from whom I learned a lot about human relations and research administration was our Vice President Bill Baker. Bill would always think long and hard before making a decision - which, however, he often communicated in incomprehensible language. In other words, he made you think hard too and hopefully come to the same conclusion. This could be very frustrating but I, for one, preferred his indirect way of operating to the "methodology" of his successor, a Noble Prize winner no less, who always seemed to act first, then talk and finally, as a last resort, start thinking.

Bill Baker was also instrumental in getting my main collaborator Bishnu Atal admitted to the U.S. I had heard of Bishnu in the late 1950s as a very good student at the Bangalore Institute of Science in India. After a very long telephone conversation I had with Bishnu, I was able to persuade our management to make an offer of employment. However, Bishnu didn't come - he couldn't get a visa. After waiting for 2 years, I asked Bill Baker to intervene, who had Bishnu declared as "crucial to our efforts in national defence". So Bishnu got a special dispensation from Washington forthwith, and he appeared at Murray Hill in no time - thanks to the excellent relations Baker had with various government departments.

John Tukey and other Bell mathematicians

Another memorable character was John Tukey who divided his time between Bell Labs and the Princeton Statistics Department. John, who seemed to thrive on 6 glasses of skim milk for lunch, is of course best known for his (re)invention (with Jim Cooley) of the Fast Fourier Transform which has revolutionized digital signal processing thanks to the joint efforts of Larry Rabiner, Jim Kaiser, Ron Schafer, and numerous collaborators inside and outside Bell. This was the genesis of the IEEE Signal Processing Society.

Incidentally, Tukey added some of the most important words to the English language: bit and the less well-known cepstrum - and thereby hangs a story: In the late 1950s U.S. and Soviet diplomats and scientists were negotiating a nuclear test ban treaty in Geneva, Switzerland. One crucial question was how to distinguish reliably between underground nuclear explosions and earthquakes. Tukey (with Bogert and Healy) suggested untangling the multiple reflections from the earth's mantle from the direct shock wave by the "cepstrum" method which meant taking the Fourier transform of the logarithm of the power spectrum of the signals received by various strategically placed geophones.

Being always close to Tukey, as a kindred (mathematical) spirit, I soon heard of the cepstrum idea and thought it was just what was needed for pitch detection for speech signals (namely to untangle the formant frequencies from the fundamental frequency, the physical correlate of pitch). This was in the summer of 1962 and I had to attend some meetings in Europe, so I assigned the problem to one of our exchange students, A. Michael Noll. When I returned to Murray Hill a few weeks later, Mike had solved the whole pitch problem. The cepstrum has since found numerous other applications in speech coding and of course geophysics.

Other mathematicians whose advice proved invaluable were Jessie MacWilliams, who taught me finite fields (which I needed for the design of diffusing surfaces in concert halls) and Dave Slepian. Calculating reverberation times of acoustic enclosures (auditoria, opera houses, etc.) is notoriously difficult. According to Dave, one has to solve an "impossible" integral equation. But a day later, Dave brought mathematician Ed Gilbert along who had the brilliant idea of turning the integral equation into two equations which could then be solved iteratively on the computer. It worked like a dream and David Hackman, a cooperative student from Columbia University, obtained some beautiful results showing the importance of absorber placement within an enclosure.

Another example of cross-discipline interaction at Bell was the work on Hadamard coding of two-dimensional images that I did with mathematician Neil Sloane (and which earned me the Erdös-number 3). Bell Labs provided, in fact, a sheer inexhaustible pool of helpful talent. How could I have done our work on concert hall quality without the scaling methods of psychologists Roger Shepard, Doug Carroll, and Joe Kruskal?

The most famous of all the Bell mathematicians was of course Claude Shannon, inventor of information theory. My contacts with him were relatively sparse but we were standing daily at lunch time in the same cafeteria chow-line which gave me, the novice, not a little moral uplift. Many years later, when I was a bit better known around the Labs, he occasionally consulted me on some acoustics problem or other. (I remember he once came to my office with a funny-looking little trumpet and asked me whether I could calculate its resonances from its shape.) Another classical Bell person was Richard Hamming, father of the Hamming distance and inventor of the Hamming error-correcting codes.

I am forever thankful for the many good years I spent at Bell Labs and for the superb people I was privileged to know there. Of course I was too young to have much interaction with the "Old Guard": Walter Brattain, Hendrik Bode and Serge Schellkunoff. (But I was once engaged in a joint project with Harry Nyquist.) And I thoroughly enjoyed my friendships with the next generation: Conyers Herring, Bob Lucky, Ron Graham, Elwyn Berlekamp, Henry Landau, Larry Shepp, Andrew Odlyzko, Ingrid Daubechies, Bob Thurston, Sid Millman, Henry Pollak, John Kelly and many others.