The Machine That's Saving the History of Recorded Sound

When recorded sound was in its infancy, more than 150 years ago, inventors still hadn't answered what was, to them, a fundamental question:

What does sound look like?

They knew what sound sounded like, of course, and even what it felt like but what would it mean to see sound on paper? It was this question that inspired the French inventor Édouard-Léon Scott de Martinville to design the phonoautograph, which is widely considered the earliest sound-recording machine. His theory was that if he could build a device that transcribed sound, he could read sound the way we read text.

"No one had really looked at sound waveforms before so he didn't really know," said Peter Alyea, a digital conservation specialist at the Library of Congress. "So he created basically what is, in current and modern terms, an oscilloscope."

De Martinville's device, modeled after the human ear canal, worked by having a stylus attached to a piece of parchment. "And so he just etched it with a diaphragm that would vibrate a little hog's bristle as he spoke into it," Alyea said. "He wasn't interested in actually recording sound and playing it back, he was interested in recording sound so he could look at it. He thought he could read the waveforms. He thought he could take someone speaking and transcribe something like, 'The cow jumped over the moon.'"

The inventor found, of course, that sound waves couldn't be read like text. The visual representation of sound varies based on amplification, not enunciation. But through his experiment, de Martinville ushered in a new era of recorded sound, the implications of which are too enmeshed in the technological world as we now know it to fully appreciate.

Here's de Martinville's April 9, 1860 recording of the French folksong, "Au Clair de la Lune," the earliest known recording of a human voice:

The clip is an odd and ethereal treasure of de Martinville's legacy. But more than that, it is a reminder of the inherent physicality of recorded sound. It took the engineering of new machinery to capture that wobbly strain at all, and more machinery still to resurrect it 148 years later. 

 "Au Clair de la Lune" is all over the Internet now, having proliferated digitally and endlessly since it was first discovered in 2008. (Before that, researchers believed a recording of Thomas Edison had made the earliest recording in the 1870s.) But in order for de Martinville's lost 20 seconds of melody to be found for the Internet age, scientists first had to figure out a way to turn his fragile paper recording—the transcription of sound de Martinville hoped he would be able to read—back into song that could be heard. 

To do so, researchers at Lawrence Berkeley National Laboratory used a combination of optical imaging and high-resolution scanning, then converted the patterns they captured into readable—that is, playable—sound. The technology, originally developed by particle physicists, allows for optically recovering sound recordings without touching the medium on which the sound is recorded. This technique has been around for more than a decade now. The machine invented at Berkley is now, through a partnership, the center of sound preservation efforts at the Library of Congress. 

"They called it IRENE because the first recording they did an image of was 'Goodnight, Irene,' by the Weavers," Alyea told me. "And then they made it a reverse acronym and decided it would stand for Image, Reconstruct, Erase Noise, Etcetera."

IRENE lives in the cool basement of the library's James Madison building. It looks, well, like a machine—all metal and lasers and motor—a little bit like a cross between a microscope and the guts of a home printer. How IRENE works: It's basically a digital-imaging device. So, say you have a vinyl record you want to preserve. IRENE scans the topography of the disc, and sends the images it produces to a computer. Separate software on the computer then converts those images into sound. 

"You have optics that magnify the surface of the disc," Alyea said. "You have a laser  that actually drives a motor that moves the whole system up and down like the autofocus on your camera. Most of these discs are not flat at all and there's a fairly small area of focus. Some light comes in here and is split and shone directly on the surface of the disc, and then there's a camera." More simply, IRENE is a mapping device that tracks the terrain of a recorded medium—like the pattern of the grooves etched onto a flat vinyl record. 

The device knows how to image the architecture of other recorded formats, too, including older shellac-coated vinyl, and glass records like the ones made during the rationing of World War II. In the ten years since IRENE was invented, institutions have discovered a spate of esoteric formats and unknown recordings, strange items in long-forgotten collections that haven't even been catalogued. 

"These whole different ranges of formats that IRENE can save that are in people's collections, and people don't even know what's on them," said Fenella France, the chief of the preservation research and testing at the Library of Congress. "Things keep coming out of the woodwork."

IRENE is even able to resuscitate the sound recorded to wax cylinders from the late 19th century, which became the first medium for commercially recorded sound. 

People played them on phonographs, and certain kinds of wax were soft enough to be shaved down, making the cylinders rewritable. 

Whereas the topography of a record is like a riverbed, a cylinder's recording is etched around its circumference like a skyline—the stylus would trace the path of a cylinder by going up and down rather than side to side like on a vinyl record—which means imaging a cylinder requires different motion than would be required for a record. For a record, a groove's squiggly line is what determines the sound you hear. With a cylinder, the depth of the groove is what counts. "So it has something called a confocal probe that uses a lens that focuses light at different distances to get three-dimensional data," Alyea said. If you tried to image a cylinder the way you image a vinyl record, "all you'd see would be a straight line," he said.

In 2012, a team of curators at the Smithsonian worked at the Library to retrieve the audio from a set of experimental recordings made by telephone pioneer Alexander Graham Bell in the 1880s. What they uncovered was remarkable. "A recording made on a brass disc covered with wax yielded a recitation from 'Hamlet,'" Carlene Stephens, a curator at the Smithsonian, told the Library of Congress in 2012. "A glass photodisc features the word 'ba-ro-me-ter' enunciated over and over…It made the hair stand up on the back of my neck. It was so thrilling. It was so eerie. It was so much a glimpse into a time period we have never heard from before, ever."

IRENE has also enabled scientists to piece together complete sound from broken media, including minor glitches like vinyl imperfections and major damage like records that are physically broken in half or worse. Here's a clip of a 1940s song that researchers reassembled with software after imaging a shellac disc that was smashed into six separate pieces: 

The clicks, pops, and skips of regular vinyl wear don't interfere with the sound IRENE images because the machine never touches the medium the way the stylus on a record player would.  "So when you play a damaged record with a stylus, you get a skip in the groove," Alyea said. "This is not a problem with IRENE because IRENE just sees that as a small little blemish and just goes right through it." 

For more severe degradation or damage, it's time consuming—but possible!—for IRENE to image shards of a record or cylinder separately, then piece back together full recordings using computer software. Eventually, scientists say, IRENE could be hooked up to additional 3D printing technology so that sound retrieved from an old format could be preserved and reprinted onto something new. But the resolution isn't quite where it needs to be. ("We're imaging on the micron level and you can't quite do that with a printer," Alyea told me. "But it certainly seems quite feasible that at some point in the near future you'd be able to print it out. That'd be fun.") In the meantime, the library is in the process of establishing workflow so it can figure out which recordings to prioritize and, hopefully, save as much historically valuable sound as possible. 

In the scope of human history, the era of recorded sound is a blip—and yet the volume and fragility of what's been created in that time is overwhelming. Already the Library of Congress is hard at work on the preservation of CDs, and  the newest wave of digitized recordings presents a litany of fresh preservation challenges. The perceived dichotomy, though—this idea that there's analog on one side and digital on the other—is all wrong. Ancient formats and modern formats may look different but they both require hardware, machinery you can hold in your hands. 

"They're very different kinds of formats, but they're all physical," Alyea said. "Still, even with digital data, there's no way to just—I mean, in theory you could have someone sit down and memorize all the values or something, but then it's in their brain. So with recorded sound, it's always something physical."