The Untold Story of the Man That Made Mainstream Encryption Possible

Longtime tech journalist Steven Levy’s new book, Facebook: The Inside Story, details how Mark Zuckerberg transformed a tasteless dorm room social networking experiment into the world’s biggest social networking business. In honor of that release, we thought we’d share an excerpt of an earlier Levy book, Crypto, about a man who ran in the opposite direction of Facebook’s data exploitation and privacy breaches. This is the story of Whit Diffie, who transformed how we think of encryption, paving the way for the digital security we enjoy today.

Bailey Whitfield Diffie, born June 5, 1944, was always an independent sort. As one early friend remarked, “The kid had an alternative lifestyle at age five.” Diffie didn’t read until he was 10 years old. There was no question of disability, he simply preferred that his parents read to him, which seemingly they did, quite patiently. Finally, in the fifth grade, Diffie spontaneously worked his way through a tome called The Space Cat, and immediately progressed to the Oz books.

Later that year his teacher at P.S. 178 — “Her name was Mary Collins and if she is still alive I’d like to find her,” Diffie would say decades later — spent an afternoon explaining something that would stick with him for a very long time: the basics of cryptography.

Diffie found cryptography a delightfully conspiratorial means of expression. Its users collaborate to keep secrets in a world of prying eyes. A sender attempts this by transforming a private message to an altered state, a sort of mystery language: encryption. Once the message is transformed into a cacophonous babble, potential eavesdroppers are foiled. Only those in possession of the rules of transformation can restore the disorder back to the harmony of the message as it was first inscribed: decryption. Those who don’t have that knowledge and try to decrypt messages without the secret “keys” are practicing “cryptanalysis.”

Although Diffie performed competently in school, he never did apply himself. Only stratospheric scores on standardized tests enabled him to enter the Massachusetts Institute of Technology in 1961 to study mathematics. He took up computer programming — at first, Diffie now says, to get out of the draft. Diffie accepted a job at the Mitre Corporation, which, as a defense contractor, could shelter its young employees from military service.

Also Read: Advisory Guidelines on Key Concepts in the PDPA: 23 Chapters

Diffie’s team did not have to work at the Mitre office, but in 1966, became a resident guest of Marvin Minsky in the MIT artificial intelligence lab. At the AI lab, information was assumed to be accessible as the air itself. There were no software locks on the operating system written by the MIT wizards.

Unlike his peers, however, Diffie believed that technology should offer a sense of privacy. Diffie would often engage his boss, the mathematician Roland Silver, in conversations on security. Inevitably, cryptography entered into their discussions.

Silver had some knowledge in the field, and explained that there was a lot of work going on — behind steep barriers erected and maintained by the government’s intelligence agencies. Diffie resented this. Cryptography is vital to human privacy! he railed to Silver. Maybe, he suggested, passionate researchers in the public sector should attempt to liberate the subject. “If we put our minds to it,” he told Silver, “we could rediscover a lot of that material.”

Silver was skeptical. “A lot of very smart people work at the NSA,” he said, referring to the National Security Agency, the U.S. government’s citadel of cryptography. Created by President Truman’s top-secret order in the fall of 1952, the National Security Agency was a multibillion-dollar organization that operated totally in the “black” region of government, where only those who could prove a “need to know” were entitled to knowledge. In the early 1970s, none of this was discussed publicly. Within the Beltway, people in the know jokingly referred to the organizational acronym as No Such Agency.

In 1969, his funding having run out, Diffie finally left Mitre. He and his girlfriend moved west, and Diffie went to work at John McCarthy’s Stanford Artificial Intelligence Lab, where he was led into a deeper consideration of privacy concerns. There, he was referred to an assistant professor of electrical engineering by the name of Martin Hellman.

Born and raised in New York, Hellman had got his doctorate from Stanford in 1969 and his first job was at IBM research, where he took a serious interest in cryptography. After he left IBM in 1970, he accepted a post as assistant professor at MIT, where he made crypto the focus of his research, before moving to Stanford. He resisted the temptation to do what the vast majority of scientists in his field had already done: work within NSA strictures. After publishing his first paper in the field of cryptography, he had been stuck for a follow-up. Enter Whit Diffie. “It was a meeting of the minds,” says Hellman. Both Diffie and Hellman firmly believed that the advent of digital communications made commercial cryptography absolutely essential. Hellman hired Diffie as a part-time researcher.

Also Read: Free 8 Steps Checklist for Companies to Prevent Data Breach

In March 1975, a dry government document produced a shock wave to the Stanford duo. It was a Federal Register posting from the National Bureau of Standards (NBS) that proposed something seldom ventured in the public literature: a brand-new encryption algorithm that had come out of work IBM had done with the government, called the Data Encryption Standard, or DES.

Though Whit Diffie and Marty Hellman regarded the Data Encryption Standard as a tainted and possibly fraudulent gambit by IBM and the United States government, its introduction was in a strange way an important gift to the Stanford researchers. By combing through the available technical data on the proposed standard—and speculat­ing on what was not made public—Diffie and Hellman had a new prism through which to consider their own efforts. Ever since Diffie had heard the first reports of the government standard, at a 1974 chowdown at Louie’s, the Chinese restaurant where Stanford geeks congregated, he had wondered about the possibility of an NSA trap­door. This led him to a deeper consideration of the concept of trap­doors. Could an entire crypto scheme be built around one?

Designing such a system would present considerable challenges, because it would have to resolve a fundamental contradiction. A trap­door provides a means for those with proper knowledge to bypass security measures and get quick access to encrypted messages, some­thing that seems efficient. But the very thought of using a trapdoor in a security system seems like a nutty risk, precisely because crafty intruders might find a way to exploit it. It’s the same problem posed by a physical trapdoor: If your enemies can’t find it, you can use it to hide. But if they do, they’ll know exactly where to look for you.

This contradiction made the prospect of designing a trapdoor incredibly daunting. After all, the strongest crypto systems were finely tuned in every aspect to prevent their contents from leak­ing. Tampering with their innards to insert a backdoor—a leak!­—could easily produce any number of unintended weaknesses. When Diffie explained this to Hellman, both of them concluded that such a system would probably be impractical. But Diffie still thought it was interesting enough to add to a list he was compiling entitled “Problems for an Ambitious Theory of Cryptography.”

One day Diffie and Hellman brought in a Berkeley computer scien­tist named Peter Blatman to attend one of the informal seminars on crypto they had been convening on campus. Afterward, Blatman mentioned that a friend of his was working on an interesting problem: How can you get a secure conversation over an insecure line when the two people in the conversation have never had previous con­tact? Obviously, if the two people hadn’t known each other previously they would have had no opportunity to exchange secret keys before a private conversation.

This was, in effect, a different formulation of the big question that had been bugging Diffie for years: Was it possible to use cryptography to protect a huge network against eavesdroppers, and wiretappers to boot?

How could you create a system where people who had never met could speak securely? Where all conversations could be conducted with high-tech efficiency—but be protected by cryptography? Where you could get an electronic message from someone and be sure it came from the person whose return address appeared?

Also Read: How to Write an Effective Privacy Statement for Websites

During his quest, Diffie had struggled to gather information in an atmosphere where almost all of it was classified. And he had wound up with more than anyone could have expected: one-way functions. Password protections. Identification Friend or Foe. Trapdoors. Some­ where in all of that had to be an answer to privacy. Diffie knew that reconciling the different protections offered by these disparate systems was crucial to his quest.

One afternoon, things suddenly became clear to Diffie: Devise a system that could not only provide everything in Diffie’s recently envisioned one-way authentication scheme but could also deliver encryption and decryption in a novel manner. It would solve the untrustworthy administrator problem, and much, much more.

He would split the key.

Diffie’s breakthrough itself involved something that, in the context of the history of cryptography, seemed an absolute heresy: a public key. Until that point, there was a set of seemingly inviolable rules when it came to encryption, a virtual dogma that one ignored at the risk of consignment to crypto hell. One of those was that the same key that scrambled a message would also be the instrument that descrambled it. This is why keys were referred to as symmetrical.

That is why keeping those keys secret was so difficult: The very tools that eavesdroppers lusted after, the decryption keys, had to be passed from one person to another, and then exist in two places, dramatically increasing the chances of compromise. But Diffie, his brain infused with the informa­tion so painstakingly collected and considered over the past half decade, now envisioned the possibility for a different approach. Instead of using one single secret key, you could use a key pair. The tried-and­ true symmetrical key would be replaced by a dynamic duo. One would be able to do the job of scrambling a plaintext message—performing the task in such a way that outsiders couldn’t read it—but a secret trapdoor would be built into the message. The other portion of the key pair was like a latch that could spring open that trapdoor and let its holder read the message. And here was the beauty of the scheme: Yes, that second key—the one that flipped open the trapdoor—was of course something that had to be kept under wraps, safe from the pry­ing hands of potential eavesdroppers. But its mate, the key that actu­ally performed the encryption, didn’t have to be a secret at all. In fact, you wouldn’t want it to be a secret. You’d be happy to see it distributed far and wide.

Now, the idea of ensuring privacy by using keys that were exchanged totally in the open was completely nonintuitive, and on the face of it, bizarre. But using the mathematics of one-way functions, it could work. Diffie knew it, and for an illuminating instant, he knew how to do it using one-way functions.

It was the answer. From that moment, everything was different in the world of cryptography.

Also Read: Top 25 Data Protection Statistics That You Must Be Informed

First, by presenting an alternative to systems that worked with a single, symmetrical key, Diffie had solved a problem that had become so embedded in cryptographic systems that it had occurred to almost no one that it could be solved: the difficulty of distributing those secret keys to future recipients of secret messages. If you were a military organization, you might be able to protect the distribution centers that handled symmetrical keys (though God knows there were lapses even in the most vital operations). But if such centers moved into the private sector, and masses of people needed to use them, there would not only be inevitable bureaucratic pileups but also a constant threat of compromise. Figure it this way: If you needed to crack an encrypted message, wouldn’t the very existence of a place that stored all the secret keys present an opportunity for some creep to get the keys by theft, bribery, or some other form of coercion?

But with a public key system, every person could generate a unique key pair on his or her own, a pair consisting of a public key and a private key, and no outsider would have access to the secret key parts. Then private communication could begin.

Here’s how it would work: Say that Alice wants to communicate with Bob. Using Diffie’s concept, she needs only Bob’s public key. She could get this by asking him for it, or she might get it from some phone-book-type index of public keys. But it has to be Bob’s personal public key, a very long string of bits that could only have been gener­ated by only one person in the world… Bob. Then, by way of a one­-way function, she uses that public key to scramble the message in such a way that only the private key—the other half of that unique key pair—performs the decrypting calculation. (Thus the secret key is the “trapdoor” in the trapdoor one-way function Diffie was thinking about.)

So when Alice sends the scrambled message off, only one person in the world has the information necessary to reverse the calculation and decipher it: Bob, the holder of the private key. Say that the scrambled message gets intercepted by someone desperate to know what Alice had to say to Bob. Who cares? Unless the snooper has access to the unique partner of Bob’s public key—the instrument Alice used to con­vert the message to seeming mush—the snoop would get no more than that mush. Without that private key, reversing the mathematical encryption process is too damn difficult. Remember, going the wrong way in a one-way function is like trying to put together a pul­verized dinner plate.

Bob, of course, has no problem reading the message intended for his eyes only. He possesses the secret part of the key pair, and he can use that private key to decipher the message in a jiffy.

Also Read: 9 Policies For Security Procedures Examples

In short, Bob is able to read the message because he is the only per­son in possession of both sides of the key pair. Those who obtain the public key have no advantage in attempting to break the message. When it comes to encrypted messages, the only value of having Bob’s public key is to, in effect, change the message to Bob-speak, the language that only Bob can read (by virtue of having the secret half of the key pair).

This encryption function was only part of Diffie’s revolutionary concept, and not necessarily its most important feature. Public key crypto also provided the first effective means of truly authenticating the sender of an electronic message. As Diffie conceived it, the trap­door works in two directions. Yes, if a sender scrambles a message with someone’s public key, only the intended recipient can read it. But if the process is inverted—if someone scrambles some text with his or her own private key—the resulting ciphertext can be unscrambled only by using the single public key that matches its mate. What’s the point of that? Well, if you got such a message from someone claiming to be Albert Einstein, and wondered if it was really Albert Einstein, you now had a way to prove it—a mathematical litmus test. You’d look up Einstein’s public key and apply it to the scrambled ciphertext. If the result was plaintext and not gibberish, you’d know for certain that it was Einstein’s message—because he holds the world’s only private key that could produce a message that his matching public key could unscramble.

In other words, applying one’s secret key to a message is equivalent to signing your name: a digital signature. But unlike the sorts of sig­ natures that are penned on bank checks, divorce papers, and baseballs, a digital John Hancock cannot be forged by anyone with the minimal skills required to replicate the original signer’s lines and loops. Without a secret key, the would-be identity thief has scant hope of producing a counterfeit signature.

Nor could a would-be forger hope to monitor a phone line, wait until his prey’s digital signature appears, and then snatch it, with the intention of reusing the signature to create faked documents or to intercept future messages. In practice, a digital signature is not applied as an appendage to the document or letter to which it is affixed. Instead it is deeply interwoven with the digits that make up the actual content of the entire message. So if the document is intercepted, the eavesdropper cannot extract from it the tools to stamp the sender’s signature on some other document.

This technique also assures the authenticity of an entire document. A foe cannot hope to change a small but crucial portion of a digitally signed document (like switching the statement “I am not responsible for my spouse’s debts” to “I take full responsibility for my spouse’s debts,” all the while maintaining the signature of the unwitting sender). If the message was digitally signed with a private key but unencrypted, such a rogue could intercept it, use the sender’s well-distributed public key to descramble it, and then make the change in the plaintext. But what then? In order to resend the text with the proper signature, our forger would require the private key to fix the signature on the entire document. That secret key, of course, would be unobtainable, remaining in the sole possession of the original signer.

If someone sending a signed message wanted secrecy in addition to a signature, that’s easy, too. If Mark wanted to send an order to his banker, Lenore, he’d first sign the request with his private key, then encrypt that signed message with Lenore’s public key. Lenore would receive a twice-scrambled message: shaken for privacy, stirred for authentication. She would first apply her secret key, unlocking a mes­sage that no one’s eyes but hers could read. Then she would use Mark’s public key, unlocking a message that she now knows only he could have sent.

Digital signatures offer another advantage. Since it is impossible for a digitally signed message to be produced by anyone but the person who holds the private key that scrambles it, a signer cannot reasonably deny his or her role in producing the document. This nonrepudiation feature is the electronic equivalent of a notary public seal.

For the first time, it became possible to conceive of all sorts of offi­cial transactions—contracts, receipts, and the like—to be performed over computer networks, with no need for one’s physical presence.

In short, Diffie had not only figured out a way to assure privacy in an age of digital communications, but he had enabled an entirely new form of commerce, an electronic commerce that had the potential not only to match but to exceed the current protocols in commercial transactions. Even more impressive, his breakthrough had been performed completely outside the purview of government agencies in close possession of even the most trivial details of the most obscure crypto­graphic system.

Also Read: The Importance of Data Protection Management System (DPMS)