James Bamford
Author, The Puzzle Palace: Inside the National Security Agency, America's Most Secret Intelligence Organization, and Body of Secrets: Anatomy of the Ultra-Secret National Security Agency

How did the NSA get started? What was its antecedent? What is its history?

The earliest form of the NSA was an organization known as the Black Chamber, which was created around 1920. It was a civilian organization designed to intercept communications and break codes, and it had a secret agreement with telecommunications companies.

From that small start, which was basically a few dozen people, it's grown into the largest intelligence agency in the world. It's a mammoth organization that occupies virtually its own city halfway between Washington and Baltimore. It's an extremely secret organization. For example, there's lots of people from the CIA who have left and written books about the CIA, former directors and all kinds of people. There's never been a single book ever written by a former NSA employee.

It was born in absolute secrecy by a memorandum signed by President Truman, as opposed to the CIA, which came through a public law. ... It was a totally black organization. In other words, virtually nobody outside of a few people in the executive branch was supposed to know about it -- certainly nobody in the public -- and even in Congress there were really only one or two people that had any idea that this agency was created. It was kept that way for about 10 years, and then information began leaking out because of a few defectors that ended up working their way to Moscow. So the secret began getting out.

But still, for most of half a century very few people knew almost anything about NSA. The old joke was it stood for "No Such Agency," and to the people inside, the joke was it stood for "Never Say Anything." I think that's held true through right up until the present, basically. ...

You said it's the largest intelligence agency in the world. How big is it?

NSA is a very interesting agency because it's not only a civilian agency; it's also a military organization, so the director of NSA wears two hats. ... He has his own military basically, his own Army, Navy and Air Force, that are used to actually collect the signals: planes that fly along borders or fly over borders and pick up signals, eavesdrop on communications; ships that sail along coasts and pick up communications. The NSA has people that they put on ships and military listening posts all over the world that are occupied and run a lot of times by the military. So it's a very, very large organization. It was up near 100,000 people during the Vietnam days, counting the military and the civilian. Now it's probably around 30,000.

Biggest intelligence budget? Bigger than the CIA?

In terms of budget it's certainly bigger than the CIA. It's hard to rate its budget because the budget of, I think, the National Reconnaissance Office [NRO] is slightly larger, but that's partly because what they're doing is putting up satellites for NSA. So if you amortize the cost of NSA satellites, it's certainly by far the most expensive intelligence agency in the country.

What is SIGINT? How does the NSA use SIGINT?

SIGINT is one of the big INTs, and INTs are what the intelligence community does. The CIA does HUMINT, which is human intelligence. It's mostly getting people to work for them overseas; getting agents, as they call them, to work for them overseas, to steal documents, pass information. There's another INT, and that's IMINT, imagery intelligence, which is taking pictures of things on the ground and so forth.

But the largest of the INTs and the one that is far more useful to most people in the intelligence community and the government is SIGINT, which is signals intelligence, and that's NSA's specialty. What it does is eavesdrop on communications. Communications travel by way of signals: microwave signals, high-frequency signals, fiber-optic signals. ... Some of those signals contain words and phrases and human communications. Other signals contain data, just reams and reams of data. And some signals just contain sort of beacon information; in other words, radar-type information. So NSA is interested in any type of signal that's out there.

So NSA would be bugging the communications of the Soviet Union during the Cold War, listening in on the telemetry of Soviet space shots, that kind of thing?

It did all that, yeah. During the Cold War the Soviet Union mostly communicated, at least through government means, by high-frequency communication, a fairly rudimentary signal being bounced up and down in between the earth and the ionosphere. In order to capture that, the NSA built these huge listening posts all around the Soviet Union with these giant, what they called "elephant cage" antennas. They look like enormous cages for elephants. They were like 100 feet tall and maybe 1,000 feet wide, and they were circular, so the idea would be to capture those signals from any direction and then be able to know exactly what direction those signals came from. ...

Then, after the Cold War, NSA shifted to focusing on a far more difficult task. ... After the Cold War they had to get into the same system that all the rest of the world uses, the regular old telecommunications system. ... So you have millions and millions of communications, and you had to pick out the exact communications you wanted. It was very difficult, because they didn't know where they were coming from, when they were coming or what they would necessarily look like. ...

... What's their method of operation? If they don't know which message is going to be the good one, how do they find it?

One of the things that NSA does is that it hires a lot of people from the telecommunications industry, people who know how the Internet works, know how they built certain systems within the Internet and so forth. For example, they'll hire somebody from Cisco who built various routers, and they'll get them into NSA, and they'll have that person do reverse engineering; in other words, take the router and show them exactly how it was put together, and you start finding the vulnerabilities.

People talk about the NSA having the fastest supercomputers in the world and having the largest concentration of top mathematicians in the world. What is that about?

... You're dealing with fairly sophisticated communications, particularly government communications; a lot of that is encrypted. It's put in the form of a code. These are very sophisticated systems that are developed by computers, and the only way to break those codes or those encryption systems is to have faster and more powerful computers than the computers that put them together in the first place. So NSA has the largest collection of supercomputers in the world. It has one whole building, several floors -- it's called the Tordella Supercomputer Building -- where all they have are supercomputers, the fastest and most powerful computers in the world. Plus, they have parallel processing there, which ties large numbers of computers together and makes them seem like one very giant computer.

So NSA uses all this computer power for a number of reasons. One of them is to try to break these codes. The hardest way is what's known as brute force, which is when you take all that computer power and put it on one problem and use all that force to try to break that system using every possible word/letter/phrase combination you could possibly use, that kind of thing.

What they look for most often are breaks in the system, what they call busts, and that's where somebody does something wrong; somebody makes a mistake; somebody is about to type a password, and they think they're typing it into their secure system, and they accidentally type their password into an open system. That's a backdoor way to get into the system. ...

Does NSA use supercomputers to do data mining?

Yeah. Among the uses for those supercomputers, one is code breaking. The other one is to sift through vast data, vast amounts of data that come in. What they're looking for mostly are numbers. If they actually have a phone number of somebody that was a member of Al Qaeda or whatever, they put the phone number in there, and then anytime that number goes through a system that they're eavesdropping on, whether it's a satellite system or a fiber-optic system or whatever, anytime that number goes through, an alert will go off in an office in NSA. One of the intercept operators will hear an alert, and then they'll start listening to the conversation, or they'll record it or automatically be recorded. That's one way. That's a very specific way. ...

But then there's a lot more general searches, so, for example, the country code for Afghanistan, they may try to do everything within that whole country code. Or they might narrow it down to the country code and the city code; the city code for Kabul, for example. So it's all based on numbers, largely in terms of what they're targeting through this vast flow of international communications.

Although it's not perfect, they do have an ability to look for words in communications, but there's a large error rate when you're looking for words.

You mean in the content, the message?

Yeah, in voice communications, for example. If you're intercepting voice communications and you're trying to pick out individual words, it's a very, very difficult process, because you say "house" one way and I say "house" another way, and computers have a difficult time picking those out unless a computer has been trained into a person's voice. ...

It's much easier, though, if you're going to do a data search on data communications, because if you write "house," you're going to write it the same way I write "house," so it's much easier to pick up those words. ...

Before 9/11 is the NSA doing much inside America?

They're doing almost nothing inside the United States. I interviewed the director ... in January of 2001, and I think he told me that there were about half a dozen people in the U.S. that they were focusing on. ...

NSA was looked at more as the nuclear weapon for eavesdropping, much too powerful to use domestically. It was never set up to use domestically. That's what the FBI was for. It's like you don't use the Army domestically, either. ... NSA was designed to eavesdrop overseas, and that's what it did for most of its existence. It was eavesdropping on international communications and overseas communications. ...

So pre-9/11, the NSA is focused overseas, doing almost nothing domestically, but using supercomputers to sift through enormous volumes of communications to find the ones it wants to focus on.

That's right. You can't get the communications that you want unless you pull it all in, so even though they were only targeting a handful of people in the United States prior to 9/11, they still were actually putting the entire spectrum of communications, satellite communications and so forth, through their filter in order to find those few that they were looking for. They were looking for other communications that they could target that they didn't need to get any warrants for, like Soviet-to-Soviet communications and so forth. ...

What's the historical relationship between the NSA and the telephone companies? Is it close?

Well, it's very close. For almost 100 years NSA and its predecessors have had an extremely close relationship with the telecommunications companies, and during most of that time, there's been illegal agreements between the telecommunications company and NSA, where the companies would actually pass information on to NSA very secretly and largely illegally.

What kind of information?

Well, whatever is going through their system in terms of communications, going back to the '20s, '30s and '40s. So Western Union would pass all its telegrams to NSA very secretly.

You mean telegrams from Americans to Americans? From Americans overseas?

All telegrams. ... During World War I there was censorship during the war. After the war the predecessor to NSA, then known as the Black Chamber, needed to get the information, so the head of the Black Chamber went to the individual telecommunications companies and made secret deals, and he got all the telegrams that would come into the country, go out of the country or go through the country.

On Americans, you're talking about.

Yeah. Whatever they wanted they were given, whether it was Americans or not; the companies didn't care. But this wasn't easy. The companies did not just roll over. They knew this was illegal, so they gave the predecessor to NSA a hard time, but eventually they gave in.

After World War II it was the same thing. The war ended, censorship ended, and all of a sudden the predecessor to NSA at the time didn't have any access to the communications coming in or going out of the country. So once again the director of the agency had to go around secretly to the heads of all the different companies and say: "Will you please cooperate? There will only be a handful of people in your company that will know about it. We will not tell anybody, and I can speak for the attorney general and the president that you won't be prosecuted, at least during this administration." That's basically what they said. ...

And the agreement was?

The way it worked was that the NSA would set up a small little phony television tape-processing company in New York City, and one agent from the NSA would go to each of the companies after midnight and knock on a back door, and the companies would pass to them -- at first they were the hard telegrams. After computers came around they'd pass them the computer-tape journals of all the telegrams. They'd pass it to them through the back door, and he would take it to this phony television tape-processing company, put it through high-speed duplication machines and duplicate all these tape journals of all these telegrams, and then get the original data back to the companies through the back door before the shift went off in the morning.

Then they would take the duplicates and they would transfer them to NSA. They were always afraid of a plane crash, so they would put them with couriers on a train, and then they would take them to NSA headquarters at Fort Meade, [Md.]. Then that day they would take the tapes containing all these telegrams, and they'd run them through these very fast computers. At the time it was known as the harvest computer, and that's exactly what they would do. They would harvest whatever information they were looking for, whether it was an American name or a phrase or a word or an address or whatever they were looking for. ...

It was called Operation Shamrock, and again, it went on from 1945 until it was discovered by Sen. [Frank] Church's investigating committee [the Senate Select Committee to Study Governmental Operations with Respect to Intelligence Activities] in 1975. That's why we have the Foreign Intelligence Surveillance Act [FISA], to prevent this from happening again.

It wasn't just telegrams; it was also telephone calls. There was another operation known as Operation Minaret where they did similar activity with telephone communications within the United States.

So this was done voluntarily by the phone companies, or were there warrants?

No. This was a very, very secret agreement between very high-level people in the government -- secretary of defense, the head of the predecessor of NSA, other people like that. Gen. Eisenhower, before he became president, even got involved. But it was a very, very secret agreement between just maybe three or four people within each company -- the president, the chairman of the board, the general counsel and one or two people who actually passed the stuff out the back door. ...

What was the response? ... Was the NSA badly burned by this whole episode?

Well, this was the most traumatic period in NSA's history by far. Very few people know the extent of the investigation that went on. You have not only the Church Committee that looked into it; you had the Rockefeller Commission [the U.S. President's Commission on CIA Activities Within the United States, headed by Vice President Nelson Rockefeller] that was set up by President Ford to also look into it.

One of the results of that was an investigation by the Justice Department. The Justice Department actually did a criminal investigation of NSA. They actually looked at NSA, the entire agency, as a potential criminal entity, and they went there, and they read the Miranda rights to senior officials. They eventually came up, after about a year, with I think there was around 21 different areas where they could possibly prosecute NSA for doing all this illegal activity. They eventually decided not to take this to court, not to have any prosecution, because any kind of court activity would necessarily involve release of enormous amounts of secret information, and they didn't want to do that. So the bottom line was they basically put this criminal case in a file and locked the door and didn't do anything.

What they did at the end was recommend that, instead of pursuing this criminal case which would involve releasing too many secrets, that Congress look into changing the law to make this illegal. And again, that was another impetus for creation of the Foreign Intelligence Surveillance Act.

... What kind of policies did NSA institute after that to ensure they didn't violate the law again?

... I got the distinct impression, having spent a lot of time at NSA over many years, that they were very compliant with law. ... Those people that I dealt with were there during the Church Committee, and they did not want to go through that again. They did not want to go through the public exposure. They didn't want to go through the potential criminal jeopardy. They didn't want to go through any of that again. So they would tell me that the DEA, for example, the Drug Enforcement Administration, would push them to go close to the edge to get this communications or whatever, and they would always push back, because they didn't want to get to the edge.

So the attitude before 9/11 is: Don't touch Americans. We've got this very narrow group for which you get a warrant.

That's right. They had enough to do with focusing on the rest of the world. They left the U.S. to the FBI. Both for legal reasons and just because of practicality, they wanted to put the bulk of their resources overseas, not domestically.

Kevin O'Connell
Consultant, Innovative Analytics and Training; former NSA employee

What was NSA like in the 1980s as a place to work?

Well, it was really a mystique. I remember in those days, the metric that the college recruiters would use is to come to college campuses very, very quietly and actually ask you if you like to do crossword puzzles. That was the way you were supposed to be evaluated in the very first instance on whether you belonged at NSA or not.

It was a very quiet place. It was a place that was steeped in the secrecy of the Cold War. Getting in was seen as very much a badge of honor and something that was very exciting. More importantly, it was very much a time when whatever the work flow was, you were privileged at the time to be exposed to some of the best technology in the world, or, in those days, 10, if not 15 years ahead of anything that was available in the commercial market in the communications arena. ...

What was the mission of NSA when you joined it?

The mission of NSA was twofold: It was to conduct signals intelligence abroad to understand both policy issues, threats, etc., but also a defensive mission that was designed to understand what the threats were to communications, not only within the government, but also more broadly for the nation. It had sort of the offensive and the defensive mission at the same time. ...

What was the ethic or the culture about dealing with the communications of Americans?

You ask a very interesting question, because it is absolutely the cardinal rule that I remember from my earliest days at NSA. It was almost something that was put into your bloodline from the very beginning. In essence, you were taught that it was not part of the NSA mission except under very exceptional and legally approved circumstances to be involved with U.S. communications. So in the day, if you came across, whether by stumbling across or by some other way, what you believed to be a U.S. communication, you immediately had to drop. You had to get away from it. It was something that every analyst knew very much in their bloodline.

You touched the third rail.

Exactly.

It was hot-button.

Exactly.

Get out of there.

Get out of there, and even under those very narrow circumstances where you had approval to do that, there were very special handling measures for the data that were not to be disseminated.

The example I love to tell is that today we would use in an instant a Google search engine, for example, to go in and type in the name of someone. In those days it was strictly forbidden, even under threat perhaps of prosecution, to type in the name of a friend or person you knew was a U.S. citizen. And NSA in those days made sure everyone knew that they were not to do that under any circumstances; that in fact if they did do it, playfully or otherwise, their name could potentially be reported to the Department of Justice for violation of law. It was a very, very, very serious matter. ...

... What are the basic kinds of data gathering and analysis that the NSA did as a routine matter?

Initially you are collecting volumes of data that may be geographically oriented volumes of data. They may have to do with a specific group that you're interested in, and in the process you've got to decide what kinds of analytic attacks that you put that data set through. Obviously some of them are very clear. Encryption would be one, trying to understand how to crack the codes of that data set.

Another one would be the language. Obviously all the data in the world does you no good unless you actually have some ability to understand the language really that you are looking for within that data set. Over time techniques have been developed to understand patterns within the data. You can look at trends within the data; you can look at what had been known in the day and I think is even known today as something called traffic analysis, understanding different kinds of patterns within the data. When you have that kind of a large data set you have to be analytically very, very aggressive.

So patterns means who is communicating with whom, how much volume there is. What does traffic analysis mean?

It could be all of those things. It could be volumes of traffic between parties. It could be kinds of information that pass between parties. It can be different parties that are linked up in communication with one another.

Do computers have the power to spot the information, [who it's] going from and to, or find key words in messages?

It's a combination of computers and human beings. It is really not something that is only done by computers. Computers can help a lot, but at the end of the day, at least with my familiarity with it, it involves somebody who is a human being who had familiarity with different targets, a terrorist group, a specific group that they were interested in, etc. ...

... In a world where pre-emption has become the name of the game, does that make an agency like the National Security Agency, with its computer searching and its capacity for data-sifting analysis, a major player?

It is certainly still a major player, but ... that kind of a national strategy would place a premium on intelligence. ... SIGINT would certainly be key to that. ...

This sounds like a tremendous change. When you joined the NSA in 1982 it's totally focused outward; it's looking at the Soviet Union and proxy wars around the world. Now we are talking about it being very much involved domestically as well as overseas.

I'd put a different nuance on it, which is that, in the early days, NSA's ability to provide information perhaps of a pre-emptive nature against Soviet ICBMs [intercontinental ballistic missiles] was as important as [counterterrorism is to] the world we live in today.

On the other hand, while NSA's mission today remains overwhelmingly foreign in nature, what we are learning as a nation is how it will contribute or not to the domestic intelligence mission. That said, for NSA to do its job effectively it is going to have to be able to look through large amounts of data that transit the globe, whether they are in the United States or outside the United States.

Bill Weaver
Associate professor of law, University of Texas, El Paso; senior adviser to the National Security Whistleblowers Coalition [NSWBC]; former Army intelligence officer subcontracted to the NSA.

To what degree did data mining become increasingly the method of the NSA in the years of the Cold War and even after the Cold War ended?

It's a function of technology. When you have the computing power, then data mining will arise automatically, and the computing power is staggering at the National Security Agency. The average person doesn't have a concept of the massive capability that is available to the National Security Agency. It's by far ahead of anything that people can imagine. So once you have that kind of technical capacity where there really is no upper limit on data collection and subjecting that data collection to analysis, then you have to do that, almost as an intelligence instrument. It's so inviting it's almost impossible to ignore.

Is that the mind-set of people in the NSA?

It is a tool. They obviously do many other things -- cryptographic work, content analysis, busting certain signals, these sorts of things -- but yes, it is a mind-set, to collect everything.

Collect everything, sift it, sort it, make sense out of it.

Make sense out of it, yes. ...

Can you give us any idea of what the scale of the NSA's supercomputing operation is, what they're capable of handling?

The NSA doesn't measure computers in computing power; it measures them in acres. That's how they talk about their computers: how many acres of computers they have. ... We're talking about millions of processors that can work on a single problem simultaneously. The amount of computing power is phenomenal. It's just staggering. ...

The important point here is that what we're talking about, especially with data mining, is not a distributed computer network, where you have lots of different computers working on things. Data mining would require that there be a database that would be centralized ... [in which data would] probably be collected from lots of other databases. I call this database OBAD: One Big-Ass Database. ... The NSA is really the only facility that could subject that database to the kind of algorithmic massage that would be necessary to come to conclusions about the threat levels, for example, every American represented. ...

Shortly after 9/11, when the president says to [former NSA director] Gen. [Michael] Hayden, "What more could you do to help us prevent the second wave of attacks here in the United States?," what does that mean that NSA can do?

NSA historically has been prohibited from collecting information on U.S. citizens. There's something called United States Signal Intelligence Directive [USSID] 18 which prohibits the interception of U.S. citizens or targeting U.S. citizens by the National Security Agency. I think that's out the window. ... So number one is the field of NSA's targets expanded.

Number two is you have a shift, if you will, from targeting for intelligence purposes to law enforcement purposes, and that's an enormous shift. That's a shift in mentality, culture, the application of resources. That is a huge shift that has occurred. ...

... [If] NSA turns from its traditional function of intelligence gathering and analysis overseas and turns it on the United States, what kind of operation does it do?

First of all, forget about the idea of the guy with the earphones on, listening to something. That's not what happens. Calls are collected; communications are collected in an automated fashion by the millions. ... What it means is they're collecting everything about everybody. It is that simple.

What programs is the NSA doing under this warrantless wiretapping the president has authorized?

You have these two programs. One program is the one that's been acknowledged: They use point-to-point communications of known people, known targets. The second program is this data mining, data-analysis program, which is collecting information about everybody from everywhere and subjecting that information to very complex and sophisticated algorithms that only the NSA can do because of their computing power, and then coming to a conclusion, based on patterns, of whether or not people represent a threat to the United States.

Now, what represents a threat to the United States is something that they decide. An algorithm can be changed. For example, we have 760 or 800 federal district court judges in this country. Is it a threat to the national security that a federal district court judge is engaging in an extramarital affair? Well, it may be, because that may make that federal district court judge, in the eyes of the federal government, subject to blackmail. So all you have to do is tweak the algorithm, and then you look at the 760 or 800 federal district court judges, their buying patterns, what they do, and you find the pattern that says this person might be having an affair. What then? So they're subjecting everybody they can collect data on to these very sophisticated algorithms to develop a pattern of threat, a pattern of activity that would yield information and intelligence that they can use. ...