Goose the Blog 2.0

NSA killed system that sifted phone data legally
Sources say project was shelved in part because of bureaucratic infighting

WASHINGTON // The National Security Agency developed a pilot program in the late 1990s that would have enabled it to gather and analyze massive amounts of communications data without running afoul of privacy laws. But after the Sept. 11 attacks, it shelved the project -- not because it failed to work -- but because of bureaucratic infighting and a sudden White House expansion of the agency's surveillance powers, according to several intelligence officials.

The agency opted instead to adopt only one component of the program, which produced a far less capable and rigorous program. It remains the backbone of the NSA's warrantless surveillance efforts, tracking domestic and overseas communications from a vast databank of information, and monitoring selected calls.

...

In what intelligence experts describe as rigorous testing of ThinThread in 1998, the project succeeded at each task with high marks. For example, its ability to sort through massive amounts of data to find threat-related communications far surpassed the existing system, sources said. It also was able to rapidly separate and encrypt U.S.-related communications to ensure privacy.

But the NSA, then headed by Air Force Gen. Michael V. Hayden, opted against both of those tools, as well as the feature that monitored potential abuse of the records. Only the data analysis facet of the program survived and became the basis for the warrantless surveillance program.

The decision, which one official attributed to "turf protection and empire building," has undermined the agency's ability to zero in on potential threats, sources say. In the wake of revelations about the agency's wide gathering of U.S. phone records, they add, ThinThread could have provided a simple solution to privacy concerns.