7/31 NSA XKeyscore collects 'nearly everything user does on the internet'

see the guardian site for entire article, photos, graphs, etc
NSA XKeyscore program collects 'nearly everything user does on the internet'
7/31/13 Glenn Greenwald, http://www.theguardian.com/world/2013/jul/31/nsa-top-secret-program-onli... (edited)
• XKeyscore 'widest-reaching' sweep of online data • NSA analysts require no prior search authorisation • Collects IPs, emails, social media activity, searches and browsing history

A top secret National Security Agency program allows analysts to search with no prior authorization through vast databases containing emails, online chats and the browsing histories of millions of individuals, according to documents provided by whistleblower Edward Snowden. The NSA boasts in training materials that the program called XKeyscore, is its "widest-reaching" system for developing intelligence from the internet. These latest revelations will add to the intense debate around NSA surveillance programs. They come as senior intelligence officials testify to the Senate judiciary committee Wednesday, releasing classified documents in response to the Guardian's earlier stories on bulk collection of phone records and Fisa surveillance court oversight.

A December 2012 document slide, "plug-ins", describes searchable fields of information, includes "every email address seen in a session by both username and domain", "every phone number seen in a session (address book entries or signature block)" and user activity--"webmail and chat activity to include username, buddylist, machine specific cookies etc"....

Training materials for XKeyscore detail how analysts can use it and other systems to mine enormous agency databases by filling in a simple on-screen form giving only a broad justification for the search. The request is not reviewed by a court or any NSA personnel before it is processed. XKeyscore, the documents boast, is NSA's "widest reaching" system developing intelligence from computer networks – what the agency calls Digital Network Intelligence (DNI). One presentation claims XKeyscore covers "nearly everything a typical user does on the internet", including the content of emails, websites visited and searches, as well as their metadata. XKeyscore and other NSA systems analysts can obtain ongoing "real-time" interception of an individual's internet activity.

Under US law, the NSA is required to obtain an individualized Fisa warrant only if the target of their surveillance is a 'US person', but no such warrant is required for intercepting communications of Americans with foreign targets. XKeyscore provides the technological capability, if not the legal authority, to target US persons for extensive electronic surveillance without a warrant provided that some identifying information, such as their email or IP address, is known to the analyst.

One training slide illustrates the digital activity constantly being collected by XKeyscore and the analyst's ability to query the databases at any time. The purpose of XKeyscore is to allow analysts to search the metadata and the content even when there is no known email account ("selector") associated with the individual targeted.
Analysts can also search by name, telephone number, IP address, keywords, the language in which the internet activity was conducted or the type of browser used.
One document notes this is because "strong selection [search by email address] itself gives us only a very limited capability" because "a large amount of time spent on the web is performing anonymous actions."... Analysts are... advised to use the metadata also stored in the databases to narrow down what to review...

One top-secret document describes how XKeyscore "searches within bodies of emails, webpages and documents", including "To, From, CC, BCC lines" and website 'Contact Us' pages.
To search for emails, an analyst using XKS enters the individual's email address into a simple online search form, along with the "justification" for the search and the time period for which the emails are sought. The analyst then selects which of those returned emails they want to read by opening them in NSA reading software.
The system is similar to how NSA analysts generally can intercept the communications of anyone they select, including, as one NSA document put it, "communications that transit the United States and communications that terminate in the United States".

A top secret 2010 guide describing NSA analyst training for general surveillance under the 2008 Fisa Amendments Act, explains that analysts can begin surveillance on anyone by clicking a few simple pull-down menus designed to provide both legal and targeting justifications. Once options on the pull-down menus are selected, their target is marked for electronic surveillance and the analyst is able to review the content of their communications:

Beyond emails, the XKeyscore system allows analysts to monitor a virtually unlimited array of other internet activities, including social media. A NSA tool called DNI Presenter, used to read the content of stored emails, also enables an analyst using XKeyscore to read the content of Facebook chats or private messages. An analyst can monitor such chats by entering the Facebook user name and a date range into a simple search screen.

Analysts can search for internet browsing activities using a wide range of information, including search terms entered by the user or the websites viewed. As one slide indicates, the ability to search HTTP activity by keyword permits the analyst access to what the NSA calls "nearly everything a typical user does on the internet".

The XKeyscore program also allows an analyst to learn the IP addresses of every person who visits any website the analyst specifies.

The quantity of communications accessible through programs such as XKeyscore is staggeringly large. One NSA report from 2007 estimated that there were 850bn "call events" collected and stored in the NSA databases, and close to 150bn internet records. Each day, the document says, 1-2bn records were added.

William Binney, former NSA mathematician, said last year NSA had "assembled on the order of 20trillion transactions about US citizens with other US citizens", an estimate, he said, that "only was involving phone calls and emails". A 2010 Washington Post article reported that "every day, collection systems at the [NSA] intercept and store 1.7bn emails, phone calls and other types of communications."

The XKeyscore system continuously collects so much internet data it can be stored only for short periods of time...content for three to five days, metadata for 30 days. One document explains: "At some sites, the amount of data we receive per day (20+ terabytes) can only be stored for 24 hours." To solve this problem, NSA created a multi-tiered system that allows analysts to store "interesting" content in other databases, such as one named Pinwale which can store material for up to five years. (ed: and building massive new sites]
XKeyscore databases contain the greatest amount of communications data collected by NSA as one document shows. In 2012, there were at least 41 billion total records collected and stored in XKeyscore for a single 30-day period....
The ACLU deputy legal director, Jameel Jaffer, told the Guardian last month that national security officials expressly said a primary purpose of the new law was to enable them to collect large amounts of American' communications without individualized warrants. "The government doesn't need to 'target' Americans in order to collect huge volumes of their communications. The government inevitably sweeps up the communications of many Americans" when targeting foreign nationals for surveillance.An example is provided by one XKeyscore document showing an NSA target in Tehran communicating with people in Frankfurt, Amsterdam and New York Jaffer said.

In recent years, the NSA has attempted to segregate exclusively domestic US communications in separate databases. But NSA documents acknowledge such efforts are imperfect, as purely domestic communications can travel on foreign systems, and NSA tools are sometimes unable to identify the national origins of communications. Moreover, all communications between Americans and someone on foreign soil are included in the same databases as foreign-to-foreign communications, making them readily searchable without warrants.

Series: Glenn Greenwald on security and liberty
Previous | Next | Index

NSA Prism program taps in to user data of Apple, Google and others
6/13 http://www.theguardian.com/world/2013/jun/06/us-tech-giants-nsa-data