FILE PHOTO: The National Security Agency (NSA) headquarters is seen in Fort Meade, Maryland, U.S. February 14, 2018. REUTERS/Sait Serkan Gurbuz
The U.S. National Security Agency collected 534 million records of phone calls and text messages of Americans last year, more than triple gathered in 2016, a U.S. intelligence agency report released on Friday said.
The sharp increase from 151 million occurred during the second full year of a new surveillance system established at the spy agency after U.S. lawmakers passed a law in 2015 that sought to limit its ability to collect such records in bulk.
The spike in collection of call records coincided with an increase reported on Friday across other surveillance methods, raising questions from some privacy advocates who are concerned about potential government overreach and intrusion into the lives of U.S. citizens.
The 2017 call records tally remained far less than an estimated billions of records collected per day under the NSA’s old bulk surveillance system, which was exposed by former U.S. intelligence contractor Edward Snowden in 2013.
The records collected by the NSA include the numbers and time of a call or text message, but not their content.
Overall increases in surveillance hauls were both mystifying and alarming coming years after Snowden’s leaks, privacy advocates said.
“The intelligence community’s transparency has yet to extend to explaining dramatic increases in their collection,” said Robyn Greene, policy counsel at the Washington-based Open Technology Institute that focuses on digital issues …
Friday’s report also showed a rise in the number of foreigners living outside the United States who were targeted under a warrantless internet surveillance program, known as Section 702 of the Foreign Intelligence Surveillance Act, that Congress renewed earlier this year.
That figure increased to 129,080 in 2017 from 106,469 in 2016, the report said, and is up from 89,138 targets in 2013, or a cumulative rise over five years of about 45 percent.
U.S. intelligence agencies consider Section 702 a vital tool to protect national security, but privacy advocates say the program incidentally collects an unknown number of communications belonging to Americans.
Homeland Security Secretary Kirstjen Nielsen speaks during the press briefing at the White House in Washington, DC, on April 4, 2018. (NICHOLAS KAMM/AFP/Getty Images)
The details of the attached Request for Information outline a plan to gather and monitor the public activities of media professionals and influencers and are enough to cause nightmares of constitutional proportions, particularly as the freedom of the press is under attack worldwide.
And “attack” is not hyperbolic.
Every day, journalists face serious consequences including physical violence, imprisonment and death. A few days ago, the Committee to Protect Journalists launched its annual Free The Press campaign to raise awareness about imprisoned journalists throughout the world. On May 3, UNESCO will once again mark World Press Freedom Day “to inform citizens of violations of press freedom — a reminder that in dozens of countries around the world, publications are censored, fined, suspended and closed down, while journalists, editors and publishers are harassed, attacked, detained and even murdered.”
Meanwhile, the United States government, traditionally one of the bastions of press freedom, is about to compile a list of professional journalists and “top media influencers”, which would seem to include bloggers and podcasters, and monitor what they’re putting out to the public.
What could possibly go wrong? A lot.
DHS’ “Media Monitoring” Plan
As part of its “media monitoring”, the DHS seeks to track more than 290,000 global news sources as well as social media in over 100 languages, including Arabic, Chinese and Russian, for instant translation into English. The successful contracting company will have “24/7 access to a password protected, media influencer database, including journalists, editors, correspondents, social media influencers, bloggers etc.” in order to “identify any and all media coverage related to the Department of Homeland Security or a particular event.”
“Any and all media coverage”, as you might imagine, is quite broad and includes “online, print, broadcast, cable, radio, trade and industry publications, local sources, national/international outlets, traditional news sources, and social media.”
The database will be browseable by “location, beat and type of influencer”, and for each influencer, the chosen contractor should “present contact details and any other information that could be relevant, including publications this influencer writes for, and an overview of the previous coverage published by the media influencer.”
One aspect of the media coverage to be gathered is its “sentiment.”
Why “Media Monitoring” and Why Now?
DHS says the “NPPD/OUS [National Protection and Programs Directorate/Office of the Under Secretary] has a critical need to incorporate these functions into their programs in order to better reach Federal, state, local, tribal and private partners.” Who knows what that means, but the document also states the NPPD’s mission is “to protect and enhance the resilience of the nation’s physical and cyberinfrastructure.”
That line makes it sound as if the creation of this database could be a direct response to the rampant allegations of Russian interference in the 2016 U.S. presidential election — though President Donald Trump, who has normalized the term “fake news”, can’t seem to decide whether that’s even an issue or not.
Facebook CEO Mark Zuckerberg thinks it is. Earlier this week, he announced the social networking site would remove “more than 270 pages and accounts operated by a Russian organization called the Internet Research Agency” in an effort “to protect the integrity of elections around the world”.
Within the context of increasing concerns over “fake news” and foreign interference in elections, an action such as the DHS’ database might seem, at first glance, to be a sensible approach.
Not exactly.
Unfortunately, increasing government encroachment on the freedom of the press is the sinister backdrop to all of this. Freedom House, which has monitored the status of the press for nearly 40 years, recently concluded that global media freedom has reached its lowest level in the past 13 years. The independent watchdog organization blames “new threats to journalists and media outlets in major democracies” as well as “further crackdowns on independent media in authoritarian countries like Russia and China.” And then it goes one step further.
“But it is the far-reaching attacks on the news media and their place in a democratic society by Donald Trump, first as a candidate and now as president of the United States, that fuel predictions of further setbacks in the years to come”, the report said.
Could the DHS media database be such a setback?
Possibly, and it’s not even the first time potential regulation of journalists has drifted across the American political scene.
Last October, an Indiana lawmaker proposed that journalists be licensed. Representative Jim Lucas’ bill was mostly a publicity stunt, but could this DHS action be a way for the government to keep track of American and foreign journalists as well as “citizen journalists”, threatening not only the freedom of the press but also individual freedom of speech?
The real question, of course, is what the government plans to do with the information it compiles, and there’s been no comment on that beyond what is in the posting, which, by the way, has interest from at least seven companies. Will those on the DHS media database be questioned more harshly coming in and out of the country? Will they have trouble getting visas to go to certain countries for their own reporting or personal vacations? Worse?
Speaking of visas — and showing that social media activity is squarely on the radar of this Administration — earlier this week, the State Department placed two notices in the Federal Register seeking comments on its proposal to require that all visa applicants to the U.S. turn over their social media information for the previous five years.
Regarding the DHS media database, we are entering potentially dangerous territory with the government keeping track of the “sentiment” of citizens and foreign nationals. If not legal challenges from organizations that defend press freedom and freedom of speech interests, the government should expect, at the very least, backlash from the public.
And that means you. If you think the idea of the U.S. government’s compiling and monitoring a list of media professionals and “top media influencers” is a potential threat to democracy, now would be the perfect time to call your local and congressional representatives to let them know how much you value a free press and the freedom of speech, just in case they’ve forgotten.
Legitimate downloads of popular software including WhatsApp, Skype and VLC Player are allegedly being hacked at an internet service provider (ISP) level to spread an advanced form of surveillance software known as “FinFisher”, cybersecurity researchers warn.
FinFisher is sold to global governments and intelligence agencies and can be used to snoop on webcam feeds, keystrokes, microphones and web browsing. Documents, previously published by WikiLeaks, indicate that one tool called “FinFly ISP” may be linked to the case.
The digital surveillance tools are peddled by an international firm called Gamma Group and have in the past been sold to repressive regimes including Bahrain, Egypt and the United Arab Emirates (UAE).
In March this year, the company attended a security conference sponsored by the UK Home Office.
This week (21 September), experts from cybersecurity firm Eset claimed that new FinFisher variants had been discovered in seven countries, two of which were being targeted by “man in the middle” (MitM) attacks at an ISPlevel – packaging real downloads with spyware.
Companies hit included WhatsApp, Skype, Avast, VLCPlayer and WinRAR, it said, adding that “virtually any application could be misused in this way.”
When a target of surveillance was downloading the software, they would be silently redirected to a version infected with FinFisher, research found.
When downloaded, the software would install as normal – but Eset found it would also be covertly bundled with the surveillance tool.
The stealthy infection process was described as being “invisible to the naked eye.”
The seven countries were not named for security reasons, Eset said. WhatsApp and VLC Player did not respond to request for comment by the time of publication.
A Microsoft spokesperson, referencing the Skype infections, told IBTimes UK: “Windows Defender antivirus cloud protection already automatically identifies and blocks the malware.
“For non-cloud customers, we’ve deployed signatures to protect against this in our free antivirus software”, the statement added.
An Avast spokesperson said: “Attackers will always focus on the most prominent targets.
“Wrapping official installers of legitimate apps with malware is not a new concept and we aren’t surprised to see the PC apps mentioned in this report.”
“What’s new is that this seems to be happening at a higher level.”
“We don’t know if the ISPs are in cooperation with the malware distributors or whether the ISPs’ infrastructure has been hijacked.”
The latest version of FinFisher was spotted with new customized code which kept it from being discovered, what Eset described as “tactical improvements”. Some tricks, it added, were aimed at compromising end-to-end (E2E) encryption software and known privacy tools.
One such application was Threema, a secure messaging service.
“The geographical dispersion of Eset’s detections of FinFisher variants suggests the MitM attack is happening at a higher level—an ISP arises as the most probable option”, the team said.
“One of the main implications of the discovery is that they decided to use the most effective infection method and that it actually isn’t hard to implement from a technical perspective”, FilipKafka, a malware researcher at Eset, told IBTimes UK.
“Since we see have seen more infections than in the past surveillance campaigns, it seems that FinFisher is now more widely utilized in the monitoring of citizens in the affected countries.”
Breaking encryption has become a major talking point of governments around the world, many of which conduct bulk communications collection. Politicians argue, often without evidence, that software from companies such as WhatsApp has become a burden on terror probes.
WhatsApp, Skype and VLC all targeted by FinFisher spyware. Image credits (L/R): iStock, Reuters, Windows Phone Store
The software’s brochure boasted: “FinFly ISP is able to patch files that are downloaded by the target on-the-fly or send fake software updates for popular software.”
It added that it “can be installed on an internet service provider’s network” and listed one use case when it was previously deployed by an unnamed intelligence agency.
Eset found that all affected targets within one of the countries were using the same ISP.
“Unprecedented”
“The deployment of the ISP-level MitM attack technique mentioned in the leaked documents has never been revealed – until now”, the researchers said in their analysis.
“If confirmed, these FinFisher campaigns would represent a sophisticated and stealthy surveillance project unprecedented in its combination of methods and reach.”
It remains unknown who was behind the fresh hacking campaigns, but FinFisher is almost exclusively tailored to government, police or intelligence agency use.
“We cannot say for sure who is behind the campaign but the ISP re-direction could be a service ordered from FinFisher”, Kafka said.
“This question should be addressed to FinFisher.”
“We [have] very limited information on this, who specifically was targeted, but generally the targets were catered to what FinFisher is generally used for”, he added.
Gamma Group did not immediately respond to a request for comment from IBTimes UK.
The variant was spotted in 7 countriesMarkusSpiske/Unsplash
This is not the first time that the company, which has offices in Europe, has been linked to questionable business practices.
In 2013, tech firm Mozilla sent it a cease and desist letter after its software was caught posing as a version of its Firefox browser.
“We cannot abide a software company using our name to disguise online surveillance tools that can be – and in several cases actually have been – used by Gamma’s customers to violate citizens’ human rights and online privacy”, it complained in a blog post.
The same year, Reporters without Borders branded Gamma Group as one of the “Corporate Enemies of the Internet” in an annual report. The creepy and invasive spyware can also be spread via more traditional means – malicious email attachments, for example.
Back in 2011, it emerged that Gamma International, a UK subsidiary, was selling a malware Trojan disguised as an update for Apple’s iTunes media player.
Before being patched, the gaping vulnerability had been exploited for approximately three years, found security journalist Brian Krebs at the time.
Facebook wants to get up close and personal with its users after a patent was revealed detailing a desire to secretly watch users through their webcam or smartphone camera, spying on your mood in order to sell you tailored content or advertisements.
The purpose behind the invasive idea is to analyze people through the camera in real time while they browse online and if it recognizes you looking happy, bored or sad, it would deliver an advert fitting your emotion. If you were forlorn, for example, it would be able to serve an ad to perk you up, or know what products you had previously looked at online and put them under your nose at just the right time.
Facebook explains in the patent application that a user who looked away during certain content (in their fictional case it was a kitten video) algorithms for the social network would know to not show more of that type of content. In another example it describes how the technology could tell if a user’s expression changed while looking at posts or pictures from a certain person and would show more or less of these in the future.
The social network has filed several patents over the years on emotion-based technology but this, based on ‘passive imaging data’ is perhaps the most unnerving, considering it would take control of cameras that weren’t even switched on by the user.
As described by CB Insights: “This patent proposes capturing images of the user through smartphone or laptop cameras, even when the user is not actively using the camera. By visually tracking a user’s facial expression, Facebook aims to monitor the user’s emotional reactions to different types of content.”
How the ’emotion-based technology’ patent works.
The New York-based intelligence firm went on to say: “On the one hand, they want to identify which content is most engaging and respond to audience’s reactions, on the other emotion-detection is technically difficult, not to mention a PR and ethical minefield.”
Other patents listed by Facebook include a text messaging platform to detect a user’s mood by measuring how hard and fast they were typing, then augment the message format, such as adding emojis or changing the font size, to match their emotion.
The patent for taking control of the camera of a user’s device was granted back in 2015 but there has been no introduction of the technology in the wild. Facebook, however, will always have to notify members in advance of any changes. Yet, this would likely be a hard sell.
A Facebook spokesperson provided IBTimesUK with the following statement: “We often seek patents for technology we never implement, and patents should not be taken as an indication of future plans.”
With the danger of online privacy edging its way to the foreground of public awareness many would no doubt be wary about giving away such intimate access. After all, even Facebook CEO Mark Zuckerberg is alert to the dangers of being spied on after a picture he posted online showed his laptop’s webcam and microphone port taped over.
“They spent a fortune tracking 26 people and recording three million conversations and apparently got nothing … I’d love to see the probable cause affidavit for that one and wonder what the court thought on its 10 day reviews when zip came in … I’m not surprised by the results because on average, a very very low percentage of conversations are incriminating, and a very very low percent results in conviction”. When reached, a spokesperson for the Justice Department did not comment.
’The #MainstreamPress says that the #NSA has “ended” its bulk phone records collection program. Does that mean we can all relax and forget about #MassSurveillance?
“FOR ALMOST FOUR years, a cottage industry of media conspiracists has devoted itself to accusing Edward Snowden of being a spy for either Russia and/or China at the time he took and then leaked documents from the National Security Agency. There has never been any evidence presented to substantiate this accusation…
…Newly obtained documents conclusively prove that the central tale invented by these Snowden-accusing commentators is a wholesale fabrication. These documents negate the edifice on which this entire fiction has been based from the start…”
“Many #Windows 10 users are unknowingly sending the contents of every keystroke they make to Microsoft due to an enabled-by-default keylogger. This function has been around since the beginning of Windows 10… If this was ever on while you used Windows 10, there’s no way for you to know that Microsoft has deleted your information…”
The TSA is introducing “more rigorous” and “comprehensive” physical inspections at airports around the country, according to Bloomberg. The security agency, which until now had the option of using five different types of physical pat-downs in the screening line, is eliminating the “options” and replacing them with a single, universal method which would involve heavier groping.
The Transportation Security Administration made the announcement to its agents this week, and in the case of Denver International Airport employees, advised employees and flight crews on Thursday that the “more rigorous” searches “will be more thorough and may involve an officer making more intimate contact than before.”
“Two U.S. Marshals, heavily armed and dressed in dystopian-style black regalia, stood next to an upright machine with a glowing green eye. Every passenger, one by one, was told to step on a mat and look into the green scanner. It was scanning our eyes and matching that scan with the passport …
It’s one thing to control who comes into a country. But surveilling and permissioning American citizens as they leave their own country, even as they are about to board, is something else.”
Documents provided by NSA whistleblower Edward Snowden reveal Palantir’s role in creating the U.S. government’s international spy machine.
Photo: Kristoffer Tripplaar/Sipa USA/AP
The Intercept can now reveal that Palantir has worked for years to boost the global dragnet of the NSA and its international partners, and was in fact co-created with American spies …
Palantir has never masked its ambitions, in particular the desire to sell its services to the U.S. government — the CIA itself was an early investor in the startup through In-Q-Tel, the agency’s venture capital branch. But Palantir refuses to discuss or even name its government clientele, despite landing “at least $1.2 billion” in federal contracts since 2009, according to an August 2016 report in Politico. The company was last valued at $20 billion and is expected to pursue an IPO in the near future. In a 2012 interview with TechCrunch, while boasting of ties to the intelligence community, Karp said nondisclosure contracts prevent him from speaking about Palantir’s government work.
Photo: Alex Karp, co-founder and CEO of Palantir Technologies
Patrick T. Fallon/Bloomberg/Getty Images
“Palantir” is generally used interchangeably to refer to both Thiel and Karp’s company and the software that company creates. Its two main products are Palantir Gotham and Palantir Metropolis, more geeky winks from a company whose Tolkien namesake is a type of magical sphere used by the evil lord Sauron to surveil, trick, and threaten his enemies across Middle Earth. While Palantir Metropolis is pegged to quantitative analysis for Wall Street banks and hedge funds, Gotham (formerly Palantir Government) is designed for the needs of intelligence, law enforcement, and homeland security customers. Gotham works by importing large reams of “structured” data (like spreadsheets) and “unstructured” data (like images) into one centralized database, where all of the information can be visualized and analyzed in one workspace. For example, a 2010 demo showed how Palantir Government could be used to chart the flow of weapons throughout the Middle East by importing disparate data sources like equipment lot numbers, manufacturer data, and the locations of Hezbollah training camps. Palantir’s chief appeal is that it’s not designed to do any single thing in particular, but is flexible and powerful enough to accommodate the requirements of any organization that needs to process large amounts of both personal and abstract data.
A Palantir promotional video.
Despite all the grandstanding about lucrative, shadowy government contracts, co-founder Karp does not shy away from taking a stand in the debate over government surveillance. In a Forbes profile in 2013, he played privacy lamb, saying, “I didn’t sign up for the government to know when I smoke a joint or have an affair. … We have to find places that we protect away from government so that we can all be the unique and interesting and, in my case, somewhat deviant people we’d like to be.” In that same article, Thiel lays out Palantir’s mission with privacy in mind: to “reduce terrorism while preserving civil liberties.” After the first wave of revelations spurred by the whistleblower Edward Snowden, Palantir was quick to deny that it had any connection to the NSA spy program known as PRISM, which shared an unfortunate code name with one of its own software products. The current iteration of Palantir’s website includes an entire section dedicated to “Privacy & Civil Liberties,” proclaiming the company’s support of both:
Palantir Technologies is a mission-driven company, and a core component of that mission is protecting our fundamental rights to privacy and civil liberties. …
Some argue that society must “balance” freedom and safety, and that in order to better protect ourselves from those who would do us harm, we have to give up some of our liberties. We believe that this is a false choice in many areas. Particularly in the world of data analysis, liberty does not have to be sacrificed to enhance security. Palantir is constantly looking for ways to protect privacy and individual liberty through its technology while enabling the powerful analysis necessary to generate the actionable intelligence that our law enforcement and intelligence agencies need to fulfill their missions.
It’s hard to square this purported commitment to privacy with proof, garnered from documents provided by Edward Snowden, that Palantir has helped expand and accelerate the NSA’s global spy network, which is jointly administered with allied foreign agencies around the world. Notably, the partnership has included building software specifically to facilitate, augment, and accelerate the use of XKEYSCORE, one of the most expansive and potentially intrusive tools in the NSA’s arsenal. According to Snowden documents published by The Guardian in 2013, XKEYSCORE is by the NSA’s own admission its “widest reaching” program, capturing “nearly everything a typical user does on the internet.” A subsequent report by The Intercept showed that XKEYSCORE’s “collected communications not only include emails, chats, and web-browsing traffic, but also pictures, documents, voice calls, webcam photos, web searches, advertising analytics traffic, social media traffic, botnet traffic, logged keystrokes, computer network exploitation targeting, intercepted username and password pairs, file uploads to online services, Skype sessions, and more.” For the NSA and its global partners, XKEYSCORE makes all of this as searchable as a hotel reservation site.
But how do you make so much data comprehensible for human spies? As the additional documents published with this article demonstrate, Palantir sold its services to make one of the most powerful surveillance systems ever devised even more powerful, bringing clarity and slick visuals to an ocean of surveillance data.
PALANTIR’S RELATIONSHIP WITH government spy agencies appears to date back to at least 2008, when representatives from the U.K.’s signals intelligence agency, Government Communications Headquarters, joined their American peers at VisWeek, an annual data visualization and computing conference organized by the Institute of Electrical and Electronics Engineers and the U.S. National Institute of Standards and Technology. Attendees from throughout government and academia gather to network with members of the private sector at the event, where they compete in teams to solve hypothetical data-based puzzles as part of the Visual Analytics Science and Technology (VAST) Challenge. As described in a document saved by GCHQ, Palantir fielded a team in 2008 and tackled one such scenario using its own software. It was a powerful marketing opportunity at a conference filled with potential buyers.
In the demo, Palantir engineers showed how their software could be used to identify Wikipedia users who belonged to a fictional radical religious sect and graph their social relationships. In Palantir’s pitch, its approach to the VAST Challenge involved using software to enable “many analysts working together [to] truly leverage their collective mind.” The fake scenario’s target, a cartoonishly sinister religious sect called “the Paraiso Movement,” was suspected of a terrorist bombing, but the unmentioned and obvious subtext of the experiment was the fact that such techniques could be applied to de-anonymize and track members of any political or ideological group. Among a litany of other conclusions, Palantir determined the group was prone to violence because its “Manifesto’s intellectual influences include ‘Pancho Villa, Che Guevara, Leon Trotsky, [and] Cuban revolutionary Jose Martí,’ a list of military commanders and revolutionaries with a history of violent actions.”
The delegation from GCHQ returned from VisWeek excited and impressed. In a classified report from those who attended, Palantir’s potential for aiding the spy agency was described in breathless terms. “Palantir are a relatively new Silicon Valley startup who are sponsored by the CIA,” the report began. “They claim to have significant involvement with the US intelligence community, although none yet at NSA.” GCHQ noted that Palantir “has been developed closely internally with intelligence community users (unspecified, but likely to be the CIA given the funding).” The report described Palantir’s demo as “so significant” that it warranted its own entry in GCHQ’s classified internal wiki, calling the software “extremely sophisticated and mature. … We were very impressed. You need to see it to believe it.”
The report conceded, however, that “it would take an enormous effort for an in-house developed GCHQ system to get to the same level of sophistication” as Palantir. The GCHQ briefers also expressed hesitation over the price tag, noting that “adoption would have [a] huge monetary … cost,” and over the implications of essentially outsourcing intelligence analysis software to the private sector, thus making the agency “utterly dependent on a commercial product.” Finally, the report added that “it is possible there may be concerns over security — the company have published a lot of information on their website about how their product is used in intelligence analysis, some of which we feel very uncomfortable about.”
A page from Palantir’s “Executive Summary” document, provided to government clients.
However anxious British intelligence was about Palantir’s self-promotion, the worry must not have lasted very long. Within two years, documents show that at least three members of the “Five Eyes” spy alliance between the United States, the U.K., Australia, New Zealand, and Canada were employing Palantir to help gather and process data from around the world. Palantir excels at making connections between enormous, separate databases, pulling big buckets of information (call records, IP addresses, financial transactions, names, conversations, travel records) into one centralized heap and visualizing them coherently, thus solving one of the persistent problems of modern intelligence gathering: data overload.
A GCHQ wiki page titled “Visualisation,” outlining different ways “to provide insight into some set of data,” puts succinctly Palantir’s intelligence value:
Palantir is an information management platform for analysis developed by Palantir Technologies. It integrates structured and unstructured data, provides search and discovery capabilities, knowledge management, and collaborative features. The goal is to offer the infrastructure, or ‘full stack,’ that intelligence organizations require for analysis.
Bullet-pointed features of note included a “Graph View,” “Timelining capabilities,” and “Geo View.”
A GCHQ diagram indicates how Palantir could be used as part of a computer network attack.
Under the Five Eyes arrangement, member countries collect and pool enormous streams of data and metadata collected through tools like XKEYSCORE, amounting to tens of billions of records. The alliance is constantly devising (or attempting) new, experimental methods of prying data out of closed and private sources, including by hacking into computers and networks in non-Five Eyes countries and infecting them with malware.
A 2011 PowerPoint presentation from GCHQ’s Network Defence Intelligence & Security Team (NDIST) — which, as The Intercept has previously reported, “worked to subvert anti-virus and other security software in order to track users and infiltrate networks” — mentioned Palantir as a tool for processing data gathered in the course of its malware-oriented work. Palantir’s software was described as an “analyst workspace [for] pulling together disparate information and displaying it in novel ways,” and was used closely in conjunction with other intelligence software tools, like the NSA’s notorious XKEYSCORE search system. The novel ways of using Palantir for spying seemed open-ended, even imaginative: A 2010 presentation on the joint NSA-GCHQ “Mastering the Internet” surveillance program mentioned the prospect of running Palantir software on “Android handsets” as part of a SIGINT-based “augmented reality” experience. It’s unclear what exactly this means or could even look like.
Above all, these documents depict Palantir’s software as a sort of consolidating agent, allowing Five Eyes analysts to make sense of tremendous amounts of data that might have been otherwise unintelligible or highly time-consuming to digest. In a 2011 presentation to the NSA, classified top secret, an NDIST operative noted the “good collection” of personal data among the Five Eyes alliance but lamented the “poor analytics,” and described the attempt to find new tools for SIGINT analysis, in which it “conducted a review of 14 different systems that might work.” The review considered services from Lockheed Martin and Detica (a subsidiary of BAE Systems) but decided on the up-and-comer from Palo Alto.
Palantir is described as having been funded not only by In-Q-Tel, the CIA’s venture capital branch, but furthermore created “through [an] iterative collaboration between Palantir computer scientists and analysts from various intelligence agencies over the course of nearly three years.” While it’s long been known that Palantir got on its feet with the intelligence community’s money, it has not been previously reported that the intelligence community actually helped build the software. The continuous praise seen in these documents shows that the collaboration paid off. Under the new “Palantir Model,” “data can come from anywhere” and can be “asked whatever the analyst wants.”
Along with Palantir’s ability to pull in “direct XKS Results,” the presentation boasted that the software was already connected to 10 other secret Five Eyes and GCHQ programs and was highly popular among analysts. It even offered testimonials (TWO FACE appears to be a code name for the implementation of Palantir):
[Palantir] is the best tool I have ever worked with. It’s intuitive, i.e. idiot-proof, and can do a lot you never even dreamt of doing.
This morning, using TWO FACE rather than XKS to review the activity of the last 3 days. It reduced the initial analysis time by at least 50%.
Enthusiasm runs throughout the PowerPoint: A slide titled “Unexpected Benefits” reads like a marketing brochure, exclaiming that Palantir “interacts with anything!” including Google Earth, and “You can even use it on a iphone or laptop.” The next slide, on “Potential Downsides,” is really more praise in disguise: Palantir “Looks expensive” but “isn’t as expensive as expected.” The answer to “What can’t it do?” is revealing: “However we ask, Palantir answer,” indicating that the collaboration between spies and startup didn’t end with Palantir’s CIA-funded origins, but that the company was willing to create new features for the intelligence community by request.
On GCHQ’s internal wiki page for TWO FACE, analysts were offered a “how to” guide for incorporating Palantir into their daily routine, covering introductory topics like “How do I … Get Data from XKS in Palantir,” “How do I … Run a bulk search,” and “How do I … Run bulk operations over my objects in Palantir.”For anyone in need of a hand, “training is currently offered as 1-2-1 desk based training with a Palantir trainer. This gives you the opportunity to quickly apply Palantir to your current work task.” Palantir often sends “forward deployed engineers,” or FDEs, to work alongside clients at their offices and provide assistance and engineering services, though the typical client does not have access to the world’s largest troves of personal information. For analysts interested in tinkering with Palantir, there was even a dedicated instant message chat room open to anyone for “informally” discussing the software.
The GCHQ wiki includes links to classified webpages describing Palantir’s use by the Australian Defence Signals Directorate (now called the Australian Signals Directorate) and to a Palantir entry on the NSA’s internal “Intellipedia,” though The Intercept does not have access to copies of the linked sites. However, embedded within Intellipedia HTML files available to The Intercept are references to a variety of NSA-Palantir programs, including “Palantir Classification Helper,” “[Target Knowledge Base] to Palantir PXML,” and “PalantirAuthService.” (Internal Palantir documents obtainedby TechCrunch in 2013 provide additional confirmation of the NSA’s relationship with the company.)
One Palantir program used by GCHQ, a software plug-in named “Kite,” was preserved almost in its entirety among documents provided to The Intercept. An analysis of Kite’s source code shows just how much flexibility the company afforded Five Eyes spies. Developers and analysts could ingest data locally using either Palantir’s “Workspace” application or Kite. When they were satisfied the process was working properly, they could push it into a Palantir data repository where other Workspace users could also access it, almost akin to a Google Spreadsheets collaboration. When analysts were at their Palantir workstation, they could perform simple imports of static data, but when they wanted to perform more complicated tasks like import databases or set up recurring automatic imports, they turned to Kite.
Kite worked by importing intelligence data and converting it into an XML file that could be loaded into a Palantir data repository. Out of the box, Kite was able to handle a variety of types of data (including dates, images, geolocations, etc.), but GCHQ was free to extend it by writing custom fields for complicated types of data the agency might need to analyze. The import tools were designed to handle a variety of use cases, including static data sets, databases that were updated frequently, and data stores controlled by third parties to which GCHQ was able to gain access.
This collaborative environment also produced a piece of software called “XKEYSCORE Helper,” a tool programmed with Palantir (and thoroughly stamped with its logo) that allowed analysts to essentially import data from the NSA’s pipeline, investigate and visualize it through Palantir, and then presumably pass it to fellow analysts or Five Eyes intelligence partners. One of XKEYSCORE’s only apparent failings is that it’s so incredibly powerful, so effective at vacuuming personal metadata from the entire internet, that the volume of information it extracts can be overwhelming. Imagine trying to search your Gmail account, only the results are pulled from every Gmail inbox in the world.
MAKING XKEYSCORE MORE intelligible — and thus much more effective — appears to have been one of Palantir’s chief successes. The helper tool, documented in a GCHQ PDF guide, provided a means of transferring data captured by the NSA’s XKEYSCORE directly into Palantir, where presumably it would be far easier to analyze for, say, specific people and places. An analyst using XKEYSCORE could pull every IP address in Moscow and Tehran that visited a given website or made a Skype call at 14:15 Eastern Time, for example, and then import the resulting data setinto Palantir in order to identify additional connections between the addresses or plot their positions using Google Earth.
Palantir was also used as part of a GCHQ project code-named LOVELY HORSE, which sought to improve the agency’s ability to collect so-called open source intelligence — data available on the public internet, like tweets, blog posts, and news articles. Given the “unstructured” nature of this kind of data, Palantir was cited as “an enrichment to existing [LOVELY HORSE] investigations … the content should then be viewable in a human readable format within Palantir.”
Palantir’s impressive data-mining abilities are well-documented, but so too is the potential for misuse. Palantir software is designed to make it easy to sift through piles of information that would be completely inscrutable to a human alone, but the human driving the computer is still responsible for making judgments, good or bad.
A 2011 document by GCHQ’s SIGINT Development Steering Group, a staff committee dedicated to implementing new spy methods, listed some of these worries. In a table listing “risks & challenges,” the SDSG expressed a “concern that [Palantir] gives the analyst greater potential for going down too many analytical paths which could distract from the intelligence requirement.” What it could mean for analysts to distract themselves by going down extraneous “paths” while browsing the world’s most advanced spy machine is left unsaid. But Palantir’s data-mining abilities were such that the SDSG wondered if its spies should be blocked from having full access right off the bat and suggested configuring Palantir software so that parts would “unlock … based on analysts skill level, hiding buttons and features until needed and capable of utilising.” If Palantir succeeded in fixing the intelligence problem of being overwhelmed with data, it may have created a problem of over-analysis — the company’s software offers such a multitude of ways to visualize and explore massive data sets that analysts could get lost in the funhouse of infographics, rather than simply being overwhelmed by the scale of their task.
If Palantir’s potential for misuse occurred to the company’s spy clients, surely it must have occurred to Palantir itself, especially given the company’s aforementioned “commitment” to privacy and civil liberties. Sure enough, in 2012 the company announced the formation of the Palantir Council of Advisors on Privacy and Civil Liberties, a committee of academics and consultants with expertise in those fields. Palantir claimed that convening the PCAP had “provided us with invaluable guidance as we try to responsibly navigate the often ill-defined legal, political, technological, and ethical frameworks that sometimes govern the various activities of our customers,” and continued to discuss the privacy and civil liberties “implications of product developments and to suggest potential ways to mitigate any negative effects.” Still, Palantir made clear that the “PCAP is advisory only — any decisions that we make after consulting with the PCAP are entirely our own.”
What would a privacy-minded conversation about privacy-breaching software look like? How had a privacy and civil liberties council navigated the fact that Palantir’s clientele had directly engaged in one of the greatest privacy and civil liberties breaches of all time? It’s hard to find an answer.
Palantir wrote thatit structured the nondisclosure agreement signed by PCAP members so that they “will be free to discuss anything that they learn in working with us unless we clearly designate information as proprietary or otherwise confidential (something that we have rarely found necessary except on very limited occasions).” But despite this assurance of transparency, all but one of the PCAP’s former and current members either did not return a request for comment for this article or declined to comment citing the NDA.
The former PCAP member who did respond, Stanford privacy scholar Omer Tene, told The Intercept that he was unaware of “any specific relationship, agreement, or project that you’re referring to,” and said he was not permitted to answer whether Palantir’s work with the intelligence community was ever a source of tension with the PCAP. He declined to comment on either the NSA or GCHQ specifically. “In general,” Tene said, “the role of the PCAP was to hear about client engagement or new products and offerings that the company was about to launch, and to opine as to the way they should be set up or delivered in order to minimize privacy and civil liberties concerns.” But without any further detail, it’s unclear whether the PCAP was ever briefed on the company’s work for spy agencies, or whether such work was a matter of debate.
There’s little detail to be found on archived versions of Palantir’s privacy and civil liberties-focused blog, which appears to have been deleted sometime after the PCAP was formed. Palantir spokesperson Matt Long told The Intercept to contact the Palantir media team for questions regarding the vanished blog at the same email address used to reach Long in the first place. Palantir did not respond to additional repeated requests for comment and clarification.
A GCHQ spokesperson provided a boilerplate statement reiterating the agency’s “longstanding policy” against commenting on intelligence matters and asserted that all its activities are “carried out in accordance with a strict legal and policy framework.” The NSA did not provide a response.
Anyone worried that the most powerful spy agencies on Earth might use Palantir software to violate the privacy or civil rights of the vast number of people under constant surveillance may derive some cold comfort in a portion of the user agreement language Palantir provided for the Kite plug-in, which stipulates that the user will not violate “any applicable law” or the privacy or the rights “of any third party.” The world will just have to hope Palantir’s most powerful customers follow the rules.
“Several human rights organizations will soon launch a campaign urging President Obama to pardon NSA whistleblower Edward Snowden, coinciding with the premiere of the Oliver Stone biopic based on his life…”
