Over the past several months, the Obama administration has defended the government's far-reaching data collection efforts, arguing that only criminals and terrorists need worry. The nation's leading internet and telecommunications companies have said they are committed to the sanctity of their customers' privacy.  I have some very personal reasons to doubt those assurances.  In 2004, my telephone records as well as those of another New York Times reporter and two reporters from the Washington Post, were obtained by federal agents assigned to investigate a leak of classified information. What happened next says a lot about what happens when the government's privacy protections collide with the day-to-day realities of global surveillance.  The story begins in 2003 when I wrote an article about the killing of two American teachers in West Papua, a remote region of Indonesia where Freeport-McMoRan operates one of the world's largest copper and gold mines. The Indonesian government and Freeport blamed the killings on a separatist group, the Free Papua Movement, which had been fighting a low-level guerrilla war for several decades.  I opened my article with this sentence:  Bush administration officials have determined that Indonesian soldiers carried out a deadly ambush that killed two American teachers.  I also reported that two FBI agents had travelled to Indonesia to assist in the inquiry and quoted a "senior administration official" as saying there "was no question there was a military involvement".  The story prompted a leak investigation. The FBI sought to obtain my phone records and those of Jane Perlez, the Times bureau chief in Indonesia and my wife. They also went after the records of the Washington Post reporters in Indonesia who had published the first reports about the Indonesian government's involvement in the killings.  As part of its investigation, the FBI asked for help from what is described in a subsequent government report as an "on-site communications service" provider. The report, by the Department of Justice's inspector general, offers only the vaguest description of this key player, calling it "Company A". The report explained:  We do not identify the specific companies because the identities of the specific providers who were under contract with the FBI for specific services are classified.  Whoever they were, Company A had some impressive powers. Through some means – the report is silent on how – Company A obtained records of calls made on Indonesian cellphones and landlines by the Times and Post reporters. The records showed whom we called, when and for how long – what has now become famous as "metadata".  Under DOJ rules, the FBI investigators were required to ask the US attorney general to approve a grand jury subpoena before requesting records of reporters' calls. But that's not what happened.  Instead, the bureau sent Company A what is known as an "exigent letter'', asking for the metadata. A heavily redacted version of the DOJ report, released in 2010, noted that exigent letters are supposed to be used in extreme circumstances where there is no time to ask a judge to issue a subpoena. The report found nothing "exigent'' in an investigation of several three-year-old newspaper stories.  The need for an exigent letter suggests two things about Company A. First, that it was a US firm subject to US laws. Second, that it had come to possess my records through lawful means and needed legal justification to turn them over to the government.  The report disclosed that the agents' use of the exigent letter was choreographed by the company and the bureau. It said the FBI agent drafting the letter received "guidance" from "a Company A analyst". According to the report, lawyers for Company A and the bureau worked together to develop the approach.  Not surprisingly, "Company A" quickly responded to the letter it helped write. In fact, it was particularly generous, supplying the FBI with records covering a 22-month period, even though the bureau's investigation was limited to a seven-month period. Altogether, "Company A" gave the FBI metadata on 1,627 calls by me and the other reporters. Only three calls were within the seven-month window of phone conversations investigators had decided to review.  It doesn't end there. The DOJ report asserts that "the FBI made no investigative use of the reporters' telephone records." I don't believe that is accurate.  In 2007, I heard rumblings that the leak investigation was focusing on a diplomat named Steve Mull, who was the deputy chief of mission in Indonesia at the time of the killings. I had known Mull when he was a political officer in Poland and I was posted there in the early 1990s. He is a person of great integrity and a dedicated public servant.  The DOJ asked to interview me. Of course, I would not agree to help law enforcement officials identify my anonymous sources. But I was troubled because I felt an honorable public servant had been forced to spend money on lawyers to fend off a charge that was untrue. After considerable internal debate, I decided to talk to the DOJ for the limited purpose of clearing Mull.  It was not a decision I could make unilaterally. The Times also had a stake in this. If I allowed myself to be interviewed, how could the Times say no the next time the government wanted to question a Times reporter about a leak?  The Times lawyer handling this was George Freeman, a journalist's lawyer, a man Times reporters liked having in their corner. George and the DOJ lawyers began to negotiate over my interview. Eventually, we agreed that I would speak on two conditions: one, that they could not ask me for the name of my source; and two, if they asked me if it was "X", and I said no, they could not then start going through other names.  Freeman and I sat across a table from two DOJ lawyers. I'm a lawyer, and prided myself on being able to answer their questions with ease, never having to turn to Freeman for advice.  Until that is, one of the lawyers took a sheaf of papers that were just off to his right, and began asking me about phone calls I made to Mull. One call was for 19 minutes, the DOJ lawyer said, giving me the date and time. I asked for a break to consult with Freeman.  We came back, and answered questions about the phone calls. I said that I couldn't remember what these calls were about – it had been more than four years earlier – but that Mull had not given me any information about the killings. Per our agreement, the DOJ lawyers did not ask further questions about my sources, and the interview ended.  I didn't know how the DOJ had gotten my phone records, but assumed the Indonesian government had provided them. Then, about a year later, I received a letter from the FBI's general counsel, Valerie Caproni, who wrote that my phone records had been taken from "certain databases" under the authority of an "exigent letter'' (a term I had never heard).  Caproni sent similar letters to Perlez, to the Washington Post reporters, and to the executive editors of the Post and the Times, Leonard Downie and Bill Keller, respectively. In addition, FBI Director Robert Mueller called Downie and Keller, according to the report. Caproni wrote that the records had not been seen by anyone other than the agent requesting them and that they had been expunged from all databases.  I'm uneasy because the DOJ report makes clear that the FBI is still concealing some aspect of this incident. After describing Caproni's letters, the report says: "However, the FBI did not disclose to the reporters or their editors that [redacted]." The thick black lines obliterate what appear to be several sentences.  If you were to ask senior intelligence officials whether I should wonder about those deletions, they'd probably say no. I'm not so sure.  The government learned extensive details about my personal and professional life. Most of those calls were about other stories I was writing. Some were undoubtedly to arrange my golf game with the Australian ambassador. Is he now under suspicion? The report says the data has been destroyed and that only two analysts ever looked at it.  But who is this "Company A" that willingly co-operated with the government? Why was it working hand-in-glove with the FBI? And what did the FBI director not tell the editors of the New York Times and the Washington Post when he called them acknowledging the government had improperly obtained reporter's records?  • This article was originally published by ProPublica and is crossposted by permission of the editors
Raymond Bonner for ProPublica: As a reporter, I've had firsthand experience of how government agencies get phone company metadata without court warrants