Government Spying

I am often critical of the United States Government, both as a whole and specific portions of it. It’s not a lack of patriotism or hatred of my country. ‘Love it or leave it’ was an oft-parroted slogan not all that many months ago, and it is because I love, in my own idiosyncratic way, this country, or at least the country the USA could be, that I criticize the country and government it is today.

That being said, it’s often easy to forget that the government excels at certain things. That these things may not be activities of which I personally approve does not, as a rule, preclude a grudging admiration for a (perhaps detestable or unsavory) job done with promptness, skill, and efficiency.

On the afternoon of Tuesday, 5 December 2006, I read an essay about some outrageous, and quite possibly illegal, red tape, foot-dragging, and buck-passing on the part of the Freedom of Information and Privacy Acts (FOIPA) staff at the Department of Customs and Border Protection (CBP), a part of the much-maligned Department of Homeland Security (DHS). I felt this was interesting and worth sharing, and made a post about it, with a link to the original article, here on Entropic Memes.

I made that post, according to the webserver logs, at 13:18:24 CST – that’s 1:18pm Central time, 2:18pm Eastern. At 16:46:55, the post was retrieved by a computer (and, presumably, person) at CBP. It took the government an, in my opinion, quite respectable three hours and twenty-eight minutes, if I’m doing the math right, to find and view this public criticism of them.

But there’s more to this story than that.

How did someone at CBP find my post? According to the logs, it was emailed to them; the referrer link is unquestionably a webmail URL. This didn’t particularly surprise me – I’d be somewhat surprised if CBP themselves were doing any kind of proactive monitoring of the internet. But it made me curious to see if I could figure out just how the government found the post, and who passed it on to someone at CBP. Was it a person? Was it an automated program?

After going thru the logs line-by-line, I’m pretty confident I can answer that question, but that answer, in turn, just creates more questions.

Entropic Memes doesn’t get a lot of hits on a typical day, and 5 December 2006 was a pretty typical day. That made it comparatively easy to narrow down the potential sources.

In the three and a half hours between the post being made and the visit from someone at CBP, the contents of the post were accessed perhaps thirty times – both directly and via syndication feeds. Most of those hits can be presumed to be benign – Googlebot, for instance, Technorati’s spider, Livejournal fetching the feed for syndication, and so on. A couple visits are somewhat suspicious, and one stands out pretty sharply. Between 13:18 and 16:46, the page was visited by exactly four “real people” using “real browsers”. There were a number of other hits – but none of those requested the stylesheet or images, a curious behaviour strongly indicative of a computer “spider”.

One of those visits was the one from CBP, following a link from a webmail program. One was a Qwest DSL customer in my local area (207.224.93.nn), running Firefox, and I’m fairly sure I know that user personally. I could be wrong, and I certainly make no claims as to that individual’s trustworthyness, but… the third was a computer in Canada (159.18.53.4), belonging to what is apparently a large multinational (but Canadian-based) IT company called CGI Systems, who do three billion a year in business – much of it with the U.S. Government.

The fourth visit – at 13:58:17, just forty minutes (!) after the post was made, was from a computer on a network officially belonging to the State of New Jersey (160.93.145.174).

Coincidence? I’m skeptical. At first I believed they were behind a cacheing firewall-cum-proxy (or proxy-cum-firewall); their IP made subsequent requests for the page (but not the stylesheet or images) roughly thirty and then ninety minutes later. However, it received a “200” status, whereas a real cache should have been checking only if the page had updated, which would have returned a 304. So… something interesting is going on at that IP. Tracerouting the IP raises the question of whether the machine at that IP is actually in New Jersey at all…

If we treat the “New Jersey” visit as suspect, an immediate question of how they found the post comes to mind. Between the post and their visit, Technorati, Moreover.com, LiveJournal, and Google spidered the site’s feed, as did two mildly suspicious addresses - 66.199.250.210 (server.bidover.com), with an user-agent of “Java/1.5.0_09″ and a very mysterious 4.79.220.52, with the ubiquitous user-agent of “Wget/1.8.1”.

There was certainly other “suspicious” traffic – but “suspicious traffic” is like conspiracies; once you start looking for them, there’s a very real danger of making a mountain out of a molehill and seeing something where nothing exists. Why, for example, is bidover.com, an auction website registered in Massachussetts but using Brazilian nameservers, spidering my blog? Why is 4.79.220.52 repeatedly requesting the front page of my site? Damned if I know.

One question that definitely begs an answer is why someone would email a link to this article to someone at CBP in the first place. It’s not like my hyperbole is particularly vitriolic, or even witty. It’s just meant to point people to the Buggrit article, which is a little too low-tech to get indexed by blog search engines. I was able to contact the author of the article I linked to, and he confirmed that the page I linked to received no visits from either the New Jersey or Customs and Border Protection IP addresses I provided.

For all of the addresses that visited the page of the CBP post during the time in question, none viewed other pages of the site. Does this suggest an automated program, using something like Technorati, to identify posts of “potential interest”, which are then being double-checked by warm bodies? Perhaps warm bodies that get their lists of URLs via email? If we assume yes, do we also assume that multiple bits of the government are performing the exact same tasks, independent of each other? I could easily believe it, but I just don’t know. Government employees messin’ with bloggers’ heads? Covert death squads receiving targeting instructions?

We’ll probably never know.

I am, naturally, interested in hearing any thoughts anyone might have to make on this; post a comment if you’ve got something to say.

Bookmark to: