OSINT: Where It Stops, Nobody Knows

I’m kind of busy wrapping and packing nondenominational-winter-holiday gifts, but here’s something I’ve been pondering lately: OSINT. Open-source intelligence. Most people agree it’s important (though many disagree on just how important), but few seem to agree on how to define it. This isn’t surprising; it’s hard to produce any kind of consensus definition of “intelligence”, in this context, after all. The thing is, while many people can agree on where OSINT starts, the point at which OSINT stops being OSINT is ill-defined and ill-discussed.

The ODNI hasn’t helped matters any, when he recently opined that OSINT sources and methods should not, themselves, be “open”. (The statement was something to the effect that the New York Times is open-source, and unclassified; but the fact that the government does or does not read the Times is actually classified.) It was an artful dodge of openness, but it, in my opinion, set the bar pretty damned low, and really doesn’t help define OSINT’s limits.

Remember, open-source intelligence has basically been around for decades, if not longer, but has only existed as a quasi-codified discipline for a decade or two, and has really only been taken seriously in the last few years. The – okay, a – problem seems to be that OSINT is most popular at the fringes of the intelligence field – law-enforcement intelligence, and the emerging business-intelligence (or “competitive intelligence”) communities, and it is here at the fringes that a lot of the – pardon the expression – cutting-edge work is being done. Yet, unfortunately, even as OSINT gets pushed and promoted by its users and advocates, the lines between it and other “INTs” get blurred.

Nobody seems to really care overly much where OSINT stops, but I do, and I think others should, as well.

Consider: Say you’re a junior intelligence analyst, and you’ve been tasked with producing an exhaustive open source summary of a new transnational terrorist group that’s recently appeared, and which has been issuing manifestos to sympathetic media outlets. You’re told to be as exhaustive as possible, without stepping outside the bounds, such as they are, of open-source intelligence work; your summary needs to be unclassified and entirely from open sources, because it may be shared with untrusted officials in foreign governments who could, theoretically, be sympathetic to the terrorists’ cause. With me so far? Good.

Now, you go and do Google searches, Lexis-Nexis searches, and all that other fun collection stuff. You gather media reports about the group, and download copies of their “press releases” and manifestos and whatever else you can get your hands on. From these, you can compile a well-sourced summary of the group’s public statements and reported activities, and there’s no real doubt that you’ve remained within the bounds of “open source” intelligence, the ODNI’s paranoia about OSINT “tradecraft” secrets notwithstanding.

But, you’re supposed to be exhaustive, right? OSINT isn’t – shouldn’t – be an excuse to do a half-assed job of things. So you dig into what little you’ve got, say. Suppose some of the group’s statements have been released on the web as MS Word or PDF files, and suppose those files contain metadata that indicates what software, operating system, and perhaps even computer hardware the author or authors used. Is that information useful and relevant? Very possibly. Is it, however, “open source”? Think carefully before answering!

Suppose those files contain an “author” field, which includes the name of a prominent member of a known terrorist group with similar aims to the one in question. Suppose, further, that the documents contain repeated misspellings or grammatical errors, which are consistent with – even identical to – those in statements known to have been written by that same terrorist from the other, older group. Is that observation “open source”, or have you “exceeded your scope” now? Are qualified inferences resulting from these observations still unclassified and “open source”? The information is there for anyone who cares to, to see, but does that automatically mean it’s “open”?

If you email this terrorist group (from a disposable Yahoo account, say), at their public contact address, they reply, and the headers of their email indicates the IP address from which the message was sent, is that “open source” intelligence collection? Anyone could get that information; it is theoretically available, in a sort of abstract sense, to anyone.

Obviously, logically, there has to be a limit where things stop being “open source”, but what the dividing line is, or how you define it, is a really murky area, at least to me. “Sources and methods” could – should? – be a criteria, but where do you draw the line? Part of the problem, it seems, is that OSINT, unlike the other “INTs”, deals less with “how” or “what” things are done than what they’re done to, and so can, at the edges, overlap with other intelligence disciplines. Is it a matter of detail, or methodology?

For me, at least, the answer to many of these questions is “I dunno”. My gut feeling is that the further you move towards analysis, the more of a grey area you’re in – there’s no doubt you can (and people do) produce (correctly-) classified judgments, inferences, and other sorts of analysis based purely on public, “open source” information. Yet, information is just data; it only becomes “intelligence” once it’s been analyzed, so OSINT by definition must involve analysis, right?

Thoughts, anyone? Or, better yet, lots and lots of grant money to study this issue in depth, anyone? 🙂

Published in: General, Security | on December 17th, 2008| 3 Comments »

Both comments and pings are currently closed.


  1. On 12/17/2008 at 5:21 pm Steven Aftergood Said:

    Whether a particular item of information is “open source” is a definitional question. Whether it is “classified” is a policy question. The latter seems more interesting and important to me.

    Generally speaking, I think that information that can be legally and openly collected (i.e. without resort to espionage or classified intelligence technologies) qualifies as open source.

    But some open source information is so obscure that it is effectively secret, even if it is not designated as such. Also, true OSINT is a discipline with its own skill sets — not everyone who reads a foreign newspaper online is an OSINT collector — and the analytical products derived from open source information, if they are any good, will not be obvious or self-evident.

    If those obscure bits of information and those analytical products have direct national security significance– then there could conceivably be a basis for classifying them. In many cases, though, — or in the overwhelming majority of cases — I believe that OSINT information and analysis serves a contextual function. Lacking immediate national security significance, it should not be classified. So it seems to me–

  2. On 12/17/2008 at 5:25 pm Rob van Stee Said:

    Just a minor comment: Wikipedia would consider these kind of observations to be “original research” (at least I’m pretty sure they would), you can read up on it. I suppose they have kind of a similar problem in what you can or cannot include or write.

    Very interesting blog you have here, I found it just a couple of weeks ago.

  3. On 12/17/2008 at 5:47 pm Nemo Said:

    Rob: Don’t get me started on Wikipedia’s “original research” policy. 🙂 I’d guess that 99% of intelligence analysis would fit Wikipedia’s definition of “original research”. I understand why they have the policies they do, but they’re really not applicable outside the Wikipedia environment (the Wikiverse?).

    Steven: The problem is that policy should, ideally, dictate definitions: if the head of the FBI, say, decides that green is now blue and white is now black, as far as the Bureau’s analysts are concerned, then that’s fine. With OSINT, though, the IC has been told “make use of this”, and “this is important”, but nobody has set out particularly clear guidelines on what “this” – OSINT – is. There is no consensus, yet; no real preconceived notions. If it’s a matter of semantics, then it’s because everyone is confused – and that, I’d argue, is a policy issue, at heart.

    I have this suspicion that politics is going to eventually mold OSINT to be more about the medium, and less about the message. Or maybe the other way around; the metaphor is far from perfect. 🙂