The Good and Bad of OSINT

I’ve written about some of the benefits of open-source intelligence, a/k/a OSINT, in the past, though it’s been a while. In recent weeks, though, I’ve been reminded of two important things about this particular intelligence discipline that are worth thinking about.

One increasing value of OSINT products is that they can generally be (more) freely shared and distributed with partners and interested parties who don’t – necessarily – have the appropriate security clearance to see “the good stuff”. (That last bit is sarcasm, by the way.) It’s not so much that OSINT is necessarily any less valuable, complete, or accurate, but that with open sources, you don’t have to protect sources and methods as is the case with other types of intelligence.

Why does this matter? In part, because it allows you to cite your sources, which adds credibility to your finished product. Consider:

“Multiple confidential human sources within the Rock Ave Rumblers report the gang’s membership at about 45 people.”

That’s great, and very credible… but almost certainly classified. How do you dumb that down so you’re not compromising your sources?

“There is strong reason to believe the Rock Ave Rumblers numbers about 45 members.”

Yay, protecting sources and methods. However, it’s always a good idea to cite sources, when you can, because it lends credibility to your statements. “Reason to believe” could be anything, up to and including really wishful thinking, right? Thus, you turn to open sources:

“A December 18, 2008 article in the Dead Tree Gazette reported the Rock Ave Rumblers to have between 40 and 50 members.”

…and all is right with the world. Right?

Not necessarily.

The problem with open-source intelligence is that it’s ridiculously easy to fudge, by “cherry picking” the sources you use. (In other words, you draw your conclusions, then hunt up information supporting it.) Sometimes – albeit pretty rarely – this can be desirable; if there’s a corroborated piece of information from a classified source that you want or need to disseminate, maybe cherry-picking open sources that are “right” isn’t too bad. But that’s very much the exception, not the rule. If you’re dealing exclusively with open sources, don’t fall into the trap of only using sources that suit your agenda. At best, it results in an intelligence product with zero value; at worst, you could be inadvertently compromising the sources you’re trying to protect. (As someone wrote in an intelligence journal I was reading a while ago, intelligence often deals more with perceptions than facts. If ninety-nine out of a hundred open sources claim X, generally the best thing to do is to go with that even if its wrong. “Local media outlets are near-unanimous in estimating the Rock Ave Rumblers at between 100 and 125 members throughout the region.” If one open source manages to “get it right”, and you quote them, you’re potentially compromising the classified sources of the “right” figure.)

It sounds obvious, but it’s easy to let your biases – or those of third parties – cloud your judgment. I really hate most things Wikipedia, but their whole “neutral point of view” dogma isn’t entirely useless. Obviously, assessing neutrality in open sources can be difficult; this is why, in my opinion, you need to gather as many sources as possible before drawing conclusions.

I mean, say you don’t know how many members the Dead Hands have in their gang, so you consult teh internets. First source: a local Weasel News report, from last January, which mentions the growing threat the 150 members pose to the community. Great, problem solved! But, wait… an article in the Dead Tree Gazette in November says they’ve only got 50 members, and one in the nearby, big-city Litterbox Liner in December says they number “about four dozen” full members. Oh, and an interview last month in the local alternative paper, the Waste of Space, with a member of a rival gang says the Dead Hand number less than a dozen. What are you to do?

Well, there’s the tried-and-true method of hedging your bets:

“Media estimates late last year generally put the number of Dead Hands members at around 50, though more recent reports have included figures as high as 150.”

The problem is, you’re implicitly raising the possibility that Dead Hand membership is rising dramatically, which has significant implications.

The issue here is one from the good old days of academic writing – primary versus secondary sources, et cetera. The media makes a great open source, but they’re rarely a primary source for anything. That’s not a bad thing by any means, but it means that – especially if you’re only looking at one or two sources – you really need to dig, if necessary, to figure out where they got their information from.

Using the Dead Hand Gang example above, suppose you hunt through Nexis-Lexis and find that the Litterbox Liner last year covered the trial of a Dead Hand member, at which a gang investigator testified that the group had “roughly four dozen full members”. That lends a certain amount of credibility to the four dozen/fifty member figure. Now, trying to figure out how Weasel News arrived at a figure three times that, you learn that, at his sentencing, Ernesto “Twitchy” Llewellyn called the judge a disreputable pederast and swore that the “five score” fellow members of his gentleman’s social club would wreak vengeance “of truly biblical proportions” down upon him. Is that a credible claim? Not really, but it’s easy to see how a media outlet with the high “journalistic” integrity of Weasel News would be all over those statements like white on some varieties of rice. And, well, if you wade through what passes for writing in the Waste of Space these days, you might find that the “less than a dozen” figure was something of a slur:

‘Slicer’ McClintock, Jr: “No, good sir, those Dead Hand children pose little threat to our territorial interests. We fear nothing from that rag-tag collection of, what, perhaps a dozen men?”
Random Intern: “What, sirrah? But a dozen men?”
‘Slicer’ McClintock, Jr: “Well, a dozen men, about thirty youth, and a talking dog, but they don’t count.”
Random Intern: “Not past ten, anyway, eh what?”

(Obviously, we’re dealing with some fairly genteel gangsters, here. But I digress.)

The point is, unless you have no other choice, don’t rely on a single open source for information, and be sure to at least make the effort to identify both the original source, and the context, for that information. Your end product will benefit from it, as will the end-users of that product. Everybody wins…

Published in: Geekiness, General, Security | on April 20th, 2009| Comments Off on The Good and Bad of OSINT

Both comments and pings are currently closed.

Comments are closed.