Security Mindsets, Redux: Trust But Verify

A couple weeks ago, I wrote a little bit about something Bruce Schneier had written on his website, about the uniquely warped mindset of people who worry a lot about security matters. Well, I had an interesting insight into that subject that I thought I’d share…

Many years ago now, Bruce Bethke wrote a novel called Headcrash, about the cyberspace (and virtual-reality) adventures of a St. Paul hacker named Jack Burroughs. At one point, Jack, with the assistance of his gun-nut sidekick Joseph “Gunner” LeMat, is preparing to enter the virtual reality version of the ‘net for the first time, using a quite hilarious array of equipment that isn’t really important to this little story. Anyway, at one point, Jack starts hyperventilating, worrying about his new, highly intimate connection to the ‘net, and whether he can be harmed or even killed by another user. He starts going on about lethal voltages and biofeedback and fried synapses and all these other crazy things, and Gunner basically says (I paraphrase here): Look, you’re safe, okay, because I’m sure the software has safety routines, and I’m sure the hardware has safety limiters. Besides, the whole system is protected by some fool-proof fail-safe devices designed specifically to protect against dangerous voltage and current. I’m sure you’ve heard of them, they’re called fuses.

Jack, of course, feels kind of stupid, and gets on with his cyberspace adventures – thereby proving two things: Jack Burroughs did not have a security mindset, and neither was he an electrical engineer. Because, if he did, the next logical step would have been to haul the manual out of the trash can, and confirm that all the fuses were of the correct rating. Like Reagan said – trust, but verify…

Published in: Geekiness, General, Security | on April 17th, 2008| 2 Comments »

Both comments and pings are currently closed.

2 Comments

  1. On 4/18/2008 at 8:35 am Asim Said:

    (btw, I love your blog. I’ve been following it for a few months now, and appreciate your good work. Please keep it up.)

    I’m curious, what do you feel the next logical step after verifying the fuse ratings would be?

    I’m asking because you seem to have equated a security mindset with an electrical engineering education. I recently graduated from an EE program and I can confirm, with great sadness, the sheer number of manuals and specification books I had to read over to get anything to work.

    But as discussions at Slashdot and Bruce Schneier’s blog have already gone over, does a security mindset come naturally to an engineer? The general consensus of the discussions was that yes, engineers certainly step through failure conditions _based on the assumption that such conditions are random_. People with security mindsets operate from the perspective that a malevolent intelligence is trying to screw them over.

    Relating to your story again, IMHO the electrical engineer would (naturally) conclude “I’ve RTFM, I’m good to go”, whereas someone with a security mindset would say “Who wrote the manual? Can I authenticate their identity? Has the manual been modified? Has the device been modified since the manual was written? …”

    What do you think?

  2. On 4/18/2008 at 1:11 pm Nemo Said:

    Thanks for the kind words! I don’t believe that a security mindset necessarily comes with a degree in electrical engineering – or, indeed, any other sort of degree, alas. I only mentioned it because in the context of the situation from Bethke’s book, it would have – potentially – been useful to Jack Burroughs to have such knowledge.

    In a situation where you’re worried about being killed by lethal voltage from what is, basically, a computer peripheral, and have been reassured that you’re safe because there are fuses meant to protect you, it seems likely that a person with a slightly paranoid security mindset would – at a minimum – check to confirm that the fuses actually existed, and were of the type and rating specified in the manual. A person with a passable knowledge of electronic theory would hopefully be able to determine whether the fuse values were reasonable; either reasonable for their given application, or reasonable to prevent a lethal combination of voltage and current from reaching the user. (“Gunner, this thing is purely USB bus-powered, right? So how come the fuse is 400VAC, 10A, slo-blo?”) Same result – feelings (or illusions?) of safety; just – perhaps – different levels of confidence about it.

    The next step? Well, if lack of fuses (or fuse-like safety features) was your only concern – which was the case with Bethke’s protagonist – fire up the equipment for a smoke test. The real issue – as I see it – is that Jack Burroughs was way too quick to accept both that the interface had fuses, and that they hadn’t been monkeyed with. That, say, the ProctoProd (a human-interface device in the novel that goes, well, you figure out where) was actually filled with plastic-explosives wasn’t Jack’s concern. (Under the circumstances, that would perhaps have been a more realistic worry than death-in-virtual reality, but…)

    I don’t think that a security mindset automatically equates with a completely paranoid, untrusting attitude. As Bruce originally wrote about the subject, it’s more to do with seeing systemic vulnerabilities that aren’t immediately obvious to the pure of heart. 🙂 It isn’t (necessarily) about assuming that everyone lies, or is out to get you, or just forgetful/accident-prone/inept, but rather recognizing that those possibilities exist, and behaving accordingly.

    In many ways, it’s a matter of trust – and of having the appropriate knowledge and frame of reference to make what is, after all, a value judgment. Trust, for all we try to quantify it, isn’t scientific, and isn’t necessarily even rational; it’s emotional, with all that implies.

    I think, obviously, that this is one of those areas where a little knowledge can potentially be more dangerous than none at all. A freshly-minted but paranoid engineer starts disassembling the whole thing, and not seeing anything immediately amiss, begins comparing the circuit with the schematic – and trying to reverse-engineer the whole thing, figuring out why each component is used, et cetera. That’s not necessarily a bad thing, but it seems excessively paranoid. Besides, once you start down that road, it isn’t too long before, when you need a new toaster, say, you buy a dozen of the same model, and spend two weeks attempting to induce different methods of failure in them before deciding they’re safe enough to brown your breakfast toast each day.

    The older and – if not wiser, at least more experienced – engineer examines the board, finds no signs – obvious or otherwise – that it’s been reworked, then decides based on his knowledge and experience that if anybody went to the fantastically unlikely trouble and expense of whipping up a four-layer PCB with 6-mil traces, blind vias, gold plating, and solder-masking and silk-screening on both sides, then had it completely populated with SMD components, just to “get” him, they deserve to win…