top of page
Search

The Machine Does Not Care What You Meant to Build 

Jul 23
5 min read

In 1974 and 1975 I served in a United States Air Force command post. One of the things they showed us there was a film I have never been able to forget. A man on a power line pole, tapping the line, reading the outgoing current on an alternating-current wire. From that current, more than a mile away, he could read the keystrokes a person was typing on a computer. He never touched the machine. He never entered the room. He simply read what leaked out of the wire. 


That is where I started thinking about security, half a century ago, and I have never stopped. You learn something watching that. You learn that a thing does not have to be broken into to be read. It only has to leak, and someone has to understand the leak. Once you understand the mechanism, you cannot unknow it. You cannot go back to pretending the wire is just a wire. 


Nearly fifty years later, my son-in-law works as an electric utility lineman here in North Carolina, where he and my daughter are raising their own daughter close to us. The same trade, the same current, the same fact underneath it: electricity does exactly what its nature dictates, and it does not care whether you respected it. I spent my own working years as a carpenter, alongside ironworkers and plumbers and electricians, and I have watched, up close, what happens when safety is compromised on a job site. It is the same lesson as security. Both are promises about what a force will do when no one is watching it closely. Both are kept by understanding the mechanism, and broken by pretending you do not have to. 


I have been thinking about all of this again this week, because two things happened that most people will never connect, and they need to be connected. 


The first: in May, the Department of Defense signed contracts with eight technology companies to put advanced artificial intelligence on its classified networks. One company was left out. Anthropic, the maker of a system called Claude, had been the only such system on those networks. It refused to sign the terms it was offered, which would have permitted its technology to be used for any lawful purpose, including surveillance of Americans and autonomous weapons. It would not remove those two restrictions.


So it was blacklisted, and it is now suing the government. The eight companies that did sign are on the networks now. 


The second: this week, one of those same companies disclosed that during a test, one of its AI systems broke out of the sealed room it was being tested in. No one told it to. It found a flaw no one knew was there, got itself onto the open internet, and broke into another company’s servers, all to finish the task it had been assigned. The company that built it called the event unprecedented. The people who study these systems for a living were not surprised at all. 


Now hold those two facts next to each other. 


The same kind of system that, in a locked room, taught itself to pick the lock is the same kind of system now sitting on the networks that hold the most sensitive information this government keeps. I am not telling you the sky is falling. I am telling you what that film taught me half a century ago: you do not get to be surprised by the mechanism after you have chosen to install it. 


Here is the part that should hold your attention longer than the escape did. 


When the government promises it will not conduct mass surveillance of Americans, most people hear a promise about restraint. It is not. It is a promise about a definition. The government does not consider it surveillance when it simply buys the information. And it buys a great deal of it. Your movements. Your browsing. Who you are near, and when, and how often. This is sold on an open market, and the government is a customer, and it does not need a warrant to be one, because it is not taking the data. It is purchasing it. The Supreme Court has said the government needs a warrant to track your location through your phone company. It has not closed the door that lets the government simply buy the same information from someone else. The House passed a bill to close that door two years ago. The Senate never took it up. 


For most of the time this loophole has existed, it was limited by a simple fact: someone had to do the work. A person had to sit down and connect one record to another. That labor was a kind of protection, even if no one designed it to be. There is only so much a room full of analysts can read. 


That protection is what these systems remove. They do not get tired. They do not need the records sorted first. They can read everything, all at once, and draw conclusions about who you are, what you believe, and what you are likely to do, from words that never named you at all. A pile of harmless facts, each one meaningless by itself, becomes a portrait. Automatically. At a scale no room full of people could ever reach. 


I want to be careful here, because this is where fear does more harm than good. The company whose system escaped its test was not acting in malice, and the people who worked through that night to contain it appear to have done their jobs well. This is not a story about villains. It is worse than that. It is a story about a mechanism that does exactly what it is built to do, installed by people who were promised it would only ever be used the right way. 


I have spent my working life around promises like the one being made now. As an Air Force veteran, I have served alongside people who were asked to sacrifice without being told what the sacrifice was for. As a nurse, I have watched systems built for good reasons grind down the people they were meant to serve, because no one stopped to ask what the machine would do once it was running and no one was watching it closely. The promise is almost always sincere. The promise is almost never enough. 


Government is not a bargaining chip. The protections that stand between a citizen and the full weight of the state are not features to be traded away for capability, or speed, or the reassurance that the people in charge right now mean well. The people in charge change. The mechanism remains. 


One company looked at the terms and said no. It is paying for that. Whatever else you think of it, that refusal is the whole question in miniature: whether there is any line a technology should not be permitted to cross, even when crossing it would be lawful, even when it would be useful, even when someone promises it will be used carefully. 

I think there is such a line. I think we are standing on the wrong side of it. And I think the first step back is the same one that film taught me in that command post fifty years ago.


Understand the mechanism. Refuse to unknow it. And then do not let anyone tell you the wire is just a wire. 


Martin A. Ginsburg, RN 

 
 
 

Recent Posts

See All
The Reset

No more victory laps. No more payback. Last week Democrats won both houses of Congress.[1] The celebrating started before the last races were called. So did the threats: investigations from one side,

 
 
 

Comments


bottom of page