If a pilot of a plane told me that they catch any critical errors they make 89% of the time, I would NOT get on that fucking plane.
Yeah but if I were buying cheap corn and they said 89% of the ears were good, I would still buy the corn. So is AI airplanes or corn?
That’s not entirely the same thing. It says it catches 89% of HARMFUL actions. That means that 11% of the time it doesn’t.
That’s more like if the corn had a parasite on it that could give you explosive diarrhea 11% of the time. Would you still eat the corn?
Also, in my example, I didn’t say the harmful actions of the pilot resulted in a crash. However, that doesn’t mean it couldn’t. How many of those 11% of the time are the “woops I deleted the production database” stories we hear all the time?
Auto mode sounds less like security and more like outsourcing judgment to the same AI you’re supposedly trying to constrain. If users are bad at spotting malicious prompts, the answer shouldn’t be: great, let’s remove them from the loop entirely. That’s not safety. That’s automated permission laundering.
I just assumed it was to increase token use and create a windfall for Anthropic
So if I remember correctly, what happens with auto mode is they run a second smaller LLM called the “classifier” to evaluate the tool uses of the main one.
I’ve used auto mode at work now for many months, and it approves most things because most things Claude does are reasonable.
Once or twice I have seen it reject Claude. I can’t remember the exact scenario, but I had asked Claude to diagnose an issue but not fix it yet, and when later on it tried to make the change the classifier rejected it, giving the reason that what it was trying to do did not match my request.
In my opinion, the trick to using auto mode safely here is to:
- Commit and push your changes before handing over the reins (it rarely commits or pushes without you telling it to, especially if you have never asked in that session).
- Don’t give it access to things it shouldn’t have access to or that can do significant damage
- Don’t give ambiguous prompts.
- Never use LLMs against untrusted code or files, as it may contain prompt injections.
“Claude is better than humans to detect errors Claude makes, therefore, you will use Claude”
Honestly I’ve been using auto for the past year and it hasn’t bitten me once. It generally still pauses if there’s a big decision to be made, and small decisions that were made are shown.
A brand new FAFO machine, great!
The OpenAI talk at DefCon shows what can happen when you let these things run without any human intervention.
The solution is, obviously, even less human intervention. 🤦🏻♂️
