- Towards AGI
- Posts
- Apple Says AI Agents May Go Rogue. Who Holds Your Kill Switch?
Apple Says AI Agents May Go Rogue. Who Holds Your Kill Switch?
here's what your agent controls are missing
Today, we're diving into:
Hot Tea: Apple's annual filing warns AI agents could act without authorization, as CrowdStrike ships an AI Kill Switch
Gen AI: Anthropic tells investors it may imminently hit its own threshold for AI-accelerated AI research
Gen AI: Revolut says self-hosted open models cut its AI coding inference bill by roughly 90%
Apple Says AI Agents May Go Rogue. Who Holds Your Kill Switch?
In its annual filing with the SEC, Apple warned that AI agents "may behave in unintended or unpredictable ways, including by taking actions that were not directed, instructed or authorised," TRT World reports. The filing landed a day after CEO John Ternus's earnings call.
On the same day, CrowdStrike launched an AI Kill Switch built to stop exactly that kind of agent. One company named the risk. The other started selling the brake.
An Agent You Cannot Stop Is An Agent You Do Not Control.
Find out which of your agents can reach data and systems nobody has mapped.
What Apple Actually Warned About
The language is specific. According to TRT World, Apple said agents could independently execute or modify software code, send messages and complete transactions, and could interact with systems inside or outside the company without the necessary permissions.
Apple also said its ability to monitor and control third-party AI agents "could be limited." That is a striking admission from a company that controls its own hardware, operating system and app store.
"The use of artificial intelligence in the development and operation of operating system software, applications and services may also give rise to additional risks from errors, bugs or vulnerabilities." - Apple annual filing, as quoted by TRT World
None of this says an Apple agent has gone wrong. It is a risk disclosure. But risk factors are where companies write down what their lawyers think could realistically hurt them, and agents have now made that list.
The Brake Arrives The Same Day
CrowdStrike's AI Kill Switch sits inside Falcon Guardian. The company says that if an agent breaks out of its intended behavior, a defender can terminate it where it runs, block it from launching again and cut off its enterprise network access. An authorized administrator triggers the action, and every action is recorded in the audit trail.
"We want our teams to use AI agents. If one goes off course, the AI Kill Switch gives our security team the final say." - Rick Orloff, CISO, Everpure
One detail worth reading closely: CrowdStrike says termination happens first at the gateway, with endpoint enforcement coming "soon."
This Is Not A Security Product Problem. It Is A Visibility Problem.
A kill switch only works on an agent you know exists. You cannot stop what you never registered, and you cannot judge "off course" if nobody wrote down what the course was.
The wrong reaction is to buy a kill switch and call the job done. You will end up with a red button wired to half your agents.
The better alternative is to treat every agent like a privileged user: known, scoped, logged and owned by a named human before it ever reaches production.
What You Should Have In Place Before Your Next Agent Ships
An agent inventory. Every agent, internal or third-party, with an owner and a stated purpose.
A written definition of normal. Which systems, data and actions each agent is allowed to touch.
A tested stop path. Someone has actually shut an agent down in a drill, and the logs prove it.
Contract language for vendor agents. If Apple says its control over third-party agents could be limited, assume yours is too.
Answering those questions starts with trusted enterprise context. DataManagement.AI helps organizations create that foundation by unifying metadata, lineage, governance, business glossaries and knowledge assets into a single intelligence layer. It maps lineage across the modern data stack and identifies downstream dependencies, so you know what an agent can reach before you need to stop it.
The organizations moving first will not just buy a kill switch. They will know every agent it needs to reach.
Anthropic Says AI May Soon Speed Up Its Own Progress. Is Your Roadmap Ready?
In a revised IPO filing on Monday, Anthropic said its internal estimates "indicate that we may imminently meet our Responsible Scaling Policy's definition of 'dramatically accelerated progress attributable to AI,'" TechCrunch reports.
In plain terms, the company is telling investors that AI helping to build AI may soon move faster than anyone planned for. That is the closest thing to an AGI timeline warning you will find in a securities document.
What The Filing Actually Says
Anthropic was careful. It said it has not designated any current model as exceeding the threshold and that it continues to assess the data. It also admitted that, in the past, it did not identify significant acceleration until after it had already happened.
"Using AI to help build more capable AI could also pose a considerable safety risk, particularly if it sharply accelerates the pace of progress." - Anthropic S-1/A, as quoted by TechCrunch
The company added that it is "considering updates to our Responsible Scaling Policy to keep humans in control of frontier model development at Anthropic."
The Numbers Behind The Warning
According to The Neuron's read of the filing, Claude was responsible for more than 90% of code contributions at Anthropic by September, up from more than 80% of merged production code in May.
An outside view is more measured. The Neuron notes that METR, which tested Claude Opus 5.5 in September, saw improvements but considered it unlikely to automate the whole research process.
The Real Bottleneck Is Not Model Speed. It Is Your Planning Cycle.
Most enterprise AI roadmaps are written on an annual cycle with one silent assumption: the models you pick in January will look roughly the same in December.
If one of the leading labs thinks its pace of progress could double, that assumption is fragile. The wrong reaction is to panic and freeze spending until things settle. They will not settle.
The better alternative is to build for replacement. Design every AI workflow so the model underneath can be swapped quickly, re-evaluated on your own cases and governed by the same rules, whatever arrives next.
Shorten your AI planning horizon. Review model choices every quarter, not every budget cycle.
Own your evaluations. Keep a test set of your real tasks so a new model can be judged in days.
Separate context from the model. Your definitions, rules and data should not need rebuilding when the model changes.
Watch the safety policies, not just the benchmarks. A lab's change to its scaling policy can change what you are allowed to deploy.
The organizations moving first will not just predict when AI accelerates. They will be built to absorb it whenever it does.
Revolut Cut Its AI Coding Bill By 90%. Could Yours Fall Too?
Revolut has cut its AI coding inference costs by roughly 90% in under a year by making self-hosted open models its default, The Register reports. David Abitbol, the fintech's ops and infrastructure lead, shared the numbers at the AI Engineer conference in New York.
The story is not that frontier models are bad. It is that Revolut stopped paying frontier prices for work that did not need them.
How Revolut Got There
The company moved from Gemini in early 2025 to Claude Code in early 2026, at a time when some engineers were "spending $1,000 a month, $10,000 a month." It now runs Z.ai's GLM models on 56 Nvidia B200 GPUs in-house.
"We were highly skeptical about the cost of AI." - David Abitbol, Revolut
It has not gone all in on one model. According to The Register, GLM has not replaced GPT or Claude entirely, and vision queries are still routed to other models. Code generation stays on GLM.
Read The Number Carefully
The 90% figure is Revolut's own, as is its claim of a 60% improvement in coding time. Neither has been independently verified. It also comes with a real hardware bill and a team able to run it.
This Fundamentally Changes How You Should Buy AI
Most enterprises still buy AI like software seats: pick a vendor, hand out licenses, watch the invoice. Revolut treated it like infrastructure, measuring cost, speed and quality by workload and routing each one to the cheapest model that does the job.
The wrong reaction is to rip out your frontier provider tomorrow. Self-hosting brings GPU capacity, security reviews and model operations you may not be ready for.
The better alternative is to know your numbers first. Abitbol's warning that "long context is a scam" is a reminder that the biggest savings often come from how you use a model, not which one you choose.
Measure cost per completed task. Not per seat and not per token alone.
Split workloads by difficulty. Routine code generation rarely needs your most expensive model.
Trim the context you send. Every unnecessary file in a prompt is a cost you pay on every call.
Price self-hosting honestly. Include hardware, staff and security before you compare it with an API bill.
Know Your Inference (KYI) is built for exactly this decision. When you understand what each AI workload costs to run, how fast it performs and how accurate it is on your own cases, you can route simple work to lighter models, save premium capacity for tasks that need it and prove the savings instead of hoping for them.
The organizations moving first will not just pick the smartest model. They will pick the right model for every task, and know exactly what each one costs.
Journey Towards AGI
Research and advisory firm guiding on the journey to Artificial General Intelligence
Know Your Inference Maximising GenAI impact on performance and Efficiency. | Model Context Protocol Connect with us, and get end-to-end guidance on AI implementation. |
Your opinion matters!
Hope you loved reading our piece of newsletter as much as we had fun writing it.
Share your experience and feedback with us below 'cause we take your critique very critically.
How's your experience? |
Thank you for reading
-Shen & Towards AGI team