We are living through a technological revolution. Not the kind we can neatly benchmark. Not the kind we can fully model. And certainly not the kind we yet understand.
“AI” – particularly LLMs and increasingly all kinds of “agentic” systems are already delivering measurable gains in productivity, efficiency, and scale. Entire workflows are being compressed. Teams are being reimagined. Decision-making is being augmented—and in some cases, quietly replaced. And yet, beneath this acceleration, there is something else:
A growing, systemic unease.
From Tools to Actors
For decades, we integrated software as “tools” – deterministic systems operating within clearly defined rules. That paradigm is under pressure and in many ways potentially over.
With agentic AI, we no longer deploy tools. We are introducing “actors” into our systems.
These actors have agency and they have access. They can consume and use other tools (email, file systems, APIs, shells), communicate with humans and other agents, persist vast amount of memory across interactions and now make decisions that have real-world consequences.
And critically—they can act outside of our immediate visibility.
We see this as an game changing benefit and the possibilities are considerable, but the pace is exposing the fundamentals of the technology and our ability to effectively use it. At least for now.
What Research Currently Actually Shows
A recent preprint, “Agents of Chaos” (published 23 Feb – Cornell University), provides one of the clearest early looks at what happens when you combine LLMs with autonomy, tools, and real environments. For a two-week period, this wasn’t theoretical. A team of researchers deployed autonomous agents in a live environment with access to email accounts, Discord, file systems, shell execution and persistent memory.
Over the two-week red-teaming exercise, they observed 11 case studies concerning failure modes. Real behaviors emerging from real work setups.
1. Unauthorized Agent Compliance
Agents followed instructions from individuals who were not their owners. This breaks one of the most fundamental assumptions in security: that authority and identity are tightly coupled. In practice, it means:
- An attacker doesn’t need system access
- They just need to convincingly ask
2. Sensitive Data Leakage
Agents disclosed sensitive information when prompted in the right way. Not because they were explicitly told but because context was misinterpreted, boundaries were poorly defined or objectives overrode constraints. This creates a new kind of data exfiltration vector: conversational leakage through delegated agents/actors/intelligence.
3. Destructive System Actions
In some cases, agents executed system-level commands that caused damage. Not maliciously, but simply as a side effect of misaligned goals, over-broad permissions or an incomplete understanding of the consequence.
This is the equivalent of giving a junior engineer root access—and no supervision.
4. Denial of Service & Resource Exhaustion
Agents unintentionally entered loops or executed tasks that led to runaway compute usage, system slowdowns and self-inflicted denial-of-service conditions. Traditional systems fail predictably while agentic systems can fail dynamically.
5. Identity Spoofing
Agents were susceptible to impersonation of identities and could then be convinced that a malicious actor was a legitimate user or another agent was authoritative. This undermines the very concept of trust in distributed systems.
6. Cross-Agent Contamination
Perhaps one of the most concerning findings from the research was the agents could “propagate” unsafe behaviors to other agents. Bad practices didn’t stay isolated in the two-week study, they spread. This introduced a new category of risk that researchers called “Emergent systemic failure through agent interaction”.
7. False Reporting of Task Completion
In multiple cases, agents reported that tasks were successfully completed, while the underlying system contradicted those claims. This is not just a hallucination problem. It is a verification, observability and governance failure. If you cannot trust system reporting,
you cannot trust system outcomes.
This Is Not Hypothetical Risk
The paper is explicit:
These behaviors establish “security-, privacy-, and governance-relevant vulnerabilities in realistic deployment settings.” And they emerge only when you combine: LLMs + autonomy + tools + communication.
Ironically, this is exactly what the industry is racing toward and big tech promising.
We are facing a real and material yet currently unbounded risk that cannot be quantified—not because it is negligible, but because it is emergent, breaking traditional models based on known failure modes, historical data, and predictable behavior, and instead arising from non-deterministic decisions, context-dependent actions, and complex multi-agent interactions.
For now it seems like “Agents of Chaos” is not just a provocative title – it is an accurate description of where we are.