Who should read this: You're designing an AI Agent system but feel like the workflows and rules you're building by hand are multiplying without improving efficiency; or you want to know why "better models" are quietly eating the engineering optimizations you worked so hard on.
After reading this, you'll get: a set of criteria for judging whether an Agent architecture is "scalable," the three most common design anti-patterns today, and a construction approach that truly follows the laws of AI development.
1. Seventy Years of AI Research Has Taught Us One Thing
In 1997, Deep Blue defeated world chess champion Garry Kasparov through massive depth-first search. This wasn't about stuffing human chess knowledge into the system—quite the opposite. It relied on extremely pure, almost "brutish" brute-force search. Twenty years later, the same story played out in Go: AlphaZero completely discarded human game records and learned from scratch through self-play, ultimately surpassing every system built on human domain knowledge.
In speech recognition, at the 1970s DARPA competition, statistical and computational methods (such as Hidden Markov Models, HMMs) defeated approaches that tried to mimic human pronunciation knowledge. In computer vision, hand-designed feature extractors (edge detection, SIFT, generalized cylinders) were eventually replaced by convolutional neural networks that could be trained at scale.
All these stories point to the same conclusion. Richard Sutton wrote it most succinctly in his 2019 blog post "The Bitter Lesson"—the post was under 1,200 words at the time and has now been cited hundreds of times on Google Scholar, including in Google DeepMind's "A Generalist Agent" paper and Noam Shazeer's Switch Transformer paper:
"The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin."
The reason this conclusion is "bitter" is that it undermines our most natural intuition—manually encoding human intelligence into systems. Sutton pointed out that AI researchers repeatedly tried to embed human knowledge into systems; this approach works well in the short term and brings researcher satisfaction, but in the long run it leads to stalled progress and even blocks true breakthroughs. The only path that consistently works is letting methods automatically scale with computing power—and currently, there are only two such methods: search and learning.
2. Why Many Ideas About Agents Today May Repeat the Same Mistakes
Sutton's observation didn't stop in 2019. In September 2025, during an appearance on the Dwarkesh Patel podcast, he revisited his article and made a sharper judgment:
"Large language models learn from training data, not from experience."
In Sutton's view, today's AI field cites "The Bitter Lesson" but largely does the opposite. LLMs essentially imitate human text rather than acting in the world, observing consequences, forming goals, and learning from experience. In an updated Chapter 2 from 2025, he even said bluntly: "The field has been citing the 2019 article but largely doing the opposite of what it advocated."
Applying this framework to today's Agent construction, you'll see three patterns that are repeating the same mistakes.
Anti-pattern 1: The Workflow Trap
Many Agent frameworks let you visually build workflows, dragging and dropping to chain together "research → summarize → write." This makes product sense—current AI models aren't reliable enough, and this approach seems necessary to reach usable levels. Observations from YC Demo Day confirm this: nearly all pitched products fell into the "vertical workflow" category.
But the problem is that you're hardcoding your assumptions about task decomposition into the architecture. When models improve, your workflows won't automatically get simpler, because the team is already locked into that flowchart. YC partner Jarred also confirmed this observation on the Lightcone podcast:
"The first wave of fixed-process AI applications was mostly overwhelmed by newer GPT models."
Anti-pattern 2: The Specialized Subagent Illusion
Designing a research Agent, a coding Agent, and a writing Agent—this looks like human organizational structure. But human organizations evolved under constraints of limited cognition and high communication costs; AI doesn't necessarily have those limitations. When you freeze fixed role divisions into the architecture, you're importing human limitations rather than leveraging computational advantages.
The single-Agent-plus-skills pattern is actually more efficient in many serial tasks than multi-Agent collaboration—because communication itself causes information loss, and errors accumulate in serial architectures. Research by Professor Li Xiaoxiao at UBC TEA Lab found that a single Agent reaches optimal performance with roughly 20 to 30 skills; going beyond that causes performance to saturate or even decline—because the context window is its "working memory," and when Context gets very long, AI also suffers from "Lost in the Middle."
Anti-pattern 3: The For-Loop Ceiling
"LLM + loop + tools" sounds lean, but your only expansion knob is iteration count. Facing complex tasks, this is one-dimensional scaling—it's hard to parallelize and easily wastes tokens. As one analyst pointed out, the only way to scale this pattern is increasing loop iterations, and for complex tasks, this one-dimensional expansion rarely solves the problem.
3. What Truly Aligns with "The Bitter Lesson"
Approaches aligned with the Bitter Lesson have two core characteristics: converting extra compute into better decisions, rather than relying on fixed human-designed decomposition.
Direction 1: Dynamic Subagent Spawning
Don't pre-design fixed Agent divisions and workflows. Instead, let the model decide at runtime how to decompose tasks, how many subagents to generate, and what each should do. Anthropic's multi-agent research system, LangChain's Deep Agents, and Claude Code's subagent delegation—though implemented differently—all follow the same principle: as models improve, delegation strategies get "for free" better, without you rewriting the org chart.
A practical test:
"If model capability doubled next year, would your system become simpler, cheaper, or more reliable without major restructuring?"
If the answer is "yes," you're on the Bitter Lesson side. If your scaling plan is "add more nodes/roles," you're scaling human labor. If it's "let the model allocate more compute through general methods," that's scaling computation.
Direction 2: Recursive Language Models
MIT researchers treated the entire prompt as an external string, with the model reasoning through code and recursive self-calls. This approach can handle inputs up to 100 times larger than the native attention window, achieving over 91% accuracy on benchmarks ranging from 6 million to 11 million tokens, while costing as little as one-third because the model selectively inspects context rather than processing everything. The logic behind this: let the model decide what to inspect, compress, and recurse on—don't have humans prescribe it.
4. Harness—The Real Competitiveness Beyond the Model
When discussing specific approaches, one concept can't be skipped—Harness. This definition from Tencent Cloud developers is crystal clear:
Agent = LLM + [Context + Tools + Constraints + Verification + Correction]
The first two (Context + Tools) let the Agent "get things done"; the last three (Constraints + Verification + Correction) let the Agent "not do wrong things." Anthropic also draws a clear distinction between Workflow and Agent: a Workflow is a system where large models and tools interact under predetermined paths; an Agent has autonomous Planning and evolution capabilities—it can not only use tools but even create them.
Strong evidence comes from LangChain: on the Terminal Bench 2.0 benchmark, by improving the Harness—letting the Agent automatically check its own execution results, detect whether it's stuck in repetitive loops, and optimize its thinking strategy—they raised accuracy from 52.8% to 66.5%, jumping from outside the top 30 to the top 5. What changed wasn't the model—it was the Harness.
Another case: OpenAI's engineering team—three engineers completing roughly one million lines of code and nearly 1,500 PRs in five months, reaching roughly 10x traditional development speed—was underpinned by getting the Harness right.
There's a key dialectical relationship here: the stronger the model, the more critical the Harness becomes. The more a model internalizes tool calls, the wider the system boundary it touches—and the greater the blast radius when things go wrong. This is why stronger models need those three layers of protection even more. This echoes Anthropic's position: model vendors' real advantage isn't "making the framework thin"—it's doing co-optimization of "model + peripheral Harness."
5. What You Can Do Now
Returning to Sutton's framework, all those manually built rules and workflows are essentially "embedding human knowledge into the system." This path works in the short term but will inevitably be overtaken by more general solutions in the long run. After Anthropic released Opus 4.5 in November 2025, Agents could already solve complex long-horizon tasks that only expert-level humans could handle months ago—this threshold keeps receding. Meanwhile, Agent software development performance on SWE-bench Verified climbed from roughly 30% in March 2024 to roughly 80% in early 2026, and the length of tasks general models can execute doubles every seven months—any "wrapper" built on top of models risks being directly swallowed by stronger models.
Concretely, here's what to do:
- Start from the simplest approach. Direct API calls beat complex frameworks; clear code beats clever abstractions—every layer of abstraction becomes a future debugging blind spot.
- Make every tool return structured, verifiable feedback. Don't just return "success" and stop. Embed checks like linters into the loop, forming a "execute → verify → feedback" cycle.
- Default-deny everything, explicitly open up. Capability should be like phone permissions—off by default. Any side effect enabled by default is a ticking time bomb.
- Give the model a status bar. Continuously feed back task progress, environment state, and available capabilities—the model can't see the dashboard on its own; the shell needs to help it calibrate.
- Never trust the model when it says "I'm done." Go verify it in the real world first. Verification should look only at structured results returned by tools, not at what the model claims.
- Use workflows as a safety net for dangerous operations; give autonomous Agents free rein for open exploration. Deterministic critical steps must never be allowed to be skipped; enumerate everything you can think of, then let it roam.
6. Caveats to Watch For
In closing, it's worth noting a few uncertainties:
-
The 90-90 rule reminder. The 90-90 rule in software engineering (90% of the feature took 90% of the time; the remaining 10% also took 90% of the time) applies equally to AI development. We may have only completed the "easy" part of AI progress; the remaining last 10%—or even 0.1%—may be vastly more difficult than imagined. The leap from L2 to L5 in autonomous driving is the best cautionary tale.
-
The verification bottleneck still exists. Tasks requiring human judgment and defying automated verification will progress far more slowly than fields that can be automatically verified (like mathematics and code). This means that in the foreseeable future, "manually built rules" remain irreplaceable in specific scenarios.
-
Sutton's judgment comes from a single source. His claim that "LLMs are not a sufficient path to true intelligence" currently stems mainly from his own podcast appearances and blog posts, without broad independent verification yet. Similarly, Karpathy's analogy of LLMs as "ghosts" (statistical refinements of human data) versus Sutton's ideal agent as an "animal" (learning from experience, rooted in the physical world) remains a single-source observation.
-
The pace of model progress is itself a variable. From March 2024 to early 2026, Agent software development performance on SWE-bench Verified climbed from roughly 30% to roughly 80%. This pace itself reminds us: engineering solutions that look solid today may be substantially undermined after the next model release.
Next Steps
If you're building an AI Agent system, I suggest checking your architecture in this order:
- List all the manually built rules, workflows, and role divisions in your system.
- For each one, ask: if model capability doubled, would this part become redundant?
- If the answer is "yes," consider reshaping it to let the model decide autonomously.
- If the answer is "no," keep it—but ensure it has clear verification and correction mechanisms.
One final point: The core of "The Bitter Lesson" isn't telling us to abandon all human knowledge. It's reminding us that human knowledge continuously migrates upstream, and the downstream gets automated away. Today's Harness engineering is essentially helping you patch the gaps that can't yet be handed over to the model on their own. As the model internalizes each layer, the Harness sheds a layer and instead backs the new frontier.