Static weights are the bottleneck. Most LLMs can infer in-session, but they don’t durably update from experience unless you retrain or fine-tune.
Context windows, RAG, and “memory” features help, but they’re working memory plus retrieval layers, not parameter or state updates with stability guarantees.
Continual learning is the real unlock, and the hard part is balancing plasticity (learn new things) with stability (don’t forget old things).
Google/DeepMind’s direction looks evolutionary: keep transformer backbones, add nested learning and surprise-gated promotion into long-term memory.
Liquid.ai’s direction is more revolutionary: move toward continuous-time, dynamical-system models that may be better shaped for ongoing adaptation.
If continual learning works, personalization stops being prompt glue.
If it fails, you get drift, poisoning, and privacy landmines. The “2026” prediction might be off, but the pressure is real: training is expensive, knowledge changes fast, and “smart-but-stuck” is now a product tax.
1. Cold open: 2026 as the “learning, not prompting” inflection
All right, so we just cruised into 2026.
And if you’ve spent any time building with LLMs in the last two years, you’ve probably felt the same friction I have: these models are impressive, but they’re weirdly static. They can reason, summarize, code, and argue. But they don’t learn the way you’d expect a capable system to learn.
There’s a claim making the rounds that I can’t stop thinking about: 2026 is going to be the year of continual learning. Not “better prompts”. Not “bigger context”. Not “more agents”. Actual learning over time.
I recently went down a rabbit hole that split into two tracks. One is coming out of Google/DeepMind research, with nested learning and memory-centric architectures like Titans and HoPE (I started from this talk and followed the citations from there: YouTube). The other is Liquid.ai’s post-transformer bet, where they’re explicitly pitching a different compute model: Liquid.ai Models.
This post is me expanding that thread, why today’s models feel smart-but-stuck, what “continual learning” really implies, and why these two camps might be the first real cracks in the “frozen weights forever” era.
There’s a question that keeps coming up when I talk to teams shipping LLM features: why do these models feel like they understand everything, yet they keep repeating the same mistakes?
A big part of it is structural: most deployed LLMs have frozen weights. They’re trained, evaluated, released, and then they stay that way until the next expensive training run or fine-tune cycle.
So yes, they can infer inside a session. They can adapt behavior via prompt conditioning. But they don’t actually update the underlying model in a durable way.
That distinction matters more than most product conversations admit, because it’s the difference between “the system can follow instructions while you remind it” and “the system got better because it experienced something.”
Context windows are working memory, not learning
We’ve all seen the progress on context length. Some models can hold an absurd amount of tokens now. You can paste in a repo, a runbook, and half a novel, and it will keep a coherent thread.
But even a massive context window still behaves like short-term working memory. It can use what’s in the window, it can chain reasoning across it, and then it’s gone. That’s not a knock. Working memory is useful. But it’s not learning.
If anything, big context windows can trick us into thinking the system is improving. It’s not. You’re just carrying more scaffolding forward each time.
“Memory” features are mostly a second system bolted on
A lot of chat products now have a “memory” feature. You see behaviors like summarizing your preferences, storing a few user facts, and retrieving those facts later. It’s useful and it works most of the time.
But most of these implementations are a separate storage and retrieval layer plus some heuristics about what to store, not the model internalizing new skills or facts into its parameters, or into a first-class adaptive state with clear stability constraints.
So the product feels better, but the underlying model is still… stuck.
RAG is looking up a textbook, not learning the skill
RAG is another case where we sometimes overstate what’s happening. With RAG, we say: “Don’t memorize everything. Retrieve relevant documents at runtime.” That’s great engineering. It’s also not learning.
A clean way to think about it: RAG is open-book, while continual learning is actually learning the material. If I keep asking you the same question and you keep looking it up in a binder, you’re still useful. You might even be more useful than someone relying on memory. But you haven’t improved, you’ve just gotten good at searching.
And the failure mode is predictable: if retrieval misses, if the embedding is off, if the doc is stale, you regress immediately.
A more realistic failure: the personalized coding assistant
The goofy “emoji hallucination” examples are funny, but engineers feel the pain more sharply in coding workflows.
Here’s a scenario I’ve watched play out repeatedly:
You use an AI coding assistant for weeks. You switch stacks (say, from Node 16 + CommonJS to Node 22 + ESM, or from Python 3.9 to Python 3.12). You settle on a style too: stricter typing, different lint rules, a new framework preference, more explicit error handling, fewer magical abstractions.
What happens? It helps… as long as you keep reminding it. Next session, it suggests patterns you abandoned. It forgets the constraints that mattered yesterday.
So you end up building a second system around it: system prompts, “project rules” documents, pinned context in every chat, custom RAG over your repo, sometimes fine-tuning if you’re serious. That works. But it’s the same underlying story: the model itself isn’t adapting. You’re doing the adaptation work externally.
The business tax: retraining is slow and expensive
This is where the “stuck” feeling turns into a real business constraint.
If the only durable way to update behavior is retraining or frequent fine-tunes, you’re paying for data pipelines (collection, cleaning, labeling), training compute, evaluation cycles, rollout risk, and regression handling. And even when you’re not training a frontier model, the economics show up proportionally. Big updates happen in batches, not continuously.
That’s why I think continual learning is getting pulled forward by pressure, not hype. It’s the same reason teams adopt streaming architectures. The world doesn’t update quarterly.
Also, this pressure isn’t just “AI teams want nicer models.” The skills gap is real, and it’s limiting how fast organizations can respond. According to the 2026 Workforce Trends Report | Leapsome, organizations are explicitly calling out AI skill gaps as a blocker to innovation. When the people side can’t scale fast enough, the value of systems that adapt faster (with less bespoke tuning) goes up.
3. Continual learning, translated: plasticity vs stability and update cadence
The “2026” framing that stuck in my head
The framing I saw was attributed to Ronak Mald (Google DeepMind / reinforcement learning background): 2024 was the year of agents, 2025 was the year of reinforcement learning, and 2026 will be the year of continual learning.
People argued about the first two years, and honestly, that’s fair. Most teams I talk to didn’t experience 2024 as “agents everywhere”. They experienced it as “ChatGPT, copilots, and a lot of prototypes.”
But here’s where it gets interesting: the direction of the claim feels right even if the calendar is off by 6 to 18 months.
Because the limiting factor right now is not raw capability. It’s adaptation.
Fluid vs crystallized intelligence is a useful analogy (if we don’t get weird about it)
I like the brain analogy here, as long as we don’t push it too far. Crystallized intelligence is drawing on stored knowledge and experience, while fluid intelligence is adapting quickly to new situations with limited examples.
Today’s LLMs feel like they have a ton of crystallized intelligence. They can talk about everything. They can pattern match like monsters.
But they’re weak on fluid intelligence in a very specific engineering sense: they don’t update their internal world model from ongoing experience in a durable way.
Continual learning is not “fine-tune more often”
In ML terms, continual learning is the idea that a model can acquire new skills and facts over time, without catastrophic forgetting (overwriting what it previously knew), ideally without requiring full retraining.
That “without forgetting” clause is the whole problem.
If you naively train on new data, you often get drift into the new distribution, degradation on older tasks, and regressions that feel like the model “lost” skills.
So continual learning is really a balancing act between plasticity (ability to learn new things) and stability (ability to retain old things). If you’re an engineer, think of it like operating a live system while doing schema migrations. You want change, but you want controlled change. You want rollbacks. You want invariants.
Update cadence is the hidden design decision
One more translation that matters: continual learning isn’t one thing. It’s a spectrum based on update frequency. Some systems “learn” once per week through batched updates. Some aim for session-level updates. Some aim for near real-time updates.
The tighter the loop, the more useful it gets for personalization and agents. And also, the more dangerous it gets.
Because the moment you let a system update frequently, you inherit all the messy operational realities we normally avoid by freezing weights: debugging becomes time-dependent, reproducibility becomes harder, and incidents can be caused by “learning,” not just code deploys.
Also, the economics matter. Training and retraining costs are now significant enough that investors are paying attention to upskilling and efficiency as a macro trend. The Strategic Institutional Investment in EdTech: Capitalizing on 2026 Upskilling Trends report frames AI upskilling as a capital allocation theme, which is a polite way of saying: “this is expensive, and the market knows it.”
Continual learning is partly a capability play, but it’s also a cost-structure play.
4. Google/DeepMind direction: nested learning and surprise-gated promotion
The core idea: two loops, two timescales
Google Research has been publishing in the direction of continual learning paradigms that look a lot like “nested learning,” meaning you have two learning loops operating at different timescales (this is the thread I followed from the talk: YouTube).
The intuition is close to how you’d describe human memory in casual terms: a fast loop that adapts quickly but doesn’t keep everything, plus a slow loop that stores what matters for the long haul.
In engineering terms, it’s like a write-heavy cache with aggressive eviction, plus a durable store with stricter write rules. We already have something like the fast loop in LLM products: the context window. It’s short-term working memory. It’s cheap to write, easy to overwrite, and it disappears at the end of the session.
But the missing piece is the bridge into durable memory.
What counts as “important” enough to store?
This is the part that made me lean in.
If you’re going to build a model that updates continuously, you need a gate. A policy that decides what gets stored, what gets discarded, what gets reinforced, and what gets forgotten.
One of the criteria that shows up in this line of thinking is surprise. Surprise, in this context, is basically the gap between the model’s prediction of the world (its current world model) and what actually happens (new evidence).
When the gap is big, you need an update.
That maps nicely to how humans work. If you tell me something I already assumed, it doesn’t stick. If you tell me something that forces me to rewrite part of my internal model, I remember it.
The honey example as a “memory write event”
The honey story is a perfect illustration of surprise gating.
Most people know honey comes from nectar. Not surprising.
But when you hear the full mechanism (nectar processed by bees, dehydrated in combs), it triggers a “wait, what?” moment. That’s surprise. That’s a write.
That’s what a continual learning system needs: a way to detect a meaningful prediction error and decide it deserves long-term storage. And importantly, it’s not just “store the fact.” It’s store it in a way that can influence future behavior without destabilizing everything else.
Why this matters more than bigger context windows
Bigger context windows help with short-term coherence. They’re great for long document QA, repo-level reasoning, and multi-step workflows that need lots of state.
They do not solve recurring user preferences, tool usage patterns over months, domain-specific facts that change weekly, or incremental skill acquisition.
Nested learning is interesting because it’s trying to formalize the pipeline: short-term experience gets filtered, compressed, and selectively promoted into durable memory.
If that works, it’s a step toward something we don’t really have today: models that “live” in the world and get better as they interact with it, without needing a quarterly retrain to become competent again.
5. Titans and HoPE: storing memories vs maintaining and reshaping them
Titans: the “filing cabinet” model of long-term memory
If nested learning is the paradigm, Titans (a memory-centric architecture) reads like one concrete attempt at building the mechanism.
The way I internalize Titans is exactly how it was described in the narrative: a filing cabinet. You observe events in the fast loop, you score them for importance (surprise is one signal), you file the important ones into a long-term store, and later, you retrieve them when relevant.
That’s already a big step forward from pure “context window amnesia.” Because now the model can carry forward useful state across time.
In product terms: the assistant doesn’t just “remember” because you pasted it in again. It remembers because the system chose to store it. And as engineers, we know why that matters. It creates an explicit mechanism you can test, tune, and govern.
HoPE: the “reorganizing brain” model (store, reshuffle, forget)
HoPE (described as a variant built on top of these ideas) adds a different flavor.
The key shift is: it’s not just storing memories. It’s self-modifying in a recurrent way, with repeated internal loops that can reorganize memory over time.
That matters because long-term memory isn’t just a write-once database. In humans, memory is strengthened when used, weakened when unused, and sometimes rewritten when new evidence conflicts.
So the “infinite looped learning levels” language maps to something like repeated consolidation passes, re-ranking of stored memories, and forgetting mechanisms based on usage and relevance.
In practice, this is the part that tends to separate “cool demo memory” from “usable continual learning system.” Because if you never forget, you fill up with junk. If you forget too aggressively, you lose stability. If you rewrite incorrectly, you drift.
Why engineers should care: the failure modes change
If you build on static models today, your failures look like:
hallucination
stale knowledge
brittle prompt dependence
inconsistent tool use
session-to-session reset
If you build on continual learning models tomorrow, your failures change shape:
drift from overfitting to recent interactions
poisoning (malicious or accidental)
privacy leakage through stored memories
inconsistent behavior across time because the system is literally changing
So yes, this is exciting. It’s also the kind of thing that forces us to build new guardrails.
A quick reality check on benchmarks
People love asking: “What’s the delta? How much better is it?”
The uncomfortable truth is that a lot of the most important continual learning wins aren’t captured by a single headline benchmark the way “MMLU +4” is.
The interesting metrics are usually things like retention over sequential tasks, performance under distribution shift, and ability to incorporate new facts without degrading older competencies.
If you’re tracking this space, look for papers that report sequential learning curves, forgetting metrics across tasks, and ablation studies on memory gating signals like surprise. Those are the “SLOs” for continual learning.
5.5 Google vs Liquid.ai: evolution vs revolution
Both camps are aiming at the same pain: models that don’t adapt. But the architectural philosophy is different enough that it’s worth making it explicit.
The evolutionary path (Google/DeepMind)
Google’s approach keeps transformer-style reasoning as the backbone and adds memory plus gated promotion on top. “Learning” in this paradigm looks like memory-centric continual learning: store, retrieve, consolidate. Experiences get promoted into durable memory, possibly with consolidation loops that reorganize stored knowledge over time.
The strengths here are clear: it’s a more incremental path from today’s deployments and easier to integrate with existing tooling. The risks are equally clear: memory governance becomes critical, drift can happen via memory accumulation, and the complexity of consolidation policies can spiral.
If you’re evaluating this direction, watch for how well surprise gating works in practice, what controls exist for forgetting, and whether sequential-task evaluations hold up.
The revolutionary path (Liquid.ai)
Liquid.ai’s bet is more fundamental. They’re proposing that the transformer may be the wrong base abstraction for lifelong learning, so they’re changing the base compute entirely. Their Liquid Foundation Models (LFMs) are inspired by continuous-time dynamics, where internal state evolves according to differential equations rather than discrete layer-by-layer steps.
In this framing, “learning” is potentially state dynamics that naturally adapt over time. The adjustment happens via dynamical system behavior, possibly enabling smoother adaptation than memory promotion alone.
The strengths here are theoretical: if the architecture fits the problem, it could be more efficient and better aligned with continual adaptation from the ground up. The risks are equally real: it’s harder to validate, harder to compare apples-to-apples with transformer baselines, and success depends on strong evidence and reproducibility.
If you’re evaluating this direction, look for adaptation tests, stability under updates, and clear baselines showing where LFMs outperform transformers on continual learning tasks.
The strategic read
If you’re an engineering leader making bets, the difference feels like this: Google’s route is evolutionary (extend what works, add a memory pipeline, and make “learning” a controlled subsystem), while Liquid.ai’s route is revolutionary (the transformer may not be the right foundation for lifelong learning, so change the foundation).
Both can be right. Or both can fail for different reasons.
6. The other bet: Liquid.ai and what to verify beyond marketing
Liquid.ai is explicitly pitching “beyond transformers”
After going deep on Google’s memory-centric direction, I looked at Liquid.ai because it’s a different kind of bet.
They’re not just saying “add memory modules to transformers.”
They’re marketing Liquid Foundation Models (LFMs) as a different architectural direction, according to their own model page: Liquid.ai Models.
The claims are directionally about better efficiency, smaller memory footprints, better adaptability, and a different underlying architecture inspired by “liquid neural networks.”
As engineers, we should treat this like any vendor claim: interesting, worth tracking, but verify.
What “continuous-time” actually means (brief technical explainer)
Transformers are fundamentally discrete-step machines. You feed a sequence of tokens, run attention and MLP blocks in layers, and you get the next-token distribution. Time is implicit, represented by token positions. It’s “time as a sequence index.”
Liquid neural network lineage points toward models that look more like dynamical systems, where the internal state evolves according to differential equations. In simplified terms, transformers update state in chunks (layer-by-layer, token-by-token), while continuous-time models evolve state smoothly over time, often described by an ODE (ordinary differential equation).
If you’ve ever worked with control systems or physics simulators, this should feel familiar. Instead of “apply layer 17,” you have “integrate state forward.”
Why might that matter for continual learning? Because continual adaptation is, at its core, about state changing in response to new inputs over time. A continuous-time framing can make “ongoing adaptation” feel less like a bolt-on feature and more like the native behavior of the system.
That’s the promise, anyway. The proof is in evals.
What I want to see them publish (or what you should demand)
Liquid.ai’s page is currently more positioning than peer-reviewed detail, so here’s the concrete checklist I’d use to evaluate whether LFMs are a real continual learning path:
Adaptation tests: can the model update behavior from small amounts of new data without full retraining?
Forgetting controls: what prevents regression on older tasks?
Memory footprint numbers: actual VRAM, throughput, latency profiles at comparable quality.
Stability over time: do repeated updates lead to drift?
And if they want to win engineers, I’d love to see reproducible benchmarks, comparisons against strong transformer baselines, clear statements about update mechanisms (weights, memory, or hybrid), and safety constraints for online learning.
The reason I’m paying attention anyway is simple: if continual learning is the goal, post-transformer architectures might end up being the cleanest way to get there.
Transformers were designed for sequence modeling with attention, not for lifelong learning. You can bolt on memory. Or you can rethink the core compute.
7. If it works / if it breaks: compounding personalization vs drift, poisoning, privacy
If it works: personalization stops being a prompt engineering exercise
The most obvious product implication is personalized AI that actually sticks.
Not “I saved your preference in a settings file.”
I mean it learns your code review style, it learns your team’s architecture constraints, it learns that you migrated to Python 3.12, it learns your organization’s incident response patterns, and it stops making the same mistakes next month.
This is where agents become less theatrical and more useful. Agents today often fail because they don’t retain tool lessons, they repeat bad plans, and they forget what “done” looks like for your environment. A continual learning agent can, in theory, tighten that loop.
Enterprise example: customer support that improves daily, not quarterly
A use case I think will hit early is customer support.
Imagine a support assistant that works tickets all day. Every resolved ticket contains a problem description, a fix, a decision trail, and a final validated answer.
Today you can do RAG over that corpus, and it helps. But the assistant still mis-prioritizes evidence, repeats old wrong answers, needs curated knowledge base updates, and needs periodic fine-tunes.
With continual learning, the ideal loop is: ticket gets resolved (ground truth), the system scores the episode (surprise, novelty, impact), it promotes the key pattern into long-term memory, and tomorrow, a similar ticket comes in and the assistant is better.
That’s not just cost savings. That’s reliability compounding.
And it matters even more when you combine it with the workforce reality. If AI skill gaps are limiting innovation (as called out in the 2026 Workforce Trends Report | Leapsome), then systems that require less bespoke prompt tuning and less handholding become strategically valuable.
What could go wrong: drift, poisoning, governance
Don’t get me wrong, this doesn’t mean we should just turn on online learning and call it a day.
Model drift happens if recent interactions dominate, and the model can slide away from its general competence. This is recency bias at the memory or update level, not just the response level.
Data poisoning can be both accidental and malicious. If users can influence what gets stored, someone will try to game it. Even without attackers, normal usage can poison a model through incorrect internal docs, outdated runbooks, or “tribal knowledge” that’s wrong but repeated.
Privacy and retention become first-class concerns. If the system stores “surprising” facts, you need hard rules about what it’s allowed to store, how long it can store it, how it can be deleted, and how it’s audited. This is where “memory” stops being a feature and becomes a compliance surface area.
Evaluation becomes continuous. With static models, you can do a big eval suite and ship. With continual learning, evaluation needs to look more like operating a production distributed system: canarying, ongoing regression tests, drift detection, periodic memory scrubs, and rollback mechanisms for bad updates.
In other words, you stop shipping a model artifact and start operating a learning system.
Continual Learning in 2026: Why “Smart but Stuck” Models Might Finally Adapt
1. Cold open: 2026 as the “learning, not prompting” inflection
All right, so we just cruised into 2026.
And if you’ve spent any time building with LLMs in the last two years, you’ve probably felt the same friction I have: these models are impressive, but they’re weirdly static. They can reason, summarize, code, and argue. But they don’t learn the way you’d expect a capable system to learn.
There’s a claim making the rounds that I can’t stop thinking about: 2026 is going to be the year of continual learning. Not “better prompts”. Not “bigger context”. Not “more agents”. Actual learning over time.
I recently went down a rabbit hole that split into two tracks. One is coming out of Google/DeepMind research, with nested learning and memory-centric architectures like Titans and HoPE (I started from this talk and followed the citations from there: YouTube). The other is Liquid.ai’s post-transformer bet, where they’re explicitly pitching a different compute model: Liquid.ai Models.
This post is me expanding that thread, why today’s models feel smart-but-stuck, what “continual learning” really implies, and why these two camps might be the first real cracks in the “frozen weights forever” era.
2. Why models feel static: context, RAG, and “memory” aren’t learning
The “amnesia” problem isn’t UX, it’s architecture
There’s a question that keeps coming up when I talk to teams shipping LLM features: why do these models feel like they understand everything, yet they keep repeating the same mistakes?
A big part of it is structural: most deployed LLMs have frozen weights. They’re trained, evaluated, released, and then they stay that way until the next expensive training run or fine-tune cycle.
So yes, they can infer inside a session. They can adapt behavior via prompt conditioning. But they don’t actually update the underlying model in a durable way.
That distinction matters more than most product conversations admit, because it’s the difference between “the system can follow instructions while you remind it” and “the system got better because it experienced something.”
Context windows are working memory, not learning
We’ve all seen the progress on context length. Some models can hold an absurd amount of tokens now. You can paste in a repo, a runbook, and half a novel, and it will keep a coherent thread.
But even a massive context window still behaves like short-term working memory. It can use what’s in the window, it can chain reasoning across it, and then it’s gone. That’s not a knock. Working memory is useful. But it’s not learning.
If anything, big context windows can trick us into thinking the system is improving. It’s not. You’re just carrying more scaffolding forward each time.
“Memory” features are mostly a second system bolted on
A lot of chat products now have a “memory” feature. You see behaviors like summarizing your preferences, storing a few user facts, and retrieving those facts later. It’s useful and it works most of the time.
But most of these implementations are a separate storage and retrieval layer plus some heuristics about what to store, not the model internalizing new skills or facts into its parameters, or into a first-class adaptive state with clear stability constraints.
So the product feels better, but the underlying model is still… stuck.
RAG is looking up a textbook, not learning the skill
RAG is another case where we sometimes overstate what’s happening. With RAG, we say: “Don’t memorize everything. Retrieve relevant documents at runtime.” That’s great engineering. It’s also not learning.
A clean way to think about it: RAG is open-book, while continual learning is actually learning the material. If I keep asking you the same question and you keep looking it up in a binder, you’re still useful. You might even be more useful than someone relying on memory. But you haven’t improved, you’ve just gotten good at searching.
And the failure mode is predictable: if retrieval misses, if the embedding is off, if the doc is stale, you regress immediately.
A more realistic failure: the personalized coding assistant
The goofy “emoji hallucination” examples are funny, but engineers feel the pain more sharply in coding workflows.
Here’s a scenario I’ve watched play out repeatedly:
You use an AI coding assistant for weeks. You switch stacks (say, from Node 16 + CommonJS to Node 22 + ESM, or from Python 3.9 to Python 3.12). You settle on a style too: stricter typing, different lint rules, a new framework preference, more explicit error handling, fewer magical abstractions.
What happens? It helps… as long as you keep reminding it. Next session, it suggests patterns you abandoned. It forgets the constraints that mattered yesterday.
So you end up building a second system around it: system prompts, “project rules” documents, pinned context in every chat, custom RAG over your repo, sometimes fine-tuning if you’re serious. That works. But it’s the same underlying story: the model itself isn’t adapting. You’re doing the adaptation work externally.
The business tax: retraining is slow and expensive
This is where the “stuck” feeling turns into a real business constraint.
If the only durable way to update behavior is retraining or frequent fine-tunes, you’re paying for data pipelines (collection, cleaning, labeling), training compute, evaluation cycles, rollout risk, and regression handling. And even when you’re not training a frontier model, the economics show up proportionally. Big updates happen in batches, not continuously.
That’s why I think continual learning is getting pulled forward by pressure, not hype. It’s the same reason teams adopt streaming architectures. The world doesn’t update quarterly.
Also, this pressure isn’t just “AI teams want nicer models.” The skills gap is real, and it’s limiting how fast organizations can respond. According to the 2026 Workforce Trends Report | Leapsome, organizations are explicitly calling out AI skill gaps as a blocker to innovation. When the people side can’t scale fast enough, the value of systems that adapt faster (with less bespoke tuning) goes up.
3. Continual learning, translated: plasticity vs stability and update cadence
The “2026” framing that stuck in my head
The framing I saw was attributed to Ronak Mald (Google DeepMind / reinforcement learning background): 2024 was the year of agents, 2025 was the year of reinforcement learning, and 2026 will be the year of continual learning.
People argued about the first two years, and honestly, that’s fair. Most teams I talk to didn’t experience 2024 as “agents everywhere”. They experienced it as “ChatGPT, copilots, and a lot of prototypes.”
But here’s where it gets interesting: the direction of the claim feels right even if the calendar is off by 6 to 18 months.
Because the limiting factor right now is not raw capability. It’s adaptation.
Fluid vs crystallized intelligence is a useful analogy (if we don’t get weird about it)
I like the brain analogy here, as long as we don’t push it too far. Crystallized intelligence is drawing on stored knowledge and experience, while fluid intelligence is adapting quickly to new situations with limited examples.
Today’s LLMs feel like they have a ton of crystallized intelligence. They can talk about everything. They can pattern match like monsters.
But they’re weak on fluid intelligence in a very specific engineering sense: they don’t update their internal world model from ongoing experience in a durable way.
Continual learning is not “fine-tune more often”
In ML terms, continual learning is the idea that a model can acquire new skills and facts over time, without catastrophic forgetting (overwriting what it previously knew), ideally without requiring full retraining.
That “without forgetting” clause is the whole problem.
If you naively train on new data, you often get drift into the new distribution, degradation on older tasks, and regressions that feel like the model “lost” skills.
So continual learning is really a balancing act between plasticity (ability to learn new things) and stability (ability to retain old things). If you’re an engineer, think of it like operating a live system while doing schema migrations. You want change, but you want controlled change. You want rollbacks. You want invariants.
Update cadence is the hidden design decision
One more translation that matters: continual learning isn’t one thing. It’s a spectrum based on update frequency. Some systems “learn” once per week through batched updates. Some aim for session-level updates. Some aim for near real-time updates.
The tighter the loop, the more useful it gets for personalization and agents. And also, the more dangerous it gets.
Because the moment you let a system update frequently, you inherit all the messy operational realities we normally avoid by freezing weights: debugging becomes time-dependent, reproducibility becomes harder, and incidents can be caused by “learning,” not just code deploys.
Also, the economics matter. Training and retraining costs are now significant enough that investors are paying attention to upskilling and efficiency as a macro trend. The Strategic Institutional Investment in EdTech: Capitalizing on 2026 Upskilling Trends report frames AI upskilling as a capital allocation theme, which is a polite way of saying: “this is expensive, and the market knows it.”
Continual learning is partly a capability play, but it’s also a cost-structure play.
4. Google/DeepMind direction: nested learning and surprise-gated promotion
The core idea: two loops, two timescales
Google Research has been publishing in the direction of continual learning paradigms that look a lot like “nested learning,” meaning you have two learning loops operating at different timescales (this is the thread I followed from the talk: YouTube).
The intuition is close to how you’d describe human memory in casual terms: a fast loop that adapts quickly but doesn’t keep everything, plus a slow loop that stores what matters for the long haul.
In engineering terms, it’s like a write-heavy cache with aggressive eviction, plus a durable store with stricter write rules. We already have something like the fast loop in LLM products: the context window. It’s short-term working memory. It’s cheap to write, easy to overwrite, and it disappears at the end of the session.
But the missing piece is the bridge into durable memory.
What counts as “important” enough to store?
This is the part that made me lean in.
If you’re going to build a model that updates continuously, you need a gate. A policy that decides what gets stored, what gets discarded, what gets reinforced, and what gets forgotten.
One of the criteria that shows up in this line of thinking is surprise. Surprise, in this context, is basically the gap between the model’s prediction of the world (its current world model) and what actually happens (new evidence).
When the gap is big, you need an update.
That maps nicely to how humans work. If you tell me something I already assumed, it doesn’t stick. If you tell me something that forces me to rewrite part of my internal model, I remember it.
The honey example as a “memory write event”
The honey story is a perfect illustration of surprise gating.
Most people know honey comes from nectar. Not surprising.
But when you hear the full mechanism (nectar processed by bees, dehydrated in combs), it triggers a “wait, what?” moment. That’s surprise. That’s a write.
That’s what a continual learning system needs: a way to detect a meaningful prediction error and decide it deserves long-term storage. And importantly, it’s not just “store the fact.” It’s store it in a way that can influence future behavior without destabilizing everything else.
Why this matters more than bigger context windows
Bigger context windows help with short-term coherence. They’re great for long document QA, repo-level reasoning, and multi-step workflows that need lots of state.
They do not solve recurring user preferences, tool usage patterns over months, domain-specific facts that change weekly, or incremental skill acquisition.
Nested learning is interesting because it’s trying to formalize the pipeline: short-term experience gets filtered, compressed, and selectively promoted into durable memory.
If that works, it’s a step toward something we don’t really have today: models that “live” in the world and get better as they interact with it, without needing a quarterly retrain to become competent again.
5. Titans and HoPE: storing memories vs maintaining and reshaping them
Titans: the “filing cabinet” model of long-term memory
If nested learning is the paradigm, Titans (a memory-centric architecture) reads like one concrete attempt at building the mechanism.
The way I internalize Titans is exactly how it was described in the narrative: a filing cabinet. You observe events in the fast loop, you score them for importance (surprise is one signal), you file the important ones into a long-term store, and later, you retrieve them when relevant.
That’s already a big step forward from pure “context window amnesia.” Because now the model can carry forward useful state across time.
In product terms: the assistant doesn’t just “remember” because you pasted it in again. It remembers because the system chose to store it. And as engineers, we know why that matters. It creates an explicit mechanism you can test, tune, and govern.
HoPE: the “reorganizing brain” model (store, reshuffle, forget)
HoPE (described as a variant built on top of these ideas) adds a different flavor.
The key shift is: it’s not just storing memories. It’s self-modifying in a recurrent way, with repeated internal loops that can reorganize memory over time.
That matters because long-term memory isn’t just a write-once database. In humans, memory is strengthened when used, weakened when unused, and sometimes rewritten when new evidence conflicts.
So the “infinite looped learning levels” language maps to something like repeated consolidation passes, re-ranking of stored memories, and forgetting mechanisms based on usage and relevance.
In practice, this is the part that tends to separate “cool demo memory” from “usable continual learning system.” Because if you never forget, you fill up with junk. If you forget too aggressively, you lose stability. If you rewrite incorrectly, you drift.
Why engineers should care: the failure modes change
If you build on static models today, your failures look like:
If you build on continual learning models tomorrow, your failures change shape:
So yes, this is exciting. It’s also the kind of thing that forces us to build new guardrails.
A quick reality check on benchmarks
People love asking: “What’s the delta? How much better is it?”
The uncomfortable truth is that a lot of the most important continual learning wins aren’t captured by a single headline benchmark the way “MMLU +4” is.
The interesting metrics are usually things like retention over sequential tasks, performance under distribution shift, and ability to incorporate new facts without degrading older competencies.
If you’re tracking this space, look for papers that report sequential learning curves, forgetting metrics across tasks, and ablation studies on memory gating signals like surprise. Those are the “SLOs” for continual learning.
5.5 Google vs Liquid.ai: evolution vs revolution
Both camps are aiming at the same pain: models that don’t adapt. But the architectural philosophy is different enough that it’s worth making it explicit.
The evolutionary path (Google/DeepMind)
Google’s approach keeps transformer-style reasoning as the backbone and adds memory plus gated promotion on top. “Learning” in this paradigm looks like memory-centric continual learning: store, retrieve, consolidate. Experiences get promoted into durable memory, possibly with consolidation loops that reorganize stored knowledge over time.
The strengths here are clear: it’s a more incremental path from today’s deployments and easier to integrate with existing tooling. The risks are equally clear: memory governance becomes critical, drift can happen via memory accumulation, and the complexity of consolidation policies can spiral.
If you’re evaluating this direction, watch for how well surprise gating works in practice, what controls exist for forgetting, and whether sequential-task evaluations hold up.
The revolutionary path (Liquid.ai)
Liquid.ai’s bet is more fundamental. They’re proposing that the transformer may be the wrong base abstraction for lifelong learning, so they’re changing the base compute entirely. Their Liquid Foundation Models (LFMs) are inspired by continuous-time dynamics, where internal state evolves according to differential equations rather than discrete layer-by-layer steps.
In this framing, “learning” is potentially state dynamics that naturally adapt over time. The adjustment happens via dynamical system behavior, possibly enabling smoother adaptation than memory promotion alone.
The strengths here are theoretical: if the architecture fits the problem, it could be more efficient and better aligned with continual adaptation from the ground up. The risks are equally real: it’s harder to validate, harder to compare apples-to-apples with transformer baselines, and success depends on strong evidence and reproducibility.
If you’re evaluating this direction, look for adaptation tests, stability under updates, and clear baselines showing where LFMs outperform transformers on continual learning tasks.
The strategic read
If you’re an engineering leader making bets, the difference feels like this: Google’s route is evolutionary (extend what works, add a memory pipeline, and make “learning” a controlled subsystem), while Liquid.ai’s route is revolutionary (the transformer may not be the right foundation for lifelong learning, so change the foundation).
Both can be right. Or both can fail for different reasons.
6. The other bet: Liquid.ai and what to verify beyond marketing
Liquid.ai is explicitly pitching “beyond transformers”
After going deep on Google’s memory-centric direction, I looked at Liquid.ai because it’s a different kind of bet.
They’re not just saying “add memory modules to transformers.”
They’re marketing Liquid Foundation Models (LFMs) as a different architectural direction, according to their own model page: Liquid.ai Models.
The claims are directionally about better efficiency, smaller memory footprints, better adaptability, and a different underlying architecture inspired by “liquid neural networks.”
As engineers, we should treat this like any vendor claim: interesting, worth tracking, but verify.
What “continuous-time” actually means (brief technical explainer)
Transformers are fundamentally discrete-step machines. You feed a sequence of tokens, run attention and MLP blocks in layers, and you get the next-token distribution. Time is implicit, represented by token positions. It’s “time as a sequence index.”
Liquid neural network lineage points toward models that look more like dynamical systems, where the internal state evolves according to differential equations. In simplified terms, transformers update state in chunks (layer-by-layer, token-by-token), while continuous-time models evolve state smoothly over time, often described by an ODE (ordinary differential equation).
If you’ve ever worked with control systems or physics simulators, this should feel familiar. Instead of “apply layer 17,” you have “integrate state forward.”
Why might that matter for continual learning? Because continual adaptation is, at its core, about state changing in response to new inputs over time. A continuous-time framing can make “ongoing adaptation” feel less like a bolt-on feature and more like the native behavior of the system.
That’s the promise, anyway. The proof is in evals.
What I want to see them publish (or what you should demand)
Liquid.ai’s page is currently more positioning than peer-reviewed detail, so here’s the concrete checklist I’d use to evaluate whether LFMs are a real continual learning path:
And if they want to win engineers, I’d love to see reproducible benchmarks, comparisons against strong transformer baselines, clear statements about update mechanisms (weights, memory, or hybrid), and safety constraints for online learning.
The reason I’m paying attention anyway is simple: if continual learning is the goal, post-transformer architectures might end up being the cleanest way to get there.
Transformers were designed for sequence modeling with attention, not for lifelong learning. You can bolt on memory. Or you can rethink the core compute.
7. If it works / if it breaks: compounding personalization vs drift, poisoning, privacy
If it works: personalization stops being a prompt engineering exercise
The most obvious product implication is personalized AI that actually sticks.
Not “I saved your preference in a settings file.”
I mean it learns your code review style, it learns your team’s architecture constraints, it learns that you migrated to Python 3.12, it learns your organization’s incident response patterns, and it stops making the same mistakes next month.
This is where agents become less theatrical and more useful. Agents today often fail because they don’t retain tool lessons, they repeat bad plans, and they forget what “done” looks like for your environment. A continual learning agent can, in theory, tighten that loop.
Enterprise example: customer support that improves daily, not quarterly
A use case I think will hit early is customer support.
Imagine a support assistant that works tickets all day. Every resolved ticket contains a problem description, a fix, a decision trail, and a final validated answer.
Today you can do RAG over that corpus, and it helps. But the assistant still mis-prioritizes evidence, repeats old wrong answers, needs curated knowledge base updates, and needs periodic fine-tunes.
With continual learning, the ideal loop is: ticket gets resolved (ground truth), the system scores the episode (surprise, novelty, impact), it promotes the key pattern into long-term memory, and tomorrow, a similar ticket comes in and the assistant is better.
That’s not just cost savings. That’s reliability compounding.
And it matters even more when you combine it with the workforce reality. If AI skill gaps are limiting innovation (as called out in the 2026 Workforce Trends Report | Leapsome), then systems that require less bespoke prompt tuning and less handholding become strategically valuable.
What could go wrong: drift, poisoning, governance
Don’t get me wrong, this doesn’t mean we should just turn on online learning and call it a day.
Continual learning introduces failure modes we’ve historically avoided by freezing weights.
Model drift happens if recent interactions dominate, and the model can slide away from its general competence. This is recency bias at the memory or update level, not just the response level.
Data poisoning can be both accidental and malicious. If users can influence what gets stored, someone will try to game it. Even without attackers, normal usage can poison a model through incorrect internal docs, outdated runbooks, or “tribal knowledge” that’s wrong but repeated.
Privacy and retention become first-class concerns. If the system stores “surprising” facts, you need hard rules about what it’s allowed to store, how long it can store it, how it can be deleted, and how it’s audited. This is where “memory” stops being a feature and becomes a compliance surface area.
Evaluation becomes continuous. With static models, you can do a big eval suite and ship. With continual learning, evaluation needs to look more like operating a production distributed system: canarying, ongoing regression tests, drift detection, periodic memory scrubs, and rollback mechanisms for bad updates.
In other words, you stop shipping a model artifact and start operating a learning system.
Further Reading
Archives
Categories
Archives
Recent Post
From Code-First to Requirements-First: How LLMs Are Inverting Software Engineering
January 18, 2026Triple Insight #2 – Learning Loops
January 18, 2026Continual Learning in 2026: Why “Smart but Stuck” Models Might Finally Adapt
January 18, 2026Categories
Meta
Calendar