∇ Guide: Test-Time Training – A Model That Changes While You Use It
What is Test-Time Training?
Test-time training (TTT) is an ML technique where a model continues to learn and adapt even during the test phase, rather than just making predictions with a fixed set of learned weights. It allows the model to fine-tune itself on the test data before making predictions, improving its accuracy. Such adaptability enhances the models' ability to handle unforeseen scenarios and distribution shifts.
New approaches apply this idea to LLMs, agents, and world models, turning inference into an active learning process that can respond to new environments and distribution shifts.
Housekeeping, folks: AI 101 is becoming ∇ Guide and moving to the end of the week. We’re going beyond the basics of AI concepts, models, and techniques, connecting these explanations to our Almanac. The symbol is called nabla, used in math to express a gradient. A small nod to how models learn, as we keep learning ourselves.
Today we want to start from a quick analysis of OpenAI’s Navier–Stokes experiment that can give you more insights about inference, or test-time, stage that you can imagine. First of all, it gives us a picture of what happens when models are allowed to do much more work at inference.
Image Credit: On the Navier–Stokes Millennium Prize Problem (OpenAI)
The chart compares GPT‑6 Astra with OpenAI’s next-generation internal model on a set of open math problems. Two things are happening at once. The new model starts from a much higher level, and additional test-time compute keeps pushing both models further. Part of the leap comes from training a stronger model, but another part comes from giving it more time, attempts, and compute after it receives the problem. In the Navier–Stokes run, around 10,000 agents spent 88 hours exploring different approaches, sharing useful findings, and improving performance through context, coordination, tools, and test-time compute.
But more globally, there is a larger shift in inference – it has stopped being one fixed model producing one answer. Current studies have turned it into an active process that can continue for hours and improve its route to the solution. But we are mostly improving the process around the model. What about the model’s evolution during inference?
When we explored test-time compute and test-time scaling to write an article about it (that was more than a year ago!), we saw that one of the potential directions for improving a model’s reasoning during inference was test-time training (TTT). We haven’t seen a boom in test-time training yet, but we’re at an interesting point where researchers are starting to explore it and what it can do.
While test-time compute means using more computation during inference while keeping the model’s weights fixed, test-time training means the model continues adapting or updating itself during the test phase. TTT challenges the traditional idea that learning ends when a model is deployed.
Here, the line between training and inference begins to blur. TTT moves toward systems that can adjust to new conditions while they are being used, which is especially useful when data changes rapidly. And that’s why it can bring great benefits to world models, which are all about fast adaptation to new states, actions and environment.
Today, we will unpack what test-time training is, how it works, where it is already used, and how it upgrades world models’ capabilities in real-world tasks. Let’s go! It will be interesting.
In today’s episode:
The essential definitions
What is Test-Time Training?
How Test-Time Training works
Studies That Mark Test-Time Training as an Emerging Trend
Test-Time Policy Optimization (TTPO)
Agentic Test-Time Training (aTTT)
World Model and Test-Time Training: Why It Is a Very Important Match
World Models Using Test-Time Training
WAM-TTT: Test-Time Training While Watching Humans
What is Test-time Mixture of World Models (TMoW)?
Not without limitations
Conclusion: Everything is a puzzle piece
Sources and further reading
Not interested in this topic? Check our latest video: Meta just gave its new personal agent, Muse, a remarkable amount of freedom. But Meta made one part of the system deliberately difficult for Muse to control: its own authority. In this episode of Attention Span, I look inside Muse Secure VM and the security architecture surrounding the agent.
Follow us on
The essential definitions
Before we move on to the core explanation of test-time training, we need to check all the definitions that will help us understand it (if you already know them, you can skip this section).
Training time – the stage when a model learns from a training dataset by updating its parameters.
Inference or test time – the stage when the trained model receives new data and produces predictions. Traditionally, its parameters remain fixed at this stage.
Model parameters – the internal values, or weights, learned during training. TTT updates some of these parameters during inference.
Training distribution – the patterns and conditions found in the data used to train the model.
Test distribution – the patterns and conditions found in the new data the model encounters after deployment.
Distribution shift – a difference between the training and test distributions. For example, new lighting, image noise, an unfamiliar accent, or a different sensor.
Self-supervised learning – learning without human-provided labels by creating a task from the data itself, such as reconstructing masked content or predicting an image’s rotation.
Main task – the prediction the model is actually expected to make, such as recognizing an object, transcribing speech, or forecasting a sequence.
Auxiliary task – the self-supervised task used to adapt the model at test time. It is not the model’s final task, but its updates should help a model to complete that task.
Auxiliary loss – a measure of how poorly the model performs the self-supervised task. TTT uses this error as a signal for updating the model.
Test-time adaptation – the process of adjusting the model to new test data.
TTT is one way to do test-time adaptation through parameter updates and self-supervised learning. Now let’s see how this happens in general.
What is Test-Time Training?
The main basics of test-training (TTT) were explained in the 2020’s paper by Yu Sun and other researchers from University of California, Berkeley. It is called “Test-Time Training with Self-Supervision for Generalization under Distribution Shifts”, and it describesTTT as a broad approach in which a model adapts while it is being used.
TTT follows the core idea that inference does not have to be passive. Training can continue during inference to improve a model’s performance on completely new data that it meets while operating in the real world.
How Test-Time Training works
TTT begins after a model has already been trained and deployed. Its goal is tolet the model adjust to the data currently in front of it. Imagine new data now arrives from the environment where the model is being used, for example, an image uploaded by a user, a frame from a robot’s camera, or a piece of recorded speech.
The model needs to make a prediction about this new input, which is its main task. But it doesn’t know the correct answer and so can’t learn from like it learned from a normal labeled training task. The model needs to take a different route here, and TTT helps it do that. The entire process looks like this:
The TTT system creates a separate, self-supervised task from the same new and sends it to the model. It is called an auxiliary task and answers to it can be generated automatically. Since the answer is generated from the input itself, the model already knows it and doesn’t need a human-provided label.
The model then makes a small update to some of its parameters. It then uses its own mistakes on this auxiliary task as a learning signal to update the parameters responsible for its shared features before making the actual prediction.
Depending on the TTT method, it updates either part of the main model or a small set of parameters like adapters or biases.
Only then the adapted model can make the main prediction.
Note that the model doesn’t learn the object’s true label from the test example, it just uses a related task to better understand the data before making its prediction.
This process will become clearer with an example from the original Yu Sun’s paper:
There, the main task was image classification, and the self-supervised task was rotation prediction. Before classifying a new image, the system rotated it by 0°, 90°, 180°, or 270° and asked the model to identify the angle. If the system rotates that image by 90°, it knows that the correct rotation label is 90°. And since the system applies the rotation itself, it already knows the correct answer and doesn’t need a human-provided label.
The model uses its mistakes on this auxiliary task to update its shared visual features. These features are also used for object recognition, so adapting them to the new image could improve the final classification.
TTT can work in several ways:
Standard TTT: the model adapts separately to each example and then resets.
Online TTT: the model keeps the update and continues adapting as more examples arrive. This option can be especially useful when data comes from the same environment or when conditions change gradually.
The model learns from a batch or sliding window of recent data.
The model can update only a small number of parameters to reduce the cost.
It is crucial to keep this in mind: A test example also contains information about the environment the model is currently operating in. TTT uses that information to make the model better suited to the data in front of it.
Although the original work is focused on image recognition, TTT has already been extended to segmentation, computer vision, time series, speech, restoring noisy audio, graphs, language, robotics, world models and many other areas. Now let us explore some advanced examples (including agentic and world models TTT!).
Studies That Mark Test-Time Training as an Emerging Trend
Don’t settle for shallow articles. Learn the basics and go deeper with us. Truly understanding things is deeply satisfying.
Join Premium members from top companies like Microsoft, NVIDIA, Google, HF, OpenAI, a16z, plus AI labs such as Ai2, MIT, Berkeley, .gov, and thousands of others to really understand what’s going on in AI.
How did you like it?
FAQ
What is test-time training?
Test-time training (TTT) is a method that lets a model continue adapting during inference. Instead of relying only on what it learned before deployment, the model uses new test data to update some of its parameters before making a prediction.
How does test-time training work?
TTT creates a self-supervised auxiliary task from new test data, so the model can get a learning signal without human-provided labels. It uses its errors on this task to update some parameters, then makes the main prediction with the adapted model.
What is the difference between test-time training and test-time compute?
Test-time compute gives a model more computation, attempts, or reasoning during inference while keeping its weights fixed. Test-time training goes further by actually adapting or updating parts of the model during inference.
Why is test-time training useful for world models?
World models operate in environments that can constantly introduce new states and conditions. During long rollouts, their predictions can drift away from reality. TTT lets them learn from information encountered during deployment, helping them adapt to conditions that were not fully represented in training.
What are the limitations of test-time training?
TTT only helps when its learning signal is useful for the model’s main task. Poor auxiliary tasks or uncontrolled updates can hurt performance, cause overfitting or model drift, and test-time updates also add extra inference compute.