TL;DR
Get business pricing on tech for your team
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
The Terminal-RSI preprint reports that a 27-billion-parameter model improved on several terminal-agent benchmarks after five stages combining examples from its own verified successes with reinforcement learning. Researchers designed and ran the stages, and the results remain limited: the final model averaged 5 of 46 tasks on a long-horizon benchmark and showed little gain on a held-out suite after its third stage.
The paper reports an equal-weight average across four Terminal-Bench suites of 50.58% for the final model, compared with 20.68% for the base model. The individual reported results include 16.95% on Terminal-Bench 3.0, 86.52% on Terminal-Bench 2.1 and 79.00% on Terminal-Bench Hard, compared with base scores of 0.00%, 51.70% and 31.00%, respectively. Each score is the arithmetic mean of three evaluation runs.
Training alternated two methods: Recursive Self-Rewrite (RSR), which turns the model’s verifier-approved attempts into screened training examples, and Recursive Policy-Refine (RPR), a reinforcement-learning stage. The sequence was Base, RSR-1, RPR-1, RSR-2 and RPR-2. Researchers selected the stages, reward design, learning rates and checkpoints. The starting point was the open Qwen3.8-27B model, and the training pool contained 3,000 tasks.
The authors also report a control in which reinforcement learning applied directly to the base model scored 0.00% on Terminal-Bench 4.0 at every recorded checkpoint. In their sequence, reinforcement learning after the first self-rewrite stage reached 16.67% on that benchmark. The paper attributes the pipeline’s results to the combination of its training stages and reward design; the reported control suggests the order of training may matter.
Terminal-RSI’s benchmark gains came through human-directed training
Researchers combined verified successes with reinforcement learning across five stages. The preprint reports measurable gains, but does not show an AI improving itself without people.
What the result shows
The reported benchmark gains followed a training pipeline designed and run by researchers.
- Training stages and their sequence
- Rewards, learning rates and checkpoints
- Whether the task’s final state passed hidden tests
- Which attempts counted as successful examples
Five stages, two methods
The pipeline alternated learning from verified attempts with reinforcement learning.
Started from the open Qwen3.8-27B model.
Turned verifier-approved attempts into screened training examples.
Applied reinforcement learning; Terminal-Bench 4.0 reached 16.67%.
Repeated the verified-example training method.
Refined the policy again to produce the final model.
Where the evidence is limited
Results on the paper’s benchmarks do not settle how well the approach transfers to unfamiliar work.
Few tasks completed
The final model averaged 5 of 46 tasks, compared with 0 of 46 for the base model.
Small gain, then flat
SWR100 rose from 3.00% to 6.00%; the result had been flat since stage three.
Three runs per score
The preprint does not establish performance across more runs or independently selected task sets.
Verified Completion Shapes Agent Training
The work addresses a practical weakness in command-line agents: a model can stop early or claim a task is finished without producing the requested result. Terminal-RSI uses hidden tests of the machine’s final state to determine success. The paper says an attempt counts only if the complete protected verifier passes, rather than on the agent’s own declaration or a partial set of checks.
That distinction also affects what the reported improvement means. The model’s progress came through repeated training choices made by people, while the verifier supplied a measurable signal for which attempts succeeded. This offers a concrete example of how verified outcomes can support agent training, but the benchmarks do not establish that the system can improve its own capabilities autonomously or reliably handle open-ended work.
Products Worth Considering
Samsung 49-Inch Odyssey G9 G91F Curved Dual QHD Gaming Monitor, 144Hz
Samsung Odyssey G9 49-Inch Curved Dual QHD Gaming Monitor
Logitech MX Keys S Combo Wireless Keyboard and Mouse, Graphite
As an affiliate, we earn on qualifying purchases.
How Terminal-RSI Was Trained
A terminal agent operates through a command line: it reads files, runs commands, inspects outputs and tries to complete tasks such as repairing a build or changing a data pipeline. These tasks can involve multiple steps, and a plausible progress report does not prove the final state is correct.
In RSR, the model attempts tasks and keeps only attempts accepted by the verifier. It rewrites a successful attempt into a plan, which is screened for leaked answers and then executed again from scratch. Attempts that pass again become supervised fine-tuning examples. In RPR, the model receives scores for task attempts and adjusts its behavior; a critic network estimates how well an attempt is going. The researchers carried that critic across stages and recalibrated it before each refinement phase.
The paper describes a reward-design change intended to distinguish completion from unsuccessful activity. Under the earlier reward, failed attempts received 50.09% of all reward. The revised reward placed every failure below every success. The authors also report 748 failed runs that declared completion within ten turns, illustrating why they relied on external verification.
“The most interesting part is not the “self” in the title. It is how the researchers stopped rewarding the agent for looking busy and started rewarding it for actually finishing.”
— Thorsten Meyer AI, source article
Limits of the Reported Results
The reported benchmark gains do not settle how well the system would perform on tasks outside the paper’s evaluation sets. On a long-horizon benchmark, the final model completed an average of 5 of 46 tasks across three runs, up from 0 of 46 for the base model. On SWR100, a set of tasks held out from training, the final score was 6.00%, compared with 3.00% for the base model; the source says that result had been flat since the third training stage.
The preprint reports results from three runs per score, but the material provided does not establish how performance would vary across more runs, other model families or independently selected task sets. It also does not show that the system chooses its own research program or conducts its training stages without people. The scope of any broader capability or reliability improvement remains unclear.
Further Tests Will Set The Scope
The immediate next step is to assess whether the reported gains hold on more tasks, with additional runs and evaluations beyond the benchmarks used in the paper. The preprint’s results are based on version 3, dated Oct. 30, 2026; the material provided does not specify a later publication or independent replication.
For now, readers can compare the reported stage-by-stage scores and the control experiment while treating broader claims about autonomous self-improvement as unproven. Further evaluations would clarify how much of the improvement transfers to new tasks and whether the verifier-based approach supports reliable completion over longer sequences of work.
Key Questions
Did Terminal-RSI improve itself without humans?
No such result is established in the preprint. The five training stages were designed and run by researchers, who also chose rewards and checkpoints.
What is Terminal-RSI?
It is a terminal-agent training pipeline described in a preprint. It alternates learning from the model’s verifier-approved attempts with reinforcement learning.
How much did the model improve on Terminal-Bench 4.0?
The paper reports an increase from 0.00% to 19.84%. Each score is the mean of three evaluation runs.
Did the final model solve most long-horizon tasks?
No. It completed an average of 5 of 46 tasks on the reported long-horizon benchmark, compared with 0 of 46 for the base model.
How did the researchers check whether a task was complete?
They used a protected verifier that checked the final state. The paper says success required the complete verifier to pass, rather than accepting the agent’s own completion claim.
Source: Thorsten Meyer AI
![Kimi-K3 Technical Report [Pdf]](https://technocapture.com/wp-content/uploads/2026/07/kimi-k3-technical-report-pdf-featured-380x200.jpg)
