What bottlenecks
Robot RSI?

A Controlled Study of Models, Skills, Feedback, and Curriculum in Agentic Robotics

Kaiyuan Chen¹, Xinyue Zhu¹, Shuangyu Xie¹, Uksang Yoo¹, Thi Thu Phuong Luong², Marcus Gualtieri², Christian Juette², Liu Ren², Spencer Huang³, Ken Goldberg¹

¹University of California, Berkeley²Bosch³NVIDIAarXiv:2611.09030

In robot recursive self-improvement (RSI), a coding agent writes a robot policy as a program, runs it in simulation, reads what happened, and rewrites it.

Self-learning is limited less by the availability of useful information than by the agent's ability to turn the information into effective policy updates.

The task: hang the wrench on the pegboard hook by its ring.

▸ Hover a round below to see what happened in it.

held-out layouts hungexample run

Both programs, step by step

✓ step reported ok robot still: looking, planning

One run · Opus 5 · Tool Hanging

each dot is one of the 4 training layouts; k0 is the first program, k4 the one kept

● hung not hung the agent edited the program round reverted

The same six steps in both programs, and every one reported ok. Only one wrench ends up on the hook. The clips replay at 4× speed, synced step by step.

We hold the loop fixed and vary one part at a time (the model, the skills, the feedback and the curriculum) across 9 manipulation tasks, 631 experiment configurations and 28K coding-agent rollouts.

9manipulation tasks

631experiment configurations

28Kcoding-agent rollouts

Part 1 · The problem

Robot failures don't explain themselves

In software, a failing test points at a line of code. In a robot, where you see a failure () is often far from where it was decided (). Three kinds of failure come up again and again.

a step of the programthe robot stood still✕ where you see it · ◆ where it was decidedtraced back✓ the step reported ok

The same , three different

Tool Hanging

Manipulation planning: decided earlier, before the robot moved and on the approach.

Syringe Packing

Motion planning: decided by what the robot is holding.

Cable Routing

Deformable manipulation: decided everywhere.

Manipulation planning · Tool Hanging

▸ Scrub the ribbon, or ask where the miss was decided.

Show round 4's program

This episode comes back in Finding 4.

Side view along the hook · mm to scale

the wrench's ring, side-on, and its centreits paththe shank's axis, which the ring's centre has to reachround 4's ring, which hung, when shown

What you see: Every step reports ok. The ring reaches the hook's tip and stops just short of it; let go, the wrench drops beside the hook.
Where it was decided: Earlier: where perception put the ring, and the height chosen for crossing the tip. Manipulation planning has to choose a grasp and an approach that suit the placement at the end.

Opus 5's first program, one training layout.example run

Motion planning · Syringe Packing

▸ Scrub the ribbon, or ask where the drop was decided.

Show round 2's program

Section through the lid's opening

Schematic. Syringe length and opening width to scale; heights and tilt from the recorded poses.

the syringe in hand, or just let goearlier in its carrythe other syringeround 2's, when shown

tilt of the syringe

What you see: Every step reports ok. Each syringe is still flat when it is let go, and lands across the lid.
Where it was decided: At the grasp. From then on the 150 mm syringe is part of the moving robot, and carried flat it cannot pass the 55 mm opening. The motion has to account for the held object, obstacles and clearance together.

Opus 5's first program, one training layout.example run

Deformable manipulation · Cable Routing

▸ Scrub the ribbon and watch the cable's shape change with every move.

Top-down, as the camera sees it · mm to scale

the cableits shape when the let-go begangoal corridors

In this run the second arm's support step also returned held=False, so one arm routed the cable alone.

What you see: Pressed against the first peg, the cable springs back off when the arm lets go, woven round neither peg.
Where it was decided: Everywhere: the cable's shape and contacts change with every move, so motions that each look reasonable can still leave it in the wrong configuration.

Opus 5's first program, one training layout.example run

The evidence an agent needs is physical: geometry, contact, motion and change over time.

From the paper: Figure 1

Figure 1, left: manipulation planning — *Manipulation planning* fails to select a grasp and approach strategy compatible with the downstream placement.

Figure 1, centre: motion planning — *Motion planning* fails to jointly reason about held-object geometry, obstacles, and clearance to produce a collision-free trajectory.

Figure 1, right: deformable manipulation — *Deformable manipulation* requires reasoning over object configuration and contact that is difficult to capture through local motion decisions.

Figure 1: Representative failure modes in Agentic Robotics.

Part 2 · What we found

Watch one agent learn

Opus 5 gets one instruction, “hang the wrench on the hook”, and writes a program. Each round the program runs on 4 training layouts; the agent reads a report and edits its own code. Step through one real run. The study holds this loop fixed and changes one part at a time, the parts tagged on the loop, scoring held-out layouts on five VABAR and four MimicGen tasks.

▸ Step through the rounds of one real run.

Each tag is one finding: the part of the loop the study changed while holding the rest fixed. Click a tag to jump to it.

hung failed edit reverted edit applied to the restored program

Run

Reada feedback report

ring path while held where the gripper let go where the episode failed k4, hung layout , side view, to scale

Editthe agent rewrites its code

Edit size, every round

changed by this editwritten by the model itself, from no library skill

Tool Hanging, Opus 5, one run. Each round runs on the same 4 training layouts; the ring path is layout hangwrench-014, the one in the clips at the top of the page.example run

The tasks

The findings are scored on held-out layouts of five VABAR tasks; the curriculum study adds four MimicGen tasks. Hover, focus or tap a VABAR task to see its goal state.

From the paper: Figure 2

Overview: the agent initializes a policy, executes it in simulation, receives feedback and edits the policy; the study varies model, skill, feedback and curriculum — Figure 2. Overview of the controlled study: a coding agent initializes a policy π₀, executes the current policy π_k in simulation, receives feedback f_k, and iteratively edits its policy. The machine above redraws this loop with one real run.

Finding 1 · Model

Models look alike when they write the first policy. They separate when they repair it.

Overall held-out success over two rounds of the same execution feedback, and what each model's edits did.

▸ Step through the two rounds of feedback.

All models

Opus 5, Fable 5.1, GPT-6 Astra and Sonnet 5 all start at 43–45% overall success. After two rounds of the same execution feedback, Opus 5 reaches 74% and Fable 5.1 64%, while GPT-6 Astra and Sonnet 5 stay at 48% and 46%. Opus 5.5 goes from 67 to 77. Writing a plausible policy and repairing one from evidence are distinct capabilities.

The squares show where the difference comes from. Each square is one edit, classified by its outcome: fixed and success rose, fixed with no gain, the same failure, or a different failure. Some of Opus 5's and Fable 5.1's edits fixed the failure and raised success; none of Sonnet 5's or GPT-6 Astra's did.

● Plug seated

✕ Plug let go beside the port

Opus 5

Sonnet 5

example run

USB-C Insertion, the same training layout, after two rounds; one run per model. Opus 5's program seats the plug. Sonnet 5's aborts and lets go of it beside the port, where the clip ends. Replays at 4× speed, each row on its own time scale.

From the paper: Table 1 and Figure 4

Table 1. Self-learning on VABAR by model: held-out per-object success (%) reached within 0, 1 and 2 rounds of feedback (@k); Δk is the gain in overall success from the k-th round. Human: the expert-written policy.

Figure 4 of the paper: for each model, the share of its edits whose failure was fixed with a success gain, fixed without one, persisted, or moved to a different step; overall success after two rounds on the right — Figure 4. Edit outcomes per model, each edit classified by what the next training batch showed; the squares above draw the same data, one square per edit.

Finding 2 · Skills

Expert skills are a boundary for weaker models and a scaffold for stronger ones.

Each model's first program, by where every line came from.

▸ Take away everything the library provided.

Show only code the model wrote itself

How to read a field

Dashed fence: the skill library the agent was given

Inside, from the centre out

expert task skills
general skills the human also used
general skills the human did not use

Outside

new code the model wrote itself

1 dot ≈ 20 lines

@0 → @2 overall success: the first program, then after two rounds of feedback

After Human, models run from the lowest @2 to the highest.

dashed fence = the library · 1 dot ≈ 20 lines

Lines per first program, averaged over each model's runs per task and then over the tasks.

We traced every line of each first program to its source: the expert task skills, the general skills the human policy also uses, the remaining general skills, or none of them, which makes it new code. New code makes up 24% of the lines for Opus 5, 18% for Opus 5.5 and 12% for Fable 5.1; Sonnet 5, DeepSeek V4, Kimi K3 and GPT-6 Astra write only 4–7%.

The models that improve most with feedback are the ones that step outside the fence: they use the expert skills as a basis and add or change task-specific logic where the provided code is not enough. The others compose behaviour the library already has. Several of them start from similar success, so the difference is not the first program but whether the agent can modify and extend the skills it was given.

From the paper: Figure 5

Horizontal stacked bars of lines of code at k = 0 per model, split into task-specific skills, general skills used by the human, general skills unused by the human, and new skills; Human on top — Figure 5. Zero-shot policy composition and attribution by code source. The fields above redraw these bars as dots, in the order of success after two rounds.

Finding 3 · Feedback over time

Report what each skill did, not every motion.

In Tool Hanging the miss shows up at the reach, but it was decided earlier, by an estimate of where the wrench's ring is.

▸ Switch the report. Where does the agent look first?

Held-out success, by report

Tool Hanging7%45%90%first program 0%

Overall25%36%52%first program 3%

✕ where the failure shows · ◆ where the agent's first edit wentnot in this reportnever reached

The wrench from above, at the start of the episode

What the agent read

What it concluded

Where it edited, round by round

Each row is the edit made after that round of this run. changed that stage; changed perception, where the bad estimate was.

example run

The three reports describe the same failed episodes of the same first program; only the granularity differs. With one report per episode the agent kept editing the reach and reached 7% held-out success; a report per skill exposed the bad estimate and reached 90%.

A report per motion has no line for steps that do not move the robot, such as perception; there the agent had to infer the estimate from the numbers (45%). Overall: 25, 36 and 52%, from 3% for the first program.

When the cause is visible at the end, coarse reports suffice: in Syringe Packing, per-episode feedback reaches 90%.

Held-out success

How far back each edit reached

Skills traced back from the failing skill before each edit, one dot per edit; mean 2.0 per episode, 2.9 per motion, 3.3 per skill.

From the paper: Figure 6 and Table 2

Histogram of skills traced back before each edit, per feedback granularity — Figure 6. Number of skills traced back from the observed failing skill before each post-failure policy edit.

Table 2. Effect of temporal feedback structure on held-out success (%).

Finding 4 · Feedback in space

Agents use object-relative geometry when it is written out, not when they must derive it.

The same failed moment, written four ways. It is the last frame of the “k0 · first program” clip at the top of this page.

▸ Step through the formats. How much is left for you to work out?

Tool Hanging15%25%63%90%

Overall19%30%40%52%

The k0 clip's last frame, close on the hook and the wrench: the wrench stands upright below the hook, its lower end still in the gripper — Side view along the shank, to scale, from the recorded poses. In the photo the gripper still holds the wrench; the drawing shows the wrench only. Labels quote the report and mark what each format adds; the report beside it has every line.

What the agent read

Left for the agent to work out: where the ring is relative to the shank.

example run

World coordinates contain everything needed to work out a grasp offset or a relative orientation, yet agents often fail to compose those transforms. Writing the same state in object frames helps, and naming the parts that matter, the wrench's ring and the hook's shank, helps more: 52% overall success with part annotations, against 40% for object frames, 30% for world-frame state and 19% for the outcome alone. On Tool Hanging: 15, 25, 63 and 90%.

The episode above is one Tool Hanging failure. The table is the whole study: held-out success for each format on each task, as in the paper's Table 3.

Held-out success, %, by report format

From the paper: Figure 7 and Table 3

The paper's Figure 7: one failed Tool Hanging state shown in four report formats — Figure 7: the four formats, illustrated on another failed Tool Hanging episode.

Finding 5 · Physical reasoning

Agents spot geometric errors but rarely plan motion around collision geometry.

The same layout, before and after two rounds. The second program's fix, drawn where it acts.

▸ Play both programs on the same layout.

Carried flat, both syringes land across the opening.

tilt

The agent's “collision model” is a stack of clearances.

tilt

syringe 1 syringe 2 syringe 1 earlier, at the time shown

Side section of the lid, schematic. To scale: the 150 mm syringe and the 55 mm opening (tinted); heights and tilt are the recorded poses. On the second section, the dimensions are the fix's constants, drawn to scale where the program applies them.

Across the paper's runs

17/19

17 of 19 first self-written policies carried or released the syringe flat over the opening; the rest did not.

The fix, as written Hover a constant's line to light its dimension on the second section.

example run

The robot must carry a 150 mm syringe through a 55 mm opening, reasoning about the syringe as part of the moving system. Feedback let agents recognize the error and adopt fixes, but they encoded them as clearance constants, staging heights and retries rather than a general collision-aware model.

Human policies make geometry explicit, with repeated perception, intermediate waypoints and dedicated motion-planning stages. In the program rows, the human program builds a collision_world, maps the syringe in the hand (hand_map) and plans the carry and the insertion as steps of their own. Both agent programs leave those columns empty.

One square per top-level step, in execution order; a column holds steps that do the same job, and a dashed slot is a step the program does not have. Rows: H the human expert, A GPT-6 Astra, F Fable 5.1, as in the paper's Figure 8; on the right, held-out success after two rounds (the figure gives the human row none). ×2: Astra's program runs this block twice. Hover or tap a step for its name.

From the paper: Figures 8 and 9

The paper's Figure 8: program structure of the human, Astra and Fable policies for five tasks — Figure 8. Policy structure by human (H), Astra (A) and Fable (F).

The paper's Figure 9: the human and agent motion plans around the syringe box — Figure 9. Human motion plan (blue) considers a collision-free spot (B) to reorient the gripper, while the agent plan (red) has limited room to reorient.

Finding 6 · Curriculum

Learning with privileged state first can speed up perception-based learning, or lock in the wrong strategy.

Each task learned two ways: exact simulator geometry first, then the camera; or the camera from the start.

▸ Pick a task: its success both ways, and how big its final programs got.

Program size

Overall

After the switch, the inherited program is patched, not rebuilt.

Two runs that locked in early

Kitchen · MimicGen

in one run

A grasp only exact geometry can hit

In one run, the privileged policy pinched a narrow tab on the pot handle, which is only a few millimeters wide, and reached 90–100% held-out success; after the switch to perception it dropped to 0–30%.

The agent kept naming the tab grasp as the cause of the failures, yet changed its reach and retries rather than the grasp.

Gear Meshing · VABAR

in one run

A camera that barely sees the pegs

The inherited policy read the pegs from the front camera, where each occupies only 2–3 raised depth pixels, instead of the overhead camera (about 18). It fell to at most 7%; a perception-only run using the overhead camera reached 97–100%.

Later iterations mainly tried to repair peg detection rather than change the inherited camera.

The privileged phase learns useful structure in Threading (86% vs 2% without it), Coffee and Mug Cleanup, but it can also commit early to choices perception cannot support: Tool Hanging falls to 16% vs 63%; overall 52 vs 46. After the switch, agents patch the inherited program rather than reorganize it: 12.0 → 12.7 skills on average, while perception-only programs end at 16.1.

Exact geometry is not the same as knowing what to do with it. In Tool Hanging the privileged phase stays below 10% held-out success even with the exact wrench pose: the pose says where the wrench is, not where or how to hold it for the carry.

From the paper: Figure 10 and Table 4

Figure 10: held-out success and stage reward per iteration for the privileged-to-perception curriculum and the perception-only control, on four MimicGen and three VABAR tasks — Figure 10. Held-out success (top) and stage reward (bottom) for the privileged-to-perception curriculum and the perception-only control, on 10 fixed held-out layouts per task. Lines are means over three runs; bands are ±1 standard error. The dotted line marks the switch to perception; shading marks the privileged phase.

Table 4. Policy size (# skills in the final policy) and held-out success (mean over the last three iterations), with and without the privileged-to-perception curriculum; mean over three runs.

Part 3 · Coverage

How much of the problem do today's agents cover?

Take the 73 VABAR Level-1 test cases that a human-written policy solves. Zero-shot generation alone solves only 7.3%. Iterative self-learning covers 16.1%, richer simulation feedback 10.5%, human task skills 27.0% and a newer model 35.0%; 4.1% remain unsolved. What the agents were missing is mostly spatial reasoning (49.9%), then motion planning (14.4%), skill composition (11.6%), manipulation planning (8.8%) and perception (8.0%).

▸ Add the levers one at a time.

7.3%zero-shot alone

49.9%missing spatial reasoning

4.1%still unsolved

The 73 cases, each split equally among the levers that solve it

What the agents were missing

Spatial reasoning4 Space 5 Physics
Motion planning5 Physics
Manipulation planningProblem
Perception3 Time 6 Curriculum
Skill composition2 Skills

What solved them

Zero-shot generation
Iterative self-learningLoop
+ Simulation feedback3 Time 4 Space
+ Human task skills2 Skills
+ Newer model1 Model
Unsolved

Width is the share of the 73 test cases (Cable Routing 26, Syringe Packing 18, Gear Meshing 10, Tool Hanging 10, USB-C Insertion 9). A case solved several ways is credited to each route equally. Hover or tap a capability to follow its cases; each tag leads to the part of the page that studies it. From the paper: Figure 11.

Data

“Across these studies, we find that self-learning is limited less by the availability of useful information than by the agent's ability to turn the information into effective policy updates.”
From the paper's introduction

Code

Run the loop on your own RoboSimStudio task

Claude Code or Codex writes a Graph-as-Policy program, runs it in simulation, reads a feedback report and edits the program. Every feedback condition on this page is a config switch.

graph-robots/agentic-self-learning

The included example: "pick up the red cube and put it in the tray", learned from the instruction alone.

▸ Flip a switch: the config, the file it comes from and the run command follow.

Form feedback.form

→ Finding 4

Under form: [] the report is the outcome alone, so Space and Granularity change nothing the agent reads.

Space feedback.spatial

Granularity feedback.granularity

→ Finding 3

Agent agent.kind

→ Finding 1

Run it

Resources

Paper, code and agent traces

Paper The full study, with the appendix: tasks, prompts, feedback reports, models, cost and the curriculum study. arxiv.org/abs/2611.09030 Code The self-learning loop for RoboSimStudio, with every feedback condition of the study as a config. github.com/graph-robots/agentic-self-learning Agent traces Every agent run of the project: what each agent was told, what it read, what it wrote, and the step-by-step session where the log survives. huggingface.co/datasets/keplerccc/agentic-robotics-coding-traces

Built on VABAR Graph-as-Policy open-robot-skills RoboSimStudio

Every study clip on this page replays a run in the traces.

Each id links to that run's archive in the Agent traces dataset, with its programs, reports and episode logs; the per-step poses the clips were rendered from are not in the release. Clips that share a bracket are rounds of one run: hover or tap one for the episode it shows, or click it on a desktop to go to where it plays.

Show where each number comes from

Citation

@misc{chen2026controlled,
  title         = {A Controlled Study of Models, Skills, Feedback, and Curriculum in Agentic Robotics},
  author        = {Chen, Kaiyuan and Zhu, Xinyue and Xie, Shuangyu and Yoo, Uksang and Luong, Thi Thu Phuong and Gualtieri, Marcus and Juette, Christian and Ren, Liu and Huang, Spencer and Goldberg, Ken},
  year          = {2026},
  eprint        = {2611.09030},
  archivePrefix = {arXiv},
  primaryClass  = {cs.RO},
  url           = {https://arxiv.org/abs/2611.09030}
}