When a robot learns a task from demonstrations, the training loss is the easy part. A policy can reproduce every recorded trajectory almost perfectly and still fail on the first real attempt, because the first small error puts it somewhere the demonstrations never went. Most of the research history of imitation learning is a sequence of fixes for that one problem. This guide walks through four of them using the original papers: behavior cloning, DAgger, Action Chunking with Transformers (ACT) and Diffusion Policy. It sits between Where Robot Training Data Comes From, which covers how demonstrations are collected, and How Robot Policies Are Evaluated, which covers how the resulting policies are scored.
Behavior cloning and why it drifts
Behavior cloning (BC) is the simplest form: record observations and the expert's actions, then train a model to predict the action from the observation. The ACT paper describes it as "casting imitation as supervised learning from observations to actions."[2] For a single prediction that is ordinary machine learning. For a robot it is not, because predictions are chained: each action changes the next observation.
Ross, Gordon and Bagnell state the consequence precisely. A classifier that makes a mistake with probability ε under the distribution of states the expert visits "can make as many as T²ε mistakes in expectation over T-steps under the distribution of states the classifier itself induces." Their intuition: "as soon as the learner makes a mistake, it may encounter completely different observations than those under expert demonstration, leading to a compounding of errors."[1]
Read that as a statement about growth, not a measured failure rate. The error does not add up, it multiplies through the horizon: doubling the length of the task can quadruple the worst-case mistakes. It also explains a pattern anyone who has trained a BC policy will recognise: the policy looks fine for the first moments of an episode and degrades towards the end. The ACT paper reports exactly that behaviour for prior methods, which "make progress in the first two subtasks" while the final success rate stays low.[2]
DAgger: let the learner drive, let the expert label
DAgger (Dataset Aggregation) attacks the mismatch directly: train on the states the learner will actually visit. The paper's description is short. In the first iteration the expert's own policy gathers trajectories, and a policy is trained to mimic them. At each later iteration the current learner collects new trajectories and "adds those" to the dataset, with the expert supplying the correct action for the states reached; the next policy is trained "under the aggregate of all collected datasets."[1]
The authors also describe an optional blend: during data collection, follow the expert with probability β and the learner otherwise, so early, weak policies do not wander into irrelevant states.[1] Their experiments were a 3D racing game (Super Tux Kart) steering task and Super Mario Bros., where they report DAgger outperforming the earlier approaches they compared against. They did not test it on a physical robot arm in that paper.
The theoretical result is the part that matters for the comparison. Theorem 3.1 says that with the number of iterations on the order of T (up to log factors), at least one of the policies trained has a loss, under its own state distribution, close to the best achievable on the aggregate data; the paper contrasts this with the T² behavior of plain supervised training.[1] We read that as the important idea rather than a practical recipe: the guarantee holds when the supervised learner can drive its training loss down, which is a strong assumption for deep policies on images.
The practical cost is the expert. Someone, or something, has to say what the right action was in states the learner wandered into. The ACT authors argue this is awkward for robots: "expert annotation can be time-consuming and unnatural with a teleoperation interface."[2] Replaying a policy's rollout and asking a human to relabel each step is much harder than demonstrating the task.
ACT: predict a chunk, not a step
ACT, from the ALOHA project at Stanford, UC Berkeley and Meta, takes a different route: it keeps plain demonstrations but changes what the policy predicts. Rather than one action per observation, it predicts the next k target joint positions and executes them in sequence before observing again. In the paper's words, every k steps "the agent receives an observation, generates the next k actions, and executes the actions in sequence," which "implies a k-fold reduction in the effective horizon of the task."[2] Fewer decision points means fewer chances to compound an error.
The authors name a second benefit that is easy to miss. Human demonstrations contain pauses. A single-step policy that sees the same observation during a pause and when moving has no way to tell the two apart, because "the behavior not only depends on the state, but also the timestep." Predicting a chunk lets the policy represent that structure.[2]
Two further design choices are worth knowing:
- Temporal ensembling. The policy is queried at every step and the overlapping chunk predictions are averaged for smoothness. The ablation shows a modest gain for ACT (3.3%) and a larger one for a simple BC baseline (4%), and a drop for VINN, a retrieval-based method. The authors caution that the settings with and without ensembling were tuned separately.[2]
- A conditional VAE (CVAE) objective. Training the transformer as a CVAE is how ACT models the variability in human demonstrations.[2] Hugging Face's LeRobot documentation describes one open implementation as a ResNet-18 vision backbone feeding a transformer encoder and decoder, about 80 million parameters, taking multiple camera images and joint positions and outputting a chunk of k future actions; it also says the latent variable is set to zero at inference.[4]
What the numbers show
The chunk-size ablation is the cleanest evidence in the paper. Averaged over four simulated settings (two tasks, each with scripted and human data) and with temporal ensembling disabled, success rose "from 1% at k = 1 to 44% at k = 100, then slightly tapers down with higher k." The authors attribute the dip at k=200 and 400, close to open-loop control, to a lack of reactivity and the difficulty of modelling long action sequences.[2] Adding chunking to two baseline methods improved them too, though ACT still beat them.[2]
| Quantity | What the ACT paper reports | Caveat |
|---|---|---|
| Chunk size effect | 1% (k=1) to 44% (k=100) average success, simulation | Simulated tasks, ensembling off |
| Real tasks, success | Slide Ziploc 88%, Slot Battery 96%, Open Cup 84%, Put On Shoe 92%, Prep Tape 64%, Thread Velcro 20% | 25 trials, one seed per task |
| Demonstrations | 50 per task (100 for Thread Velcro), about 10 to 20 minutes of data | Wall-clock 30 to 60 minutes with resets |
| Control rate | 50 Hz, 400 to 700 steps per 8 to 14 second episode | Teleoperation at 5 Hz took 62% longer in a user study |
| Hardware | Under $20k for the bimanual teleoperation system | Authors' own cost figure |
The spread from 96% down to 20% is the honest summary. ACT works very well on some contact-rich tasks with a few dozen demonstrations and poorly on others; for Thread Velcro the authors trace failures to low-contrast visual observations of a thin cable tie and imprecise insertion.[2] They also list tasks beyond the system's reach, such as buttoning a dress shirt.[2] Treat "80 to 90% with 10 minutes of data" as an upper-range result on a well-chosen task, not a rate to expect elsewhere.
Diffusion Policy: model the whole distribution of actions
The third approach changes the model class. Diffusion Policy represents the policy "as a conditional denoising diffusion process": it starts from noise and refines it, conditioned on recent observations, into a sequence of actions.[3]
The motivation is multimodality. Demonstrations often contain several valid ways to reach the same immediate goal, and a model that regresses to a single action tends to average them. Our illustration, not the paper's: if some demonstrators pass an obstacle on the left and others on the right, a regression policy can learn to drive straight into it. The authors define the property they care about as "multiple ways of achieving the same immediate goal," and show it in a planar pushing task (Push-T).[3] A generative policy can sample one mode instead of averaging.
Like ACT, it predicts sequences. The paper defines an observation horizon, an action prediction horizon of Tp steps and an execution horizon of Ta steps that run on the robot before re-planning, which it calls receding-horizon control, "achieving a balance between long-horizon planning and responsiveness."[3] The two methods arrive at a similar structural idea from different directions.
The reported results and the cost
Across 15 tasks from four manipulation benchmarks, the authors report an average improvement of 46.9% over the best baseline per task. The paper's appendix defines it as the mean, over tasks, of the relative improvement of the best Diffusion Policy variant over the best baseline score, so it is an average of relative gains on the authors' chosen comparisons, not an absolute success rate.[3]
Diffusion is slower than a feed-forward network, and the paper says so. Running the denoising loop for every action is addressed with DDIM sampling: 100 training iterations and 10 inference iterations gave 0.1 s of inference latency on an Nvidia 3080 GPU in the authors' real-world experiments. In one of their real-robot setups the policy predicted commands at 10 Hz, linearly interpolated to 125 Hz for execution. The limitations section states that Diffusion Policy "has higher computational costs and inference latency compared to simpler methods like LSTM-GMM," which action-sequence prediction only "partially mitigates," and that it "may not suffice for tasks requiring high rate control." It also says the method "inherits limitations from behavior cloning, such as suboptimal performance with inadequate demonstration data."[3]
Choosing between them
| Method | Core idea | Needs | Main limit (from the papers) |
|---|---|---|---|
| Behavior cloning | Supervised learning from demonstrations | Demonstrations only | Compounding error; T²ε worst case[1] |
| DAgger | Expert labels the learner's own states, data pooled each round | An expert available during training | Expert annotation is slow and awkward for teleoperated robots[2] |
| ACT | Predict k actions per observation; CVAE; optional temporal ensembling | Demonstrations; fast control loop | Task-dependent: 20% to 96% in real tasks[2] |
| Diffusion Policy | Denoise a noise sample into an action sequence | Demonstrations; GPU for inference | Higher compute and latency than simple policies[3] |
Our interpretation, not a finding of any one paper: these are not rivals in a strict sense. ACT and Diffusion Policy both reduce the number of decisions per task through sequence prediction, DAgger addresses a different failure (distribution shift) and could in principle be combined with either, and all three inherit BC's dependence on demonstration quality. The ACT paper itself mentions that DAgger-style corrections are hard to collect with its teleoperation setup, which is why it looks for a fix that works from demonstrations alone.[2]
How to read a claim that says "learned from demonstrations"
- Ask how many trials and seeds. The ACT real-world table reports 25 trials and one seed per task. That is normal for robotics, and also thin for ranking close results.
- Check whether the number is task-specific. A policy that reaches 96% on battery insertion and 20% on cable-tie threading has no single "success rate."
- Look for the control rate and latency. Chunking and diffusion change how often a policy can react. A method that is fast enough at 10 Hz may not be at 50 Hz.
- Check who corrected the robot. If a human relabelled the policy's own rollouts, that is DAgger-style expert time, and it should be counted in the data cost.
- Remember the data. Both papers say their policies depend on demonstration quality. How demonstrations were gathered is covered in the robot training data guide, and how language-conditioned policies build on this is in Vision-Language-Action Models.
What we did not verify
The numbers above come from the papers as published in 2010, 2023 and 2024 (the Diffusion Policy version we read is v5, March 2024). We did not run any of these methods, reproduce any result or test current implementations, so no statement here is "in our testing." We did not read follow-up work that may have changed the picture, including newer large robot foundation models. The LeRobot page was read on 8 November 2026 and describes one open implementation of ACT, not the original authors' code. For an applied comparison of hosted and open robot models, see Physical Intelligence's Pi API Beta vs the Open openpi Weights.
