Sample a view
Draw (h, K) from the task distribution. Separate the generated history C from the continuation S.
Autoregressive video distillation
Under review at ICLR 2027
HorizonDMD organizes distribution matching over student-generated history–horizon views, using the visible prefix for scoring and the continuation for direct generator supervision.
Selected scenes spanning object interactions, wildlife, materials, and motion. Each clip shows one complete autoregressive rollout of approximately five seconds, generated using EMA weights and four denoising steps per block.
The examples use matched prompts and initial noise, EMA weights, and four denoising steps per block. HorizonDMD examples are curated from multiple trained variants. Play each pair together to compare the same moment in each rollout.
Compare four ways of organizing supervision, using the same prompt and initial noise, EMA weights, and four denoising steps per block. The four videos share one playback timeline.
Autoregressive video diffusion supports streaming generation, but preserving video quality under few-step distillation remains challenging. On-policy learning trains students on their own generated histories, leaving an open design space regarding which temporal distributions are matched along each rollout. Based on this insight, we introduce HorizonDMD, a framework for distribution matching over history–horizon views that pair a generated history prefix with a contiguous continuation. We formulate an ideal objective over noisy continuations conditioned on noisy histories, with full-video matching as a special case. In practice, the teacher and auxiliary score models process the same jointly noised visible prefix. The auxiliary model is trained on that prefix, while direct generator supervision is restricted to the continuation. Under consistent marginals across views, positive weighted coverage preserves the zero-divergence target of full-sequence matching. We also establish prefix–conditional gradient equivalence with a fixed clean-history distribution and show that prefix scores generally differ from restricted full-video scores. Extensive experiments and ablation studies demonstrate the effectiveness of our HorizonDMD framework. Specifically, matched ablations show a 29.6% relative increase in dynamic degree over full-video supervision, alongside higher VBench Total and Quality scores. Our best evaluated model raises Quality from 84.98 to 85.34 and Total from 84.38 to 84.64 relative to our direct reproduction of Self-Forcing DMD.
Method overview
A history–horizon view specifies the visible prefix used for scoring and the continuation receiving direct supervision.
Draw (h, K) from the task distribution. Separate the generated history C from the continuation S.
The teacher and auxiliary score models read the same jointly noised prefix V = (C, S), formed by concatenating history and continuation. Later blocks are excluded.
Apply the generator’s direct loss and normalization on S. Train the auxiliary model to regress the entire visible prefix V.
Change the history and horizon below.
Views are sampled from π(h, K), with generator loss weighted by K / E[K]. The score models can attend bidirectionally within the visible prefix.
Under the paper’s stated assumptions, the analysis establishes properties of ideal objectives and gradients.
When all views use consistent marginals and complete-history conditionals of the same student and reference sequence distributions, positive weighted coverage of every autoregressive conditional preserves the zero-divergence target of full-sequence matching in the ideal objective.
With the clean-history distribution held fixed, noisy-prefix KL and conditional-horizon KL have the same gradient with respect to the conditional generator.
Prefix scores generally differ from full-video scores restricted to the same continuation. Marginalizing the excluded future blocks changes the score itself.
VBench results from the latest manuscript: fixed-endpoint supervision controls and evaluated configurations compared with Self-Forcing DMD.
A–D share a common training endpoint and matched generator and auxiliary update counts. PrefixHorizon (C) increases dynamic degree by 29.6% relative to FullVideo (A), with higher Total and Quality scores.
| Setting | Total ↑ | Quality ↑ | Semantic ↑ | Dynamic ↑ |
|---|---|---|---|---|
| A: FullVideo | 84.1603 | 84.6364 | 82.2557 | 49.72 |
| B: FullScoreHorizon | 84.1286 | 84.7242 | 81.7464 | 53.33 |
| C: PrefixHorizon | 84.5356 | 85.1865 | 81.9321 | 64.44 |
| D: SingleBlock | 83.9365 | 84.3844 | 82.1447 | 46.39 |
FullVideo (A) is a recipe-matched full-video DMD control. These A–D runs are separate from the Self-Forcing DMD reproductions and HorizonDMD configurations below.
| Method | Total ↑ | Quality ↑ | Semantic ↑ |
|---|---|---|---|
| Self-Forcing DMD Direct reproduction | 84.38 | 84.98 | 81.97 |
| Self-Forcing DMD Higher-scoring reproduction | 84.52 | 85.05 | 82.37 |
| HorizonDMD H1 · single-view, pfull = 1/3 | 84.64 | 85.34 | 81.86 |
| HorizonDMD H2 · single-view, pfull = 3/8 | 84.46 | 85.28 | 81.20 |
| HorizonDMD H3 · two-prefix | 84.44 | 85.14 | 81.64 |
| HorizonDMD H4 · two-prefix, split RNG | 84.44 | 85.00 | 82.21 |
The two Self-Forcing DMD rows are independent training reproductions. Among the results shown, H1 achieves higher Total and Quality than both reproductions; the higher-scoring Self-Forcing DMD run has the highest Semantic score.
H1–H4 report the checkpoint with the highest observed Total score in each evaluated range, selected without a separate validation set. Checkpoint search ranges and training compute are not matched. The quantitative Self-Forcing comparison in Table 2 uses independently sampled initial noise; the matched A–D comparison in Table 1 uses paired initial noise. The playable qualitative comparisons above use matched initial noise and show selected trajectories. They are separate from the fixed-endpoint A–D controls. See the paper for full evaluation details.
Learn more
The paper includes the objective, theoretical analysis, implementation details, and evaluation.
Read the paper