Autoregressive video distillation

HorizonDMD

Horizon Distribution Matching
for Autoregressive Video Distillation

Anonymous authors

Under review at ICLR 2027

HorizonDMD Loading video wall…
TL;DR

HorizonDMD organizes distribution matching over student-generated history–horizon views, using the visible prefix for scoring and the continuation for direct generator supervision.

HorizonDMD in motion

Selected video examples

Selected scenes spanning object interactions, wildlife, materials, and motion. Each clip shows one complete autoregressive rollout of approximately five seconds, generated using EMA weights and four denoising steps per block.

Self-Forcing DMD vs. HorizonDMD

Matched prompts and initial noise

The examples use matched prompts and initial noise, EMA weights, and four denoising steps per block. HorizonDMD examples are curated from multiple trained variants. Play each pair together to compare the same moment in each rollout.

A–D supervision controls

Matched qualitative comparisons

Compare four ways of organizing supervision, using the same prompt and initial noise, EMA weights, and four denoising steps per block. The four videos share one playback timeline.

AFullVideoBFullScoreHorizonCPrefixHorizonDSingleBlock

Abstract

Autoregressive video diffusion supports streaming generation, but preserving video quality under few-step distillation remains challenging. On-policy learning trains students on their own generated histories, leaving an open design space regarding which temporal distributions are matched along each rollout. Based on this insight, we introduce HorizonDMD, a framework for distribution matching over history–horizon views that pair a generated history prefix with a contiguous continuation. We formulate an ideal objective over noisy continuations conditioned on noisy histories, with full-video matching as a special case. In practice, the teacher and auxiliary score models process the same jointly noised visible prefix. The auxiliary model is trained on that prefix, while direct generator supervision is restricted to the continuation. Under consistent marginals across views, positive weighted coverage preserves the zero-divergence target of full-sequence matching. We also establish prefix–conditional gradient equivalence with a fixed clean-history distribution and show that prefix scores generally differ from restricted full-video scores. Extensive experiments and ablation studies demonstrate the effectiveness of our HorizonDMD framework. Specifically, matched ablations show a 29.6% relative increase in dynamic degree over full-video supervision, alongside higher VBench Total and Quality scores. Our best evaluated model raises Quality from 84.98 to 85.34 and Total from 84.38 to 84.64 relative to our direct reproduction of Self-Forcing DMD.

Method overview

Teach the continuation, in context.

A history–horizon view specifies the visible prefix used for scoring and the continuation receiving direct supervision.

1

Sample a view

Draw (h, K) from the task distribution. Separate the generated history C from the continuation S.

2

Score the prefix

The teacher and auxiliary score models read the same jointly noised prefix V = (C, S), formed by concatenating history and continuation. Later blocks are excluded.

3

Update the continuation

Apply the generator’s direct loss and normalization on S. Train the auxiliary model to regress the entire visible prefix V.

Shared autoregressive student rollout and DMD noising. Self-Forcing supervises the full clip; HorizonDMD uses history and horizon views, with full-video matching as a special case.
Shared rollout, different supervision views. Full-video matching is the special case with no history and a horizon spanning the entire clip. Open full-size figure ↗
HorizonDMD method: sample history–horizon views, jointly noise the visible prefix, score with a frozen teacher and a learned auxiliary model, and apply the horizon loss.
The paper's history-horizon views: full video, global prefix, and history-conditioned continuationThe paper's HorizonDMD generator update through prefix scoring and a detached target
The supervision framework. Open full-size figure ↗
Explore history depth and horizon

Explore a supervision view

Change the history and horizon below.

(h, K) = (2, 3)
Student-generated rollouttime
History CContinuation SOutside view U
Teacher + auxiliary score
Same noisy prefix · V
Direct generator loss
Continuation · S
Auxiliary regression
Entire visible prefix · V

Views are sampled from π(h, K), with generator loss weighted by K / E[K]. The score models can attend bidirectionally within the visible prefix.

What does the theory establish?

Under the paper’s stated assumptions, the analysis establishes properties of ideal objectives and gradients.

Conditional coverage

When all views use consistent marginals and complete-history conditionals of the same student and reference sequence distributions, positive weighted coverage of every autoregressive conditional preserves the zero-divergence target of full-sequence matching in the ideal objective.

Prefix–conditional gradient equivalence

With the clean-history distribution held fixed, noisy-prefix KL and conditional-horizon KL have the same gradient with respect to the conditional generator.

Prefix scoring versus full-video masking

Prefix scores generally differ from full-video scores restricted to the same continuation. Marginalizing the excluded future blocks changes the score itself.

Quantitative results

VBench results from the latest manuscript: fixed-endpoint supervision controls and evaluated configurations compared with Self-Forcing DMD.

Matched supervision controls

A–D share a common training endpoint and matched generator and auxiliary update counts. PrefixHorizon (C) increases dynamic degree by 29.6% relative to FullVideo (A), with higher Total and Quality scores.

Paper Table 1 · scores on a 0–100 scale · higher is better
SettingTotal ↑Quality ↑Semantic ↑Dynamic ↑
A: FullVideo84.160384.636482.255749.72
B: FullScoreHorizon84.128684.724281.746453.33
C: PrefixHorizon84.535685.186581.932164.44
D: SingleBlock83.936584.384482.144746.39

FullVideo (A) is a recipe-matched full-video DMD control. These A–D runs are separate from the Self-Forcing DMD reproductions and HorizonDMD configurations below.

Comparison with Self-Forcing DMD

Paper Table 2 · scores on a 0–100 scale · higher is better
MethodTotal ↑Quality ↑Semantic ↑
Self-Forcing DMD Direct reproduction84.3884.9881.97
Self-Forcing DMD Higher-scoring reproduction84.5285.0582.37
HorizonDMD H1 · single-view, pfull = 1/384.6485.3481.86
HorizonDMD H2 · single-view, pfull = 3/884.4685.2881.20
HorizonDMD H3 · two-prefix84.4485.1481.64
HorizonDMD H4 · two-prefix, split RNG84.4485.0082.21

The two Self-Forcing DMD rows are independent training reproductions. Among the results shown, H1 achieves higher Total and Quality than both reproductions; the higher-scoring Self-Forcing DMD run has the highest Semantic score.

Checkpoint selection and sampling protocol

H1–H4 report the checkpoint with the highest observed Total score in each evaluated range, selected without a separate validation set. Checkpoint search ranges and training compute are not matched. The quantitative Self-Forcing comparison in Table 2 uses independently sampled initial noise; the matched A–D comparison in Table 1 uses paired initial noise. The playable qualitative comparisons above use matched initial noise and show selected trajectories. They are separate from the fixed-endpoint A–D controls. See the paper for full evaluation details.

Learn more

Read the full framework.

The paper includes the objective, theoretical analysis, implementation details, and evaluation.

Read the paper