Evoke

<h1 align="center">
<sub>

A three-step, CFG-free world model that remembers where it has been, takes direction mid-flight, and keeps going — 1.5 s of video every 2.11 s on a single H200.

✨ Highlights

• 🏆 State of the art on WBench — as a three-step world model, while staying competitive in
  visual quality on VBench-Long and VBench-2.0.
• ⚡ 3 steps, zero CFG. 1.5 s of 384×640 video every 2.11 s on one H200 — one forward per
  step, not two. Few-step speed without few-step ceilings.
• 🌍 Endless, not windowed. Scene geometry lives in an external, camera-indexed world state
  bank instead of the denoiser's context. Only what the current view needs is retrieved, so the
  context stays bounded however long the session runs — no trading session length for memory.
• 🎛️ Re-promptable mid-flight. Per-chunk conditioning lets you change the prompt while the
  rollout is running: the sky ignites, the storm rolls in, no cut and no restart.
• 🧑‍🏫 A teacher rebuilt for the long horizon. Chunk-wise grouping, distant-frame retrieval and a
  linear-attention global state make its memory and compute grow linearly — which is what makes
  30 s self-forced supervision affordable.

🎬 Demos

▶  Watch the overview

Every clip was produced by the launchers in this repo, on the data bundled in examples/ — no external
dataset, no cherry-picking across seeds. The Move / Rot joystick is the camera-control HUD burned
into geo_pred.mp4.

Re-prompting mid-rollout

Per-chunk conditioning lets the prompt change while the rollout is running — no cut, no restart.
Each schedule below switches at chunk 3 of 6 (213 frames, 8.9 s).

MODE=segment NUM_CHUNKS=6 MAX_CASES=0 bash scripts/inference/infer_post_distill.sh

Conditioning modes

4 chunks each (141 frames, 5.9 s; t2v is 140).

MODE=v2v NUM_CHUNKS=4 bash scripts/inference/infer_post_distill.sh   # or MODE=i2v / MODE=t2v

The other two models

Both run on the same bundled inputs as the demos above, so they are directly comparable.

NUM_CHUNKS=4 bash scripts/inference/infer_stage1.sh
bash scripts/inference/infer_evoke_teacher.sh

🧠 How it works

The student is autoregressive over latent chunks (latent_window_size = 9). Each chunk is laid out
along the RoPE frame index as:

prefix | long(16) | mid(2) | warp(W) | prev_short(1) | noise(W)

Every tier lives at the same latent resolution (res/8); compression comes only from the patchify
convolution kernel, and no low-resolution latents are stored. In the short tier
[prefix | warp | prev_short] the residual MLP and the per-stage compression act only on the warp
frames in the middle.

The world state bank itself has three operations: write — a monocular depth model estimates depth
for the emitted chunk under its known poses, unprojected into a persistent point cloud; read — the
current camera pose addresses the bank directly, with sources ranked by co-visibility, up to eight
fused, and a batched z-buffered scatter returning a warped image plus a per-pixel visibility mask; and
evict — an optional retention window, which hour-scale runs enable explicitly.

⚙️ Environment

Python 3.10 + CUDA 12.4. The pins in requirements.txt are the environment actually in use —
torch 2.4 / deepspeed 0.14.5 / flash-attn are load-bearing, not aspirational.

pip install torch==2.4.0 torchvision==0.19.0 --index-url https://download.pytorch.org/whl/cu124
pip install -r requirements.txt

Two things pip will not do: diffusers is pinned to a development fork that is not on PyPI
(install it first or nothing imports), and postprocess_viz.py needs the ffmpeg binary on
PATH. Both depth backends (ViGeo, Depth-Anything-3) are vendored under
evoke/third_party/, so only their weights are downloaded — see Weights.

📦 Weights

Everything goes under models/ (gitignored). Every released EVOKE directory is the parent of a
transformer/, because it loads as from_pretrained(path, subfolder="transformer").

models/
├── evoke-base/                                   vae / text_encoder / tokenizer / scheduler only
├── ViGeo1.1/vigeo.pt                             depth backend -- REQUIRED
├── DA3/{config.json,model.safetensors}           depth backend -- OPTIONAL
└── evoke/
    ├── stage1_camera_control/transformer/        multi-step camera-controllable model
    ├── stage2_few_step_training/transformer/     few-step distillation (3-step pyramid)
    ├── stage3_long_distillation/transformer/     30s long-video distillation (post-distill init)
    ├── stage3_post_distillation/transformer/     the shipped model
    └── evoke_teacher/{high,low}_noise/           the two DMD teacher experts -- training only

# EVOKE -- the four released models, the teacher, and the base components
huggingface-cli download SII-YuanyangYin/Evoke --local-dir models

# ViGeo -- REQUIRED. The depth backend behind the world state bank; every shipped
# recipe uses it (DEPTH_BACKEND=vigeo, cloud_warp.backend: vigeo).
huggingface-cli download pkqbajng/ViGeo --local-dir models/ViGeo1.1

Depth-Anything-3 is optional — nothing in the default path touches it, and you only need it if you
set DEPTH_BACKEND=da3. Get the da3-giant weights from
depth-anything-3 and drop
config.json + model.safetensors into models/DA3/. Switching backend is a recipe change, not a
speed knob — training and inference must agree on it.

Both depth backends ship under CC-BY-NC-4.0, which is more restrictive than this repo's
Apache-2.0. Check their licences before any commercial use.

🚀 Inference

384 × 640 @ 24 fps. One chunk = 36 frames = 1.5 s, so NUM_CHUNKS=20 is a 30 s clip. All four
commands run on the data bundled in examples/ — one case each, four for segment — no external
dataset:

MODE=t2v     NUM_CHUNKS=20              bash scripts/inference/infer_post_distill.sh   # prompt only
MODE=i2v     NUM_CHUNKS=20              bash scripts/inference/infer_post_distill.sh   # first frame + pose
MODE=v2v     NUM_CHUNKS=20              bash scripts/inference/infer_post_distill.sh   # ref video + pose
MODE=segment NUM_CHUNKS=6  MAX_CASES=0  bash scripts/inference/infer_post_distill.sh   # prompt switches mid-rollout

Results land in <OUT_ROOT>/<case>/geo_pred.mp4. Every distilled model was trained on v2v
conditioning alone (geo_condition_{i2v,t2v}_ratio: 0.0), so MODE=i2v|t2v on them is zero-shot
and the launchers say so at startup. Only stage1_camera_control has all three modes in distribution
(ratios 0.1 / 0.2).

Everything else — the mode × model matrix, hour-scale rollouts, per-chunk log format, and how to point
the launchers at your own data — is in scripts/inference/README.md.
Every launcher also has -h.

🗝️ Training

One launcher per released model. Each initialises from its own released checkpoint, so you
continue from where we left off — nobody reproduces a stage from scratch, and the pretraining data is
not part of this release:

bash scripts/training/train_stage1_camera_control.sh      # no teacher (not a distillation)   1x8
bash scripts/training/train_stage2_few_step_training.sh   # teacher: stage1_camera_control    1x8
bash scripts/training/train_stage3_long_distillation.sh   # teacher: evoke_teacher, 2 experts 6x8
bash scripts/training/train_stage3_post_distillation.sh   # teacher: stage1_camera_control    6x8

Post-distillation goes back to the stage-1 teacher on purpose: it is a short run that firms up camera
control, not another long-horizon distillation.

Each writes to models/train/<same-name>/; move or symlink it into models/evoke/ to serve it.

All four start with no external dataset — they point at the single 60 s clip in examples/data/,
so they run as a pipeline check, not a real training run (one clip overfits immediately). For a real
run swap data_yaml_path to the production mix named beside it in the config.

Scale is set by ACCELERATE_CONFIG — the topology is baked into the accelerate yaml, so do not
override it with --num_machines. To merge a LoRA checkpoint into a full transformer, see
tools/merge_lora_ckpt.py (use --dtype fp32: the delta is ~5e-4 of the weight magnitude and bf16
swallows it).

📝 Notes

• The warp / attention recipe must match between training and inference. A mismatch silently
  degrades quality rather than failing — every knob in the launchers is annotated with the config
  field it mirrors.
• Resolution is data driven, but keep the width a multiple of 64 so the long tier and the
  quarter-resolution pyramid stage both divide evenly.

👍 Acknowledgement

The EVOKE teacher is built on LingBot-World. The
vae / text encoder / tokenizer / scheduler in models/evoke-base come from the released
Helios base, which traces them to Wan.

🔒 License

Apache-2.0, see LICENSE. Vendored third-party code keeps its own license and provenance under
evoke/third_party/*.

✏️ Citation

@misc{evoke2026,
  title         = {Alaya-EVOKE: From Linear-Scaling Supervision to Endless World},
  author        = {Yuanyang Yin and Gongxuan Wang and Yifan Zhan and Chuanhao Li and Kaipeng Zhang and Feng Zhao},
  year          = {2026},
  eprint        = {2608.13546},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV},
  url           = {https://arxiv.org/abs/2608.13546},
}