GSSC-S2D2

The paper organises the method into three pillars that share one structured-source discrete-diffusion core:

• PS³ — Paired Sparse–Dense Scene Synthesis: the offline data-augmentation pipeline (pyramid multinomial diffusion 𝒮₁→𝒮₂→𝒮₃ + Jensen–Shannon filter + rare-class object bank + HDL-64E ray-tracer) that builds a 32,039-frame synthetic (sparse, complete) pool; pooled with the 19,130 real frames this gives a 51,169-frame total training set (2.67× expansion).
• SGSC — Semantic-guided Generative Scene Completion: the from-noise regime that completes a scene by sampling from the prior (no frozen 3D base), reaching 30.5 % val mIoU on SemanticKITTI seq 08. The paper sets that beside PaSCo without differencing the two: PaSCo's panoptic-mode evaluator is not protocol-matched to ours, and the single-sample predicate admits its single subnet (28.2 %) rather than the three-subnet 30.1 % its table reports. The SGSC checkpoint is not part of this release; the released surface is the deployment-oriented S²D² pillar.
• S²D² — Structured Source Discrete Diffusion: the one-step deployment regime that refines a frozen base SSC model's prediction — the headline 38.54 % val / 38.8 % test result at N = 1 with no TTA (39.2 % test with four steps and an 8-view D₄ ensemble), and the focus of this repository.

[!IMPORTANT]
One-pass refinement of a frozen base SSC model via discrete diffusion on the probability simplex. No distillation and no test-time adaptation, at a +2.1 pp hidden-test gain over the frozen SCPNet base it refines (36.7 → 38.8) — the previous best published score under the paper's causal, single-sweep, single-sample predicate — and +2.36 pp on val seq 08 (36.17 → 38.54, single correction step). Four correction steps with an 8-view D₄ ensemble read 39.2, our entry on the Codabench results tab; they sit outside that predicate, so the +2.1 pp margin is not read off them. The correction step costs 107.2 ms, i.e. 9.33 FPS for the added pass alone; the paper is explicit that this is "an incremental pass, not a deployable rate", and that neither it nor the 3.23 FPS end-to-end pipeline matches the sensor's 10 Hz cadence. Replaces the base argmax with one cheap correction step (validated on SCPNet) — and the same mechanism transfers to 2D BEV semantic segmentation (paper Sec. 4 secondary task, +1.3 BEV mIoU over the base-derived BEV: 34.8 % → 36.1 % on val seq 08, scored by the training-time 2D BEV evaluator on 100 fixed val frames (seed 42) — not the 4,071-frame semantic-kitti-api protocol every 3D number here is scored under).

[!TIP]
In a hurry? Skip to Quick start for the 3-command reproduction recipe of the headline 38.54 % val mIoU, on a single H100 — once the SemanticKITTI voxel labels and the SCPNet base predictions are on disk. The voxel tree needs registration at semantic-kitti.org and scripts/download_assets.py does not fetch it (see docs/DATASET.md); without it the evaluator finds an empty frame list and writes no labels. Budget ≥ 7 min of GPU for the correction pass alone (107 ms × 4,071 val frames), plus the 8.4 GiB of prediction reads, the label writes and the CPU scorer on top. No end-to-end wall-clock is published for this recipe.

[!NOTE]
Assets. The pretrained checkpoints and base-model predictions are public; scripts/download_assets.py provisions them, and docs/DATASET.md documents each artefact's on-disk layout, its measured size and its manual provisioning route, including local re-dump recipes for the JS3C-Net and LMSCNet prediction sets. A fresh clone cannot reach 38.54 % without those assets. The commands below are the exact recipe — the "verified end-to-end" labels record what the maintainers measured locally, not what an external clone can run today.

What's new

• 2026-08-27 — The manuscript is public as an arXiv preprint — arXiv:2608.26737, DOI 10.48550/arXiv.2608.26737 — and is still under review. The BibTeX block below and CITATION.cff carry the eprint id and DOI, so this work is now citable without pointing at an unpublished manuscript. The released checkpoints and both datasets are collected on its Hugging Face paper page.
• 2026-08-25 — Release v2.4.2. Every released artefact is now linked from the first screenful: the nav row carries the project page, the checkpoints, the PS³ pool (free mirror and the citable DataPort DOI) and the baseline predictions. Previously the first link to any of them sat 71 % down the file and the project page was never linked at all. Also republishes 18 stale config.json SHA256s in docs/MODEL_ZOO.md.
• 2026-08-25 — Release v2.4.1. Brings uv.lock and SECURITY.md back in step with the shipped version, so uv lock --check is clean on a fresh checkout; supported-versions table moves to 2.4.x. No API, config or measured-value change.
• 2026-08-25 — Release v2.4.0: the published reproduction command was silently evaluating a subset, and now is not. scripts/download_assets.py fetches and unpacks the released .tar.zst archives instead of reading a per-frame tree on the Hub. Several directories of the released corpus — 609,349 files across the three prediction trees and the object bank — hold more than the 10,000 files Hugging Face serves from one directory, so scpnet_predictions/08/ truncated at 10,000 files covering 3,334 of sequence 08's 4,071 val frames — the headline --include 'scpnet_predictions/08/*' fetch returned an incomplete val set, exited 0 and reported no error. Re-measured after the fix against the live repo: 12,213 files, 4,071 distinct frame ids, byte-identical to the reference tree. --include keeps its meaning and gets cheaper (val 08 is now one 703 MiB / 737 MB archive rather than 9.07 GB of per-frame transfers), sizes throughout the docs now separate download from on-disk, and zstandard>=0.22 is a dependency so uv sync alone is enough.
• 2026-08-23 — Release v2.3.8, re-cut. The project is now MIT-licensed, with third-party attribution collected in THIRD_PARTY_NOTICES.md. Adds configs/train/pyramid_{s2,s3}.yaml and the dispatcher fix that makes them runnable, plus scripts/prepare_256_data.py and scripts/prepare_multi_frame_data.py, which five code sites already pointed at. Corrects the JS3C-Net protocol attribution throughout: 26.7 % is the derived-BEV row under the paper's internal evaluator and 24.3 % is the same run under the official semantic-kitti-api — re-measured at 24.32 % over all 4,071 val frames of seq 08. The commit history was rewritten ahead of first publication so that author metadata carries only the contact addresses the paper lists; every tree hash is unchanged.
• 2026-08-13 — Releases v2.3.1 – v2.3.7: a run of correction-only patch releases — no API, config or measured-value change. They removed a JS3C-Net cross-base figure that appears nowhere in the paper and restored 24.3 % (+1.6 pp) as that base's headline; retracted the margins that had been quoted off the eight-view D₄ row in favour of the paper's own +2.1 pp over the previous best published score under the predicate (SCPNet, 36.7); withdrew the "real-time" and "cheapest deployable" wording the paper explicitly disclaims; stopped three docs asserting a hidden-test measurement for our SCPNet port, which was never submitted to the scoring server; corrected an auxiliary-loss weight that two comments quoted 100× too high; recorded that the PS³ Jensen–Shannon filter ships as screened data rather than as code (the supplementary specifies it completely); priced a full 100K-iteration launch at ~90 GPU-h, the ~37 GPU-h figure being what reaches the released step-40000 checkpoint; and brought all four version declarations back in step so uv lock --check is clean. Per-release detail in CHANGELOG.md.
• 2026-08-12 — Release v2.3.0, the submission snapshot. Adds configs/infer/test_1step.yaml, the hidden-test single-sample (N = 1) configuration — until then the 38.8 % headline row had no runnable command while the D₄-TTA row did. Also adds the per-frame VRU instrument (scripts/perframe_vru.py), DW-IoU (src/gssc/utils/dw_iou.py), a --tau flag so the paper's temperature-invariance claim can be checked rather than taken on trust, configs/eval/round2_a.yaml, and configs for ablation rows that previously had none (train/{57k_mf,T10,T50,c1_lossmatched_t99}). Fixes three recipes that could not run as advertised (an unbound S3DSKDDataset, RareClassEnhancer reading dropped paste-budget fields, and eval/val_d4tta set to correction_steps: 1 where 4 is what reproduces the +D₄ val number), and corrects the retrain per-class deltas and the headline +2.36 pp val delta.
• 2026-06-10 — Release v2.2.0: zero-shot cross-dataset evaluation. The frozen headline checkpoint transfers with no fine-tuning and no target labels — SSCBench-KITTI-360 (val seq 06) 5.8 → 6.2 mIoU (+0.4) and 18.1 → 19.5 CompIoU (+1.4); SemanticPOSS (val seq 02, TALoS Tab. 4 map) 1.0 → 6.5 mIoU (+5.5) and 31.8 → 54.9 CompIoU (+23.1) — via scripts/eval_kitti360.py and scripts/eval_semanticposs.py. Also reconciles every JS3C-Net cross-base figure across the docs to one labelled scheme, and re-exports the LMSCNet EMA checkpoint with its 45 BatchNorm buffers so it reproduces 16.6 % val mIoU directly.
• 2026-05-26 — Release v2.1.0: LMSCNet third-base support. Stacked on the lightweight dense-2D-CNN LMSCNet (Roldao et al., 3DV 2020), one-step S²D² lifts val mIoU 14.8 % → 16.6 % (+1.8 pp) under the official semantic-kitti-api evaluator (paper tab:portable_s2d2; the LMSCNet base is re-scored from on-disk predictions, superseding the earlier 12.10 summary). Together with the v1.1.0 JS3C-Net row this gives three structurally different frozen bases (dense 2D CNN, point-voxel hybrid, sparse 3D CNN) all lifted by the same recipe and hyperparameters — base-agnostic by construction, not by tuning. Reproduce: python scripts/eval.py eval/lmscnet_val_1step --checkpoint data/checkpoints/gssc_lmsc/gssc_lmsc_s2d2_real/model_ema.safetensors (or python scripts/reproduce_table.py tab:portable_s2d2, which runs every cross-base row). We also trimmed the release surface: 22 unreferenced development modules were pruned, and the remaining V2/V3 FiLM denoiser variants are kept as clearly labeled research-reference prototypes (excluded from the public API and CI gating), so the dense-Conv3d headline path is the single supported surface.
• 2026-05-18 — Release v2.0.0: remove the legacy SCPNet-specific BEV-derivation flag (the pre-v1.1.1 name of --bev_from_base) in favour of --bev_from_base. base_pred_dir is now the preferred config key (used by the JS3C-Net and LMSCNet bases); scpnet_pred_dir is still accepted and remains in the SCPNet configs for backward compatibility. Headline numerical artefacts unchanged.
• 2026-05-14 — Release v1.1.0: JS3C-Net cross-base support. Stacked on the point-voxel hybrid JS3C-Net (Yan et al., AAAI 2021), one-step S²D² lifts JS3C-Net val mIoU 22.7 % → 24.3 % (+1.6 pp) — the paper's headline for this base, and what scripts/reproduce_table.py yields: derived BEV, scored by the official semantic-kitti-api on all 4,071 frames of seq 08. The same derived-BEV setting read instead by the paper's internal training-time evaluator reports 26.7 % (+4.0 pp against the official 22.7 % base, +4.3 against the internal 22.4 %); what separates 26.7 from 24.3 is the evaluator, not the BEV source, and the paper keeps that row only for continuity with earlier drafts, unmatched in protocol to the SCPNet and LMSCNet rows (supplementary Tab. XV). GT-BEV conditioning is a separate diagnostic, not either of those numbers: running the released real-only checkpoint with GT BEV instead of the derived one reads 26.05 % under the official api, a repo-measured figure the paper does not print — see docs/REPRODUCIBILITY.md. Release-asset layout migrated to per-checkpoint safetensors subdirs matching the modern HF Hub convention.
• 2026-04 — Public release v1.0.0. Headline checkpoint released under the repository's MIT licence; eval round-trip verified at 38.54 % val mIoU.
• 2026-04 — Secondary BEV-task reproduction path added (eval/bev_secondary config + driver, checkpoint bev/bev_s2d2_scpnet). LiDAR-only BEV refinement at 36.1 % mIoU, scored by the training-time 2D BEV evaluator on 100 fixed val frames (seed 42) — not the 4,071-frame semantic-kitti-api protocol.
• 2026-03 — 39.2 % mIoU on the SemanticKITTI hidden-test leaderboard (N = 4 with an 8-view D₄ ensemble; the single-step no-TTA submission scored 38.8 %) — paper under review; the preprint is on arXiv.

Method at a glance

_{<b>PS³ — the offline data-augmentation pipeline</b>, run once before any S²D² training: <i>input → generate → pair → pool</i>. <b>Input</b>: the real SemanticKITTI training split (10 sequences, 19,130 densely labelled 256²×32 frames, 20 classes, ≈5 % occupancy) is the only source signal; the split follows the standard SSC convention — sequences 00–07, 09, 10 train, sequence 08 val, sequences 11–21 hidden test (full layout in <a href="docs/DATASET.md">docs/DATASET.md</a>). <b>Pyramid</b>: three cascaded multinomial diffusions (𝒮₁ 32²×4 → 𝒮₂ 64²×8 → 𝒮₃ 256²×32, 100 steps, 20 classes), each denoiser ε_θ<i>i</i> conditioned on the trilinearly upsampled clean output ŷ_<i>i</i>−1 of the level below; the inset at right expands ε_θ itself (x_<i>t</i> → E1–E3 → bottleneck → D3–D1 → x̂₀), with the timestep <i>t</i> and the conditioning <b>c</b> entering at the bottleneck. 𝒮₃ tiles the 256² grid into 4 × 144²×32 sub-scenes, first-seen-wins. Raw samples are then screened on Jensen–Shannon divergence against a class-histogram prior, keeping the lowest-D_JS half. <b>Object bank</b>: instances drawn from a 57,789-crop, 8-rare-class bank (bicycle, motorcycle, truck, other-vehicle, person, bicyclist, motorcyclist, trunk) are pasted onto drivable surfaces, up to 10 per scene and <i>before</i> resampling. <b>Halo</b>: a virtual Velodyne HDL-64E then ray-traces the enriched dense scene Ỹ back into the sparse sweep X̃ a real sensor would have returned — 64 beams × 2,048 azimuth bins = 131,072 rays, up to 2 returns per beam, σ_r = 3 cm, Bresenham-3D marching — so each generated scene arrives already paired with its own observation at no annotation cost (≈1 % of voxels occupied, against ≈5 % in the dense scene). <b>Output</b>: 32,039 synthetic pairs pooled with the 19,130 real frames give a 51,169-frame training set, a 2.67× expansion, usable as a drop-in replacement for the real split. Source: paper Fig. 2 (<code>fig:ps3_pipeline</code>). The S²D² deployment path this repository ships is the next figure.}

Why it works. Pure-noise diffusion wastes capacity learning to invert random corruption. By starting from the base model's structured prediction x_src rather than x_T ∼ π, the network only has to learn the residual between x_src and ground truth — a much smaller chunk of probability mass to transport. One correction step suffices for the headline 38.54 % val mIoU; four steps + D₄ TTA push to 39.2 % test.

_{<b>Top — the intuition.</b> The learned velocity field <b>v</b>_θ = softmax(<i>f</i>_θ(<b>x</b>_<i>t</i>, <i>t</i>, <b>c</b>)) − <b>x</b>_src drags the frozen base model's prediction (grey cluster) onto the ground truth (orange cluster) in a single Euler step <b>x</b>_<i>t</i>−1 = <b>x</b>_<i>t</i> + Δᾱ_<i>t</i>·<b>v</b>_θ on the per-voxel simplex; richer conditioning <b>c</b> = {<b>L</b>, <b>B</b>} (raw-sweep occupancy and the base-derived BEV map) sharpens the transport. <b>Middle — why one step is viable.</b> DDPM starts at pure noise <b>x</b>_<i>T</i> and follows a curved path over <i>T</i> steps; S²D²'s forward path <b>x</b>_<i>t</i> = ᾱ_<i>t</i><b>x</b>₀ + (1−ᾱ_<i>t</i>)<b>x</b>_src starts at the base prediction and is a straight chord, so a single step (<i>N</i> = 1) lands the clean scene shown at right (held-out val seq 08, frame 001850). Three results carry that: Prop. 3, the per-step ELBO is an equality rather than a bound; Thm. 1, the error telescopes to Σ_<i>k</i> w_<i>k</i>ε_<i>k</i> + 2ᾱ_<i>T</i> ≤ max_<i>k</i> ε_<i>k</i> + 2ᾱ_<i>T</i>, so nothing compounds across steps; and Cor. 1, at <i>N</i> = 1 only the single residual ε₁ at <i>t</i> = <i>T</i> survives. All three are proved in supplementary App. B. <b>Bottom — one scene.</b> A motorcyclist on val seq 08 frame 001390: the frozen base scores 27.3 % IoU on that class, one Euler step reaches <b>41.5 %</b> (+14.2 points on this sample), against 322 ground-truth motorcyclist voxels. Those are per-sample chips rendered for the figure, not dataset numbers and not tabulated in the paper; the full-val motorcyclist gain for the released checkpoint is 4.1 → 12.4 (+8.3), tabulated in paper <code>tab:perclass_delta</code> beside a retrain column that does not reproduce it — the paper reads that one class as a seed-and-split artefact, so discount it (the FAQ below carries the retrain figures). Source: paper Fig. 5 (<code>fig:s2d2</code>).
The denoiser <i>f</i>_θ is not drawn here: it is the four-level dense 3D U-Net of paper Fig. 4 (<code>fig:sgsc</code>), channels 32 → 256, retrained from scratch against the structured source, with <b>L</b> and <b>B</b> injected additively at every level and the timestep entering only through AdaGN.}

Headline numbers — SemanticKITTI hidden test (single-frame single-sample LiDAR)

_{<b>Source.</b> Every mIoU and IoU_cmpl cell above — baseline rows and both S²D² rows — is paper-reported, in the paper's main results table (Tab. I); the Δ column is those values differenced against SCPNet's 36.7. Both of our rows are scores the blind test server returned (supplementary App. G).}

On full SemanticKITTI val seq 08 (note: val numbers below, distinct from the 39.2 % hidden-test figure above):

• val: 38.54 % mIoU (single correction step, $N{=}1$, no TTA) — verified end-to-end by the maintainers (requires the released assets); the same $N{=}1$ no-TTA setting scores 38.8 % on the hidden test (the 38.54 val / 38.8 test pair in the test table above) — to our knowledge the best causal, single-sweep, single-sample result on the leaderboard to date, +2.1 pp over the previous best published score under that restriction (SCPNet, 36.7)
• val: 38.73 % mIoU (N = 4 correction steps + D₄ TTA) → the same recipe scores 39.2 % on the hidden test
• +2.36 absolute over our SCPNet base (36.17 % val)

_{Three frozen sources (JS3C-Net, SCPNet, and TALoS's test-time adaptation of SCPNet), our S²D² on the SCPNet base, and ground truth, left → right. <b>Top — bicyclist</b>, val seq 08 frame 003096: per-sample IoU 0.7 / 31.3 / 33.0 % for the three sources against our <b>56.9 %</b>, with false positives cut 3,009 → 1,956 and true positives raised 1,830 → 2,724 against 2,833 ground-truth voxels. <b>Bottom — motorcyclist</b>, frame 001417: 0.0 / 33.0 / 30.0 % against our <b>62.3 %</b>, with false positives cut 621 → 94 against 315 ground-truth voxels. On both scenes the single correction roughly doubles SCPNet's per-sample rare-class IoU while cutting phantom voxels 1.5× (top) and 6.6× (bottom); JS3C-Net collapses on both, predicting no motorcyclist voxels at all. The bulk of each scene — road, sidewalk, building, vegetation — moves by under 1 % IoU. These chips are per-sample figures read off the <i>N</i> = 4, +<i>D</i>₄-TTA configuration, so they characterise the correction rather than the <i>N</i> = 1 operating point the headline reports; the raw voxel counts behind them are supplementary Tab. XIV. Source: paper Fig. 6 (<code>fig:qualitative</code>).}

Per-class IoU on val seq 08 (single correction step, verified)

_{From python scripts/eval.py eval/val_1step --checkpoint data/checkpoints/gssc_mf/gssc_31k_mf_step40000/model_ema.safetensors. Numbers below are the paper's per-class val row for this checkpoint (SCPNet + S²D², N = 1), as printed in the supplementary's full per-class table (tab:supp_portable_full).}

Quick start (reproduce 38.54 % val in three commands)

# 1. Clone + install (uv recommended)
git clone https://github.com/BillyChern/GSSC-S2D2.git && cd GSSC-S2D2
curl -LsSf https://astral.sh/uv/install.sh | sh
uv venv --python 3.10 && uv sync && uv pip install spconv-cu126==2.3.8
#    ^ CUDA 11.8 users: use spconv-cu118==2.3.8 instead (see the spconv note below)
source .venv/bin/activate
#    ^ `uv venv` creates .venv but does not put it on PATH; steps 2 and 3 call a bare
#      `python`, which without this line is the system interpreter (use `uv run python …`
#      instead if you prefer not to activate)

# 2. Pull pretrained checkpoint + the SCPNet predictions step 3 reads (val seq 08)
python scripts/download_assets.py --checkpoints
python scripts/download_assets.py --predictions --include 'scpnet_predictions/08/*'
# → data/checkpoints/gssc_mf/gssc_31k_mf_step40000/model_ema.safetensors           (~140 MB; full subdir 265 MiB / 278 MB — see docs/MODEL_ZOO.md)
# → data/scpnet_predictions/08/   12,213 files = all 4,071 val frames, 8.4 GiB / 9.1 GB on disk
#    The predictions ship as zstd archives; --include picks the ONE that can hold seq 08,
#    so this transfers 703 MiB / 737 MB and unpacks only sequence 08 out of it. Needs zstd
#    (the `zstandard` package, which `uv sync` installs, or the `zstd` binary).
#    Drop --include for the whole SCPNet group: 3.10 GiB / 3.33 GB down, 324 GiB / 348 GB
#    unpacked (see docs/DATASET.md). --include narrows the prediction/object-bank groups;
#    --checkpoints is always a whole-repo snapshot.

# 3. Reproduce the headline 38.54 % val mIoU
python scripts/eval.py eval/val_1step \
    --checkpoint data/checkpoints/gssc_mf/gssc_31k_mf_step40000/model_ema.safetensors

Note on spconv. spconv is deliberately not pinned in uv.lock because it ships a CUDA-specific wheel; uv sync will not install it. Install it explicitly with the pinned, CUDA-coherent line shown above (uv pip install spconv-cu126==2.3.8). Note that the pinned torch==2.4.0 PyPI wheel bundles its own CUDA 12.1 runtime libraries (nvidia-*-cu12), so torch carries cu121 regardless of your local toolkit; only the separately-installed spconv wheel must be matched to the local CUDA. The validated combination is cu121-torch (2.4.0 stock wheel) + cu126-spconv (2.3.8) — match the spconv wheel to your local CUDA toolkit if you deviate (e.g. for CUDA 11.8 use uv pip install spconv-cu118==2.3.8). Both libraries load on a ≥12.6 driver via CUDA minor-version forward compatibility, so the install line stays correct on the reference CUDA 12.8 box in the hardware table below. The stock PyPI wheel is sufficient: the SCPNet v1→v2 kernel-shape "patches" are applied in code at weight-load time, so no custom-built spconv is required.

Once the released assets are in place, that is the whole pipeline. Expected output (truncated):

[gssc.inference.evaluate] Eval config: eval/val_1step
[gssc.inference.evaluate] Sequences=08 steps=1 tta=none
[gssc.inference.evaluate] Stage 1 done. Predictions written under <tmpdir>
[gssc.inference.evaluate] Stage 2 (score): ... evaluate_completion.py ...
[gssc.inference.evaluate] mIoU         38.54 %
[gssc.inference.evaluate] IoU_cmpl     52.66 %

============================================================
 GSSC-S2D2 evaluation: eval/val_1step
============================================================
  mIoU       : 38.54 %
  Completion : 52.66 %   (val seq 08)
------------------------------------------------------------
 Per-class IoU:
  bicycle               24.30 %
  bicyclist             23.20 %
  car                   51.40 %
  motorcyclist          12.40 %
  pole                  37.80 %
  ...
============================================================

_{The per-class lines above are an abbreviated, illustrative excerpt; the
38.54 % val mIoU is the anchor number to check against. The eval script
prints the full 19-class table at runtime, so the exact log format may evolve
across releases — match on the mIoU value, not the verbatim layout.}

_{Note: the IoU_cmpl 52.66 % here is on val seq 08, distinct from the 58.8–59.0 % IoU<sub>cmpl</sub> reported on the hidden test in the headline table above — the same val-vs-test split flagged for the mIoU column.}

For the full hidden-test leaderboard submission flow (39.2 % via D₄ TTA), see docs/INFERENCE.md.

Secondary task: LiDAR-only BEV refinement

S²D² is not specific to 3D scene completion. The same correction-sampling
mechanism, applied to a 2D BEV diffusion model, refines the SCPNet-derived
base BEV map and lifts BEV mIoU from 34.8 % (the parameter-free
base-derived projection) to 36.1 % (S²D²-refined) on val seq 08, a
+1.3 pp gain — paper tab:bev_results. Both figures are scored by the
training-time 2D BEV evaluator on 100 fixed val frames (seed 42), not the
4,071-frame semantic-kitti-api protocol every 3D number in this README uses,
so they are not comparable with the 3D mIoU figures above.

# After step 3 above (predictions already downloaded), eval the BEV pipeline:
python scripts/eval.py eval/bev_secondary \
    --checkpoint data/checkpoints/bev/bev_s2d2_scpnet/model.safetensors

To retrain the BEV-A checkpoint from scratch (~24 h on 1× H100):

python scripts/train.py train/bev_secondary

Cross-dataset zero-shot (KITTI-360, SemanticPOSS)

The headline SemanticKITTI checkpoint transfers to other LiDAR domains with no
fine-tuning and no target labels — the frozen gssc_31k_mf_step40000
checkpoint at N=1, single-frame, no TTA. On SSCBench-KITTI-360 val (seq 06,
1,812 frames, 16 shared classes; same-sensor near-domain) S²D² lifts the SCPNet
base from 5.8 → 6.2 mIoU (+0.4) and 18.1 → 19.5 CompIoU (+1.4); the paper
notes the mIoU delta sits inside the assumed single-seed band, so there the claim
rests on completion IoU. On the harder cross-sensor SemanticPOSS val (seq 02,
≈500 frames, 11-class TALoS Tab. 4 map) it lifts 1.0 → 6.5 mIoU (+5.5) and 31.8 → 54.9 CompIoU (+23.1).

# SSCBench-KITTI-360 zero-shot (after provisioning per docs/DATASET.md)
python scripts/eval_kitti360.py eval/kitti360_zeroshot_1step \
    --checkpoint data/checkpoints/gssc_mf/gssc_31k_mf_step40000/model_ema.safetensors

# SemanticPOSS zero-shot
python scripts/eval_semanticposs.py eval/semanticposs_seq02 \
    --checkpoint data/checkpoints/gssc_mf/gssc_31k_mf_step40000/model_ema.safetensors

Both runs are frozen-checkpoint zero-shot: the SemanticKITTI-trained weights are
applied as-is, with no adaptation to the target domain.

Repository layout

GSSC-S2D2/
├── src/gssc/                       # the Python package
│   ├── models/                     # denoiser: dense 3D U-Net (Conv3d, this release ≈ 35M); "Sparse" in class names (e.g. SceneCompletionUNetSparse) refers to the aux LiDAR encoder, not the denoiser. Plus SCPNet base, pyramid, BEV variant
│   ├── diffusion/                  # forward process, Dirac posterior, correction sampler, D4 TTA
│   ├── data/                       # SemanticKITTI loader, synthetic pool, object bank, HDL-64E ray-tracer
│   ├── losses/                     # Lovász-Softmax (the KL posterior, auxiliary and focal-CE terms live in gssc/diffusion/multinomial.py)
│   ├── training/                   # canonical trainer + EMA + logging
│   ├── inference/                  # eval (3D SSC mIoU + Completion IoU, 2D BEV mIoU), D4 TTA, prediction generation
│   └── utils/                      # config loader, checkpoint I/O + binding checks, v1.x deprecation shims, DW-IoU
├── configs/                        # Hydra-style configs: plain YAML, no Hydra runtime (one per recipe in the paper)
│   ├── train/{0K_sf,10K_sf,20K_sf,31k_sf,31k_mf,57K_sf,57k_mf,T10,T50,T100skewed,bev_secondary,c1_lossmatched_t99,js3c_real,js3c_real_derived,js3c_real_gtbev,lmscnet_real}.yaml
│   ├── eval/{val_1step,val_d4tta,step_sweep,timestep_ablation,data_scaling_sf,dwiou_sweep,round2_a,bev_secondary,js3c_val_1step,js3c_val_d4tta,js3c_val_paper,js3c_val_realistic,lmscnet_val_1step,kitti360_zeroshot_1step,semanticposs_seq02}.yaml
│   └── infer/{test_1step,test_d4tta,val_1step,val_d4tta}.yaml   # test_1step is the N=1 no-TTA hidden-test row
├── scripts/                        # one-command drivers
│   ├── train.py
│   ├── eval.py
│   ├── infer.py
│   ├── prepare_data.py
│   ├── download_assets.py
│   ├── reproduce_table.py
│   ├── eval_kitti360.py            # cross-dataset zero-shot
│   ├── eval_semanticposs.py        # cross-dataset zero-shot
│   ├── dump_js3c_predictions.py    # base-prediction dumpers
│   ├── dump_lmscnet_predictions.py
│   └── ...                         # fps_measure_dense_vs_sparse, score_kitti360, writeback_scpnet_baseline_kitti360, check_no_ai_attribution
├── docs/                           # full reproducibility documentation
│   ├── REPRODUCIBILITY.md
│   ├── DATASET.md
│   ├── MODEL_ZOO.md
│   ├── TRAIN.md
│   ├── INFERENCE.md
│   └── BASELINES.md
├── tests/                          # pytest unit + smoke tests
├── examples/                       # Jupyter notebooks for new users
├── external/                       # third-party (semantic-kitti-api, multinomial_diffusion)
├── assets/                         # README figures (teaser, architecture, qualitative)
├── data/                           # runtime dataset root (gitignored): checkpoints + base predictions land here
├── .github/                        # CI workflows (test, lint, release) + issue/PR templates
├── CITATION.cff                    # citation metadata
├── CONTRIBUTING.md                 # code-quality standards
├── CHANGELOG.md                    # release history
├── SECURITY.md                     # security policy
├── .pre-commit-config.yaml         # pre-commit hooks (ruff, ruff-format, pytest smoke)
├── pyproject.toml                  # uv-managed Python project
├── uv.lock                         # pinned dependency lockfile
├── THIRD_PARTY_NOTICES.md          # upstream licences + the attributions our artefacts carry
└── LICENSE                         # MIT

Reproducing every paper number

The repo ships the exact recipe + checkpoint for every reported number.

Full mapping with anticipated wall-clock and disk requirements: docs/REPRODUCIBILITY.md.

Retraining the headline (≈ 37 GPU-hours to step 40000, one H100 80 GB)

python scripts/train.py train/31k_mf --gpu 0 --seed 42
# 100K iterations, batch size 4 (four samples on ONE card), AdamW lr 1e-4
# The released checkpoint is step 40000 (~37 GPU-h); running all 100K costs ~90 GPU-h
# Logs → outputs/train_31k_mf/{tensorboard,step_*.pt,best.pt}

The released scene-completion trainer is single-GPU: gssc.training.train_scene_completion implements no DataParallel, no DistributedDataParallel and no torch.distributed, and scripts/train.py launches one plain subprocess. --gpu 0,1 only sets CUDA_VISIBLE_DEVICES=0,1, making a second card visible and leaving it idle, so there is no wall-clock halving and the GPU-hours above are wall-clock on one card (docs/TRAIN.md has the full note; the pyramid trainers are a separate entry point and do implement DDP).

A from-scratch seeded retrain lands at ≈ 38.05 % val 1-step mIoU (success criterion 38.05 % ± 0.3 %, i.e. [37.8 %, 38.4 %]), within seed noise of the 38.54 % released headline checkpoint — reproduce the per-class delta over the SCPNet base, not the absolute (see docs/REPRODUCIBILITY.md). See docs/TRAIN.md for every other recipe (data scaling, timestep ablations, pyramid diffusion, BEV second task).

Hardware + environment

Pinned versions in uv.lock. See docs/REPRODUCIBILITY.md for the exact environment matrix.

Three ideas behind S²D²

1. Structured source. Replace the noise endpoint with a learned base model's prediction $x_{\text{src}}$. The forward process becomes the Dirac mixture $x_t = \bar\alpha_t \cdot x_0 + (1 - \bar\alphat) \cdot x{\text{src}}$, a deterministic interpolant between ground truth and $x_{\text{src}}$.
2. S²D² correction sampling. A non-noise deterministic reverse process (we specialise the non-noise correction sampler of Cold Diffusion, Bansal et al. 2022, to our linear simplex interpolant) that routes the full residual $\hat{\mathbf{x}}0 - \mathbf{x}{\text{src}}$ directly per step. At $N = 1$, the iterate at $t = T$ coincides with $\mathbf{x}_{\text{src}}$, giving a Lipschitz-free single-step bound (cor:supp_onestep, with the Lipschitz term priced in cor:supp_lipprop; supplementary App. B-C).
3. Pyramid diffusion data augmentation. A coarse-to-fine pyramid ($32^2{\times}4$ → $64^2{\times}8$ → $256^2{\times}32$) generates synthetic $(\text{sparse}, \text{complete})$ pairs. Combined with a 57,789-instance / 8-rare-class object bank (bicycle, motorcycle, truck, other-vehicle, person, bicyclist, motorcyclist, trunk) and HDL-64E Bresenham3D ray-tracing, this yields the 32,039-scene synthetic pool used by the headline configuration (pooled with the 19,130 real frames for a 51,169-frame total, a 2.67× expansion).

The mathematical derivations are in the supplementary, App. B (suppB:math): prop:supp_forward, prop:supp_posterior, prop:supp_elbo, prop:supp_meanflow, thm:supp_error, cor:supp_onestep, cor:supp_margin, cor:supp_lipprop, prop:supp_proj. The flow-matching correspondence (Lipman et al.) is argued in prose under Velocity / mean-flow formulation (App. B-B, suppB:velocity) — the paper states no numbered flow-matching proposition, so earlier revisions of this file pointed at a label that does not exist.

Asset releases

Sizes above were measured on the staged release payload — download figures from the uploaded archives, on-disk figures with du -Lsb (apparent size, symlinks dereferenced) — and are quoted as GiB / GB so the two conventions cannot be confused; docs/DATASET.md carries the same table per archive and per sequence. The three prediction trees and the object bank are 609,349 files uncompressed, and several of their directories exceed the 10,000 files Hugging Face serves per directory, so they ship as ten zstd archives — nine holding the predictions, one the object bank — rather than as a per-frame tree; scripts/download_assets.py downloads only the archives a request needs and extracts them into the data/<prefix>/… layout the docs describe, so no manual tar step is involved. Unpacking needs zstd (the zstandard package or the binary). The 0K, 10K and 20K data-scaling rows are retrains, not released pools — 0K means real frames only, and the 10K / 20K subsets are not staged for release.

These artefacts are released under two different terms. GSSC-authored code and the GSSC-trained model weights are MIT. The synthetic pool, the object bank and the base-model prediction dumps are derived from SemanticKITTI, which is distributed under CC-BY-NC-SA 4.0, so they inherit its non-commercial, share-alike restriction; the model weights, although MIT as our contribution, were also trained on SemanticKITTI, so downstream use of the weights still carries that non-commercial caveat. SemanticKITTI raw data follows its own license (see semantic-kitti.org). scripts/download_assets.py populates every entry in the table, the synthetic pool included: --synthetic-pool {31K,57K} pulls the requested archive from the free Hugging Face mirror and then prints the IEEE DataPort DOI to cite it by. What it cannot do is fetch from DataPort itself — that record publishes no direct file URL and releases files only to a signed-in, subscribed session — so the mirror is the route the script takes. If a repo is unreachable the command exits with the manual-build instructions in docs/DATASET.md, which also documents manual provisioning for the rest.

Code-quality + testing

This codebase targets production-grade standards. CI enforces a subset (ruff check and mypy on the public API surface, plus a torch-free pytest smoke pass — see .github/workflows/); the remaining checks below are run locally before release:

ruff check src/ tests/ scripts/      # style + import order   (CI-enforced)
mypy                                 # static types           (CI-enforced; scope = the `[tool.mypy] files` list in pyproject.toml)
mypy src                             # static types, whole tree (local, advisory — not clean today, and no workflow runs it)
pytest tests/ -v                     # unit + smoke tests      (light subset CI-enforced)
pytest --cov=gssc --cov-report=term  # coverage (local)
vulture src/                         # dead code (local)
bandit -r src/                       # security (local)

Style conventions, commit conventions, and hard requirements: CONTRIBUTING.md.

FAQ

Q. Why use this over running SCPNet alone?
A. We add +2.1 absolute mIoU on the hidden test set with a single extra forward pass (107.2 ms marginal added-step latency on H100 — the same figure as the 9.33 FPS marginal throughput quoted above, since 1000 / 107.2 ≈ 9.33), no extra training data beyond what SCPNet was trained on, and no distillation. (+2.5 is the four-step, 8-view-ensemble row, so it is not what one extra pass buys.) The gains concentrate on safety-critical rare classes. For the released checkpoint these are motorcyclist +8.3, bicyclist +5.3, truck +5.3, other-vehicle +2.5, person +1.2 on val seq 08 (paper tab:perclass_delta, Released column). A separate, independent spconv-v2 from-scratch retrain (documented in docs/REPRODUCIBILITY.md, not the shipped release weights) holds most of that structure on a different seed — truck +5.4, bicyclist +5.2, other-vehicle +2.3, person +1.9, each within 0.7 pp of the released checkpoint — but the motorcyclist gain does not reproduce: it collapses from +8.3 to +0.3, and that single class accounts for 87 % of the retrain's lower overall delta (+1.9 against +2.36). Treat the rare-class gains as reproducible in structure, with motorcyclist as the documented seed-sensitive exception, not as a per-class guarantee (paper tab:perclass_delta, Retrain column).

Q. Can S²D² be applied to a different base SSC network?
A. Yes — the framework is base-model-agnostic. We provide a working SCPNet integration; switching to JS3C-Net or any other base requires only providing per-voxel categorical predictions as x_src. See docs/BASELINES.md.

Q. Do we need the synthetic pool to use the released checkpoint?
A. No. Eval-only deployment uses the released weights plus the SCPNet predictions for val seq 08 — a ~96 GB stack once the raw SemanticKITTI voxels are counted (80 GB raw + 1.6 GB voxel labels + 4.9 GB checkpoints + 9.1 GB SCPNet seq 08; docs/DATASET.md itemises it). The synthetic pool (136 GB for the 31K headline variant, 246 GB for the 57K variant) is only needed for retraining from scratch.

Q. Why does the train script use a YAML "config" rather than direct CLI args?
A. Every paper artefact corresponds to a config file. python scripts/train.py train/31k_mf runs the exact headline recipe with no chance of accidentally diverging from the paper.

Q. Single-seed numbers — why no error bars?
A. Same convention as every method in the leaderboard table: a SemanticKITTI SSC training run is expensive (ours: ~37 GPU-hours to the released step-40000 checkpoint, ~90 for the full 100K-iteration launch), and the official scoring server takes a single submission. We use a single seed (42) to match this convention. See the paper's §IV Baselines and protocol run-in, and supplementary Appendix H (Single-seed reporting), for the variance-disclosure discussion.

Q. Where do I find figures and paper-typesetting source?
A. The three README figures (assets/teaser.png, assets/architecture.png, assets/qualitative.png) live under assets/ and are embedded above. The full paper figure set and the LaTeX typesetting source live in the paper repo, which stays private; the manuscript itself is public as an arXiv preprint. This repo focuses on method reproduction; the shipped notebook (examples/quickstart.ipynb) walks through the headline eval end-to-end.

Citation

@misc{chen2026gssc,
  title         = {Generative Semantic Scene Completion},
  author        = {Chen, Shi and Ge, Weifeng},
  year          = {2026},
  eprint        = {2608.26737},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV},
  doi           = {10.48550/arXiv.2608.26737},
  url           = {https://arxiv.org/abs/2608.26737},
  note          = {Under review}
}

(Machine-readable: CITATION.cff)

If you train on the PS³ synthetic pool, cite the deposit as well — it carries its own DOI and is a separate artefact from the paper:

Shi Chen, Weifeng Ge, "PS3-SemanticKITTI: Paired Sparse-Dense Synthetic Scenes for LiDAR Semantic Scene Completion", IEEE Dataport, August 23, 2026, doi:10.21227/nqgf-9k39

@misc{chen2026ps3semantickitti,
  title        = {PS3-SemanticKITTI: Paired Sparse-Dense Synthetic Scenes for LiDAR Semantic Scene Completion},
  author       = {Chen, Shi and Ge, Weifeng},
  year         = {2026},
  publisher    = {IEEE Dataport},
  doi          = {10.21227/nqgf-9k39},
  url          = {https://dx.doi.org/10.21227/nqgf-9k39},
  note         = {Dataset}
}

IEEE DataPort's own generated entry uses @data{nqgf-9k39-26, ...}; @data is not a standard BibTeX type, so the portable @misc form is given here. Both resolve to the same deposit. The pool is derived from SemanticKITTI, so its citation requirements (below) apply on top of this one.

Cite this DOI whichever host you took the bytes from. The archives are also mirrored, byte-identical, on Hugging Face at Stone-Chern/PS3-SemanticKITTI — that mirror is the free download route (public, no IEEE subscription, anonymous download) and is what scripts/download_assets.py --synthetic-pool {31K,57K} fetches, but it is not a citable identifier and does not replace the DOI above.

The submission snapshot referenced in the paper supplementary is the v2.4.1 release; its Hydra configs hold the same hyperparameters listed in the paper's reproducibility appendix. Keep this tag in step with the one named in the paper's reproducibility appendix — the paper cites the tag by name, so bumping one without the other strands the reference.

License

Every upstream licence this project redistributes or derives from, with the copyright holder and the exact files affected, is recorded in THIRD_PARTY_NOTICES.md; the distributed wheel ships a copy at gssc/THIRD_PARTY_NOTICES.md. The bullets below are the summary, not the notice.

• Code, configs, documentation: MIT License.
• Released model weights: the GSSC-authored code and weights are licensed MIT. The weights were trained on SemanticKITTI, which is distributed under CC-BY-NC-SA 4.0 (non-commercial); downstream use of the weights therefore inherits that dataset's non-commercial restriction, so the MIT grant on our contribution does not by itself authorise commercial use of the trained weights.
• Synthetic pool + object bank: derived from SemanticKITTI and therefore distributed under the same CC-BY-NC-SA 4.0 (non-commercial, share-alike) terms; see semantic-kitti.org/dataset.html. The pool is deposited on IEEE DataPort as doi:10.21227/nqgf-9k39 — the identifier to cite — and mirrored for free download on Hugging Face at Stone-Chern/PS3-SemanticKITTI; that redistribution is what the CC-BY-NC-SA 4.0 grant permits, and the mirrored archives ship the same LICENSE. It has its own citation (see Citation above).
• Base-model prediction dumps (scpnet_predictions/, js3cnet_predictions/, lmscnet_predictions/) — the largest published artefact at 411 GiB / 441 GB: model output computed over SemanticKITTI, so CC-BY-NC-SA 4.0 applies to them as well. Each additionally carries its producing model's attribution — SCPNet (no upstream licence; weights redistributed with the authors' permission), JS3C-Net (MIT, © 2020 Xu Yan), LMSCNet (Apache-2.0, © 2020 Inria and AKKA Technologies) — see THIRD_PARTY_NOTICES.md §8–§11.
• SemanticKITTI raw data: governed by its own license — see semantic-kitti.org. Its terms require citing both J. Behley, M. Garbade, A. Milioto, J. Quenzel, S. Behnke, C. Stachniss and J. Gall, "SemanticKITTI: A Dataset for Semantic Scene Understanding of LiDAR Sequences", ICCV 2019, and A. Geiger, P. Lenz and R. Urtasun, "Are We Ready for Autonomous Driving? The KITTI Vision Benchmark Suite", CVPR 2012, pp. 3354–3361. Anything we publish that is derived from it (the pools, the bank, the prediction dumps) carries the same obligation.
• SSCBench-KITTI360 (evaluation-only): governed by its own terms; see the SSCBench repository.
• SemanticPOSS (evaluation-only): governed by its own terms; see the SemanticPOSS dataset page.
• SCPNet base weights (scpnet_v2_port.pth): SCPNet's own released checkpoint, carried unmodified and redistributed here with the SCPNet authors' permission under this repository's MIT licence — the "port" in the filename refers to the spconv-2.3 kernel-shape patches gssc.inference.run_scpnet applies at load time, not to a modified file; the upstream SCPNet code and weights remain governed by their own terms (see THIRD_PARTY_NOTICES.md §8).
• Third-party code: not confined to external/. Alongside the vendored trees there, parts of the package itself are third-party in origin — notably src/gssc/models/pyramid_unet.py (a substantial copy from Pyramid Discrete Diffusion) and the src/gssc/_improved_diffusion/ fork. Each retains its original licence and carries an attribution header; the full inventory, file by file, is in THIRD_PARTY_NOTICES.md.

Acknowledgements

Supported by NSFC Grant No. 624B1006. Thanks to the SemanticKITTI authors for the public benchmark and raw data. Open-source software this project builds on:

• Pyramid Discrete Diffusion (Liu et al., 2023) — multi-scale discrete diffusion for 3D scene synthesis; foundation of our offline data augmentation pipeline (𝒮₁/𝒮₂/𝒮₃ + rare-class object bank + HDL-64E ray-tracing).
• improved-diffusion (OpenAI, MIT) — diffusion training and sampling scaffolding our diffusion code adapts.
• SCPNet (SCPNet/Codes-for-SCPNet) — sparse-3D-CNN base SSC model used as the frozen headline base.
• JS3C-Net (yanx27/JS3C-Net) — point-voxel hybrid base used for the cross-base lift.
• LMSCNet (astra-vision/LMSCNet, Apache-2.0) — dense 2D-CNN base used for the third cross-base lift.
• D3PM / Multinomial Diffusion (Austin et al., NeurIPS 2021) — discrete diffusion family that S²D² generalises with a structured source.
• Cold Diffusion (Bansal et al., 2022) — non-noise correction sampling whose linear-simplex specialisation underlies our $N=1$ deployment.
• spconv 2.3 (traveller59/spconv) — sparse 3D convolution backend.
• TALoS (NeurIPS 2024) — line-of-sight test-time adaptation for SemanticKITTI SSC, included as the leaderboard reference baseline; its test row is in the headline table above. Its adaptation aggregates other moments, future ones included, so the paper excludes it from the causal, single-sweep, single-sample comparison and from Tab. I's bolding.

⭐ Star history

<a href="https://www.star-history.com/#BillyChern/GSSC-S2D2&Date">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=BillyChern/GSSC-S2D2&type=Date&theme=dark" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=BillyChern/GSSC-S2D2&type=Date" />
<img alt="Star history" src="https://api.star-history.com/svg?repos=BillyChern/GSSC-S2D2&type=Date" />
</picture>
</a>