Promote Sign in Sign up

Repository

GSFixer

[ICML 2026] GSFixer: Improving 3D Gaussian Splatting with Reference-Guided Video Diffusion Priors

GVC Lab
Repository

README

GSFixer: Improving 3D Gaussian Splatting with Reference-Guided Video Diffusion Priors

Xingyilang Yin*<sup>1,2,3</sup>, Qi Zhang*<sup>2</sup>, Jiahao Chang<sup>4</sup>, Ying Feng<sup>2</sup>,

Qingnan Fan<sup>2</sup>, Xi Yang<sup>5</sup>, Chi-Man Pun<sup>1,📫</sup>, [Huaqi Zhang]()<sup>2,</sup>, Xiaodong Cun<sup>3,📫</sup>

ArXiv | PDF | Model | Bench | International Conference on Machine Learning (ICML) 2026

<sup>1</sup> University of Macau, <sup>2</sup> vivo BlueImage Lab, <sup>3</sup> GVC Lab, Great Bay University, <sup>4</sup> CUHKSZ, <sup>5</sup> Xidian University

🤗 If you find GSFixer useful, please help ⭐ this repo, which is important to Open-Source projects. Thanks 🙏!

TL;DR: using 3D geometric (VGGT) and 2D semantic (DINOv2) features of reference views (sparse input views) to guide the video diffusion process, enabling consistency in restoring artifact novel views.

Below shows some examples:

<table class="center">
<tr style="font-weight: bolder;">
<td>&emsp;&emsp;3DGS Artifact&emsp;&emsp;&emsp;&emsp;&emsp; Difix3D+&emsp;&emsp;&emsp;&emsp;&emsp; GenFusion&emsp;&emsp;&emsp;&emsp;&emsp;GSFixer (Ours)&emsp;&emsp;&emsp;&emsp;&emsp; GT</td>
</tr>
<tr>
<td>
<img src=assets/comparison1.gif style="width: 100%; height: auto;">
</td>
</tr>
<tr>
<td>
<img src=assets/comparison2.gif style="width: 100%; height: auto;">
</td>
</tr>
<tr>
<td>
<img src=assets/comparison3.gif style="width: 100%; height: auto;">
</td>
</tr>
</table>

⚙️ Setup

1. GSFixer

git clone https://github.com/GVCLab/GSFixer.git
cd GSFixer

2. Setup environments

GSFixer is tested with CUDA 12.1 on H20 & H100.

conda create -n gsfixer python=3.10
conda activate gsfixer
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu121
pip install -r requirements.txt
pip install Reconstruction/submodules/simple-knn
pip install Reconstruction/submodules/diff-gaussian-rasterization

3. Download pretrained models

Download the pretrained models (e.g., GSFixer, CogVideoX-I2V, VGGT...) using HuggingFace.

sh download/download_hf.sh

4. Download the datasets

Download Mip-NeRF 360 dataset from their project page.

Download DL3DV-Benchmark dataset using their download script.

Download the DL3DV-Res dataset using HuggingFace:

python download/download_data_hf.py

💫 Inference

1. Inference Restoration Demo

cd Restoration
python examples/gsfixer/inference_gs_restoration_demo.py \
--model_name PATH-to-CogVideoX-5b-I2V \
--transformer_path PATH-to-GSFixer \
--dinov2_ckpt PATH-to-dinov2-with-registers-large \
--vggt_ckpt PATH-to-vggt-model.pt \
--blip_path PATH-to-blip2-opt-2.7b \
--images_root ./demo/032dee9fb0a8bc1b90871dc5fe950080d0bcd3caf166447f44e60ca50ac04ec7 \
--outpath ./output_gsfixer_inference_results \

2. Generative Novel View Restoration

Run evaluate.sh using the following script. Please refer to the args to set up parameters.

cd Restoration
bash examples/gsfixer/evaluate.sh

3. Generative 3D Reconstruction

Run train.py using the following script. Please refer to the args to set up parameters.

cd Reconstruction
# for per-scene results
python train_mipnerf360_sparse_view_recon_per_scene.py

# for all results
python train_mipnerf360_sparse_view_recon_all_scene.py

Including but not limited to: CogVideoX, diffusers, VGGT, DINOv2, Difix3D+, GenFusion, 3DGS-Enhancer, ReconX, ViewCrafter, TrajectoryCrafter, ReCamMaster...

📜 Citation

If you find the work useful, please consider citing:

@article{yin2025gsfixer,
  title={GSFixer: Improving 3D Gaussian Splatting with Reference-Guided Video Diffusion Priors},
  author={Yin, Xingyilang and Zhang, Qi and Chang, Jiahao and Feng, Ying and Fan, Qingnan and Yang, Xi and Pun, Chi-Man and Zhang, Huaqi and Cun, Xiaodong},
  journal={arXiv preprint arXiv:2508.09667},
  year={2025}
}