PolyLayout

PyTorch implementation and pre-trained weights for the paper PolyLayout: Multi-room Manhattan Layout Estimation (ECCV 2026).

PolyLayout is a multi-view multi-room layout estimation method. Given posed perspective images it jointly estimates the layout of multiple rooms.

Project page: https://ghanning.github.io/PolyLayout/

Code base

PolyLayout is built upon the excellent PixLoc code base. The PixLoc master branch is available in this repository under the name pixloc.

Installation

Install PolyLayout in editable mode as follows:

git clone https://github.com/ghanning/PolyLayout.git
cd PolyLayout/
virtualenv venv
source venv/bin/activate
pip install -e .

Running the demo notebook requires some extra dependencies that can be installed with:

pip install -e .[extra]

Datasets

Download the Aria Synthetic Environments, ScanNet++ and 2D-3D-Semantics datasets from their respective web sites and unpack into a subdirectory named "datasets". The expected directory structure is shown below.

.
└── datasets
    ├── 2d3ds
    │   ├── area_1
    │   ├── area_2
    │   ├── area_3
    │   ├── area_4
    │   ├── area_5a
    │   ├── area_5b
    │   └── area_6
    ├── ase
    │   ├── 10698
    │   ├── 11336
    │   ├── 12462
    .   .
    └── scannetpp
        ├── data
        ├── metadata
        └── splits

Note: Only ScanNet++ is used to train PolyLayout, but we provide code to run the room layout estimation also on Aria Synthetic Environments and 2D-3D-Semantics.

Image undistortion

The PolyLayout code currently supports only pinhole cameras. Follow the steps below to undistort the images in each dataset.

Aria Synthetic Environments

Undistort the fisheye images as explained here.

ScanNet++

~Use the ScanNet++ Toolbox to undistort the DSLR fisheye images by following the instructions here.~

Note: As of April 30, 2025, undistorted DSLR images are included in the ScanNet++ dataset and this step can thus be skipped.

2D-3D-Semantics

Split the panorama images into perspective views as detailed here.

Preprocessing for training on ScanNet++

Depth maps

Render depth maps for the undistorted DSLR images using the render-undistorted branch in my fork of the ScanNet++ Toolbox as described here, but set render_undistorted to True.

2D-3D correspondences

Run our preprocessing script to find the 2D-3D point correspondences used in training:

python -m pixloc.pixlib.preprocess_scannetpp

Line segments (optional)

While line segments are not required to train PolyLayout they improve its performance at inference time. To extract line segments with DeepLSD first install it with

pip install -e .[deeplsd]

then download the pre-trained weights

mkdir weights
wget https://cvg-data.inf.ethz.ch/DeepLSD/deeplsd_md.tar -O weights/deeplsd_md.tar

and run the extraction for Aria Synthetic Environments, ScanNet++ and 2D-3D-Semantics:

./scripts/line_segments_ase.sh
./scripts/line_segments_scannetpp.sh
./scripts/line_segments_2d3ds.sh

Alternatively, you can download the extracted line segments for ASE (27 MiB), ScanNet++ (777 MiB) and 2D-3D-Semantics (8 MiB) and unpack them with the command

unzip line_segments_ase.zip -d datasets/ase
unzip line_segments_scannetpp.zip -d datasets/scannetpp
unzip line_segments_2d3ds.zip -d datasets/2d3ds

Training

Training is done in three stages. First the edge detector is pre-trained by running:

python -m pixloc.pixlib.train --conf pixloc/pixlib/configs/pretrain_polylayout_scannetpp.yaml polylayout_scannetpp_pretrain

Next the full network is trained, with weights initialized from the previous stage:

python -m pixloc.pixlib.train --conf pixloc/pixlib/configs/train_polylayout_scannetpp.yaml polylayout_scannetpp train.load_experiment=polylayout_scannetpp_pretrain

Finally we fine tune the model with a lower learning rate:

python -m pixloc.pixlib.train --conf pixloc/pixlib/configs/train_polylayout_scannetpp.yaml polylayout_scannetpp_fine_tuned train.load_experiment=polylayout_scannetpp train.lr=0.66e-07 train.epochs=5

Tip: Pass the --wandb_project <PROJECT> argument to the training script to log the results to Weights & Biases.

Evaluation

We supply a script to run PolyLayout and output the room layout predictions to a JSON file.

Aria Synthetic Environments

python -m pixloc.run_PolyLayout --experiment polylayout_scannetpp_fine_tuned --conf pixloc/pixlib/configs/eval_polylayout_ase.yaml --split {val,test} --output OUTPUT

ScanNet++

python -m pixloc.run_PolyLayout --experiment polylayout_scannetpp_fine_tuned --conf pixloc/pixlib/configs/eval_polylayout_scannetpp.yaml --split multi_room --output OUTPUT

2D-3D-Semantics

python -m pixloc.run_PolyLayout --experiment polylayout_scannetpp_fine_tuned --conf pixloc/pixlib/configs/eval_polylayout_2d3ds.yaml --split test --output OUTPUT

The resulting predictions can be evaluated using the code in the MultiViewRoomLayout repository.

Pre-trained weights

Pre-trained weights for a model trained on ScanNet++ as outlined above can be found here (298 MiB). Extract the checkpoint with

mkdir -p outputs/training && unzip polylayout_scannetpp_fine_tuned.zip -d outputs/training

Demo

Try out PolyLayout on Aria Synthetic Environments and ScanNet++ with the Jupyter notebook demo_PolyLayout.ipynb.

BibTeX citation

Use the BibTeX reference below to cite our work.

@inproceedings{hanning2026polylayout,
  title={{PolyLayout: Multi-room Manhattan Layout Estimation}},
  author={Hanning, Gustav and Liu, Shaohui and Pautrat, Rémi and Pollefeys, Marc and Åström, Kalle and Larsson, Viktor},
  booktitle={European Conference on Computer Vision},
  year={2026},
}

In addition, please consider citing the PixLoc paper:

@inproceedings{sarlin21pixloc,
  title={{Back to the Feature: Learning Robust Camera Localization from Pixels to Pose}},
  author={Paul-Edouard Sarlin and Ajaykumar Unagar and Måns Larsson and Hugo Germain and Carl Toft and Viktor Larsson and Marc Pollefeys and Vincent Lepetit and Lars Hammarstrand and Fredrik Kahl and Torsten Sattler},
  booktitle={CVPR},
  year={2021},
}