AutoSSVH: Exploring Automated Frame Sampling for Efficient Self-Supervised Video Hashing

[toc]

1. Introduction

This repository contains the PyTorch implementation of our work at CVPR 2025:

AutoSSVH: Exploring Automated Frame Sampling for Efficient Self-Supervised Video Hashing. Niu Lian, Jun Li, Jinpeng Wang, Ruisheng Luo, Yaowei Wang, Shu-Tao Xia, Bin Chen.

We present "AutoSSVH", a novel approach in the field of self-supervised video hashing. "AutoSSVH" integrates an adversarial strategy-based sampler with Hash Voting, which accelerates adversarial convergence. Inspired by GANs, our sampler automatically selects frames that are difficult to reconstruct, thereby enhancing the semantic information of the hash codes. Hash Voting, based on ensemble learning, employs a P2Set paradigm to strengthen the neighborhood information of the hash codes. Extensive experiments demonstrate the efficiency and robustness of "AutoSSVH". We hope that our research provides valuable insights for future strategies in adversarial sampling.

Besides, we invite readers to refer to our previous work ConMH and S5VH.

In the following, we will guide you how to use this repository step by step. 🤗🐶

2. Preparation

git clone https://github.com/EliSpectre/CVPR25-AutoSSVH.git
cd CVPR25-AutoSSVH/

2.1 Requirements

• python==3.11.8
• numpy==1.26.4
• pytorch==2.0.1
• torchvision==0.15.2
• scipy==1.5.4
• h5py==3.1.0
• addict==2.4.0
• faiss==1.8.0

2.2 Download the video feature datasets and organize them properly

Before running the code, make sure that everything is ready. The working directory is expected to be organized as below:

CVPR25-AutoSSVH/

• checkpoint/

• activitynet/

• AutoSSVH_16bit

• AutoSSVH_32bit

• AutoSSVH_64bit

fcv/

• AutoSSVH_16bit

• AutoSSVH_32bit

• AutoSSVH_64bit

hmdb/

• AutoSSVH_16bit

• AutoSSVH_32bit

• AutoSSVH_64bit

ucf/

• AutoSSVH_16bit

• AutoSSVH_32bit

• AutoSSVH_64bit

• data/

• activitynet/

• train_feats.h5

• test_feats.h5

• re_label.mat

• query_feats.h5

• q_label.mat

• fcv/

• fcv_train_feats.h5

• fcv_test_feats.h5

• fcv_test_labels.mat

• hmdb/

• hmdb_train_feats.h5

• hmdb_train_labels.mat

• hmdb_test_feats.h5

• hmdb_test_labels.mat

• ucf/

• ucf_train_feats.h5

• ucf_train_labels.mat

• ucf_test_feats.h5

• ucf_test_labels.mat

• configs/

• dataset/

• inference/

• Loss/

• model/

• optim/

• utils/

• preprocess.py

• train.py

• eval.py

• requirements.txt

You can downloaded video features from the following Baidu Cloud links and put them into dataset-specific folder under the data/ folder.

2.3 Train

The training command is as follows:

python train.py --config configs/AutoSSVH/AutoSSVH_.py --gpu 

Options:

• `:act,fcv,ucf,hmdb`
• ``: specify the gpu id

The logs, model checkpoints will be generated under the logs/ and checkpoint/ folders, respectively.

2.5 Test

We provide the evaluation code for model checkpoints (if exist).
The test command is as follows:

python eval.py --configs/AutoSSVH/AutoSSVH_.py --gpu 

3. Results

Dataset
Code Length
MAP@5
MAP@20
MAP@40
MAP@60
MAP@80
MAP@100





ActivityNet
16
0.176
0.091
0.055
0.040
0.031
0.026


32
0.250
0.136
0.082
0.061
0.049
0.038


64
0.290
0.164
0.098
0.071
0.055
0.045


FCVID
16
0.347
0.256
0.225
0.207
0.193
0.180


32
0.483
0.336
0.289
0.263
0.244
0.228


64
0.539
0.381
0.334
0.309
0.290
0.273


UCF101
16
0.423
0.343
0.288
0.244
0.210
0.184


32
0.519
0.448
0.396
0.353
0.309
0.269


64
0.570
0.500
0.452
0.413
0.373
0.329


HMDB51
16
0.161
0.113
0.081
0.064
0.053
0.045


32
0.227
0.163
0.123
0.097
0.081
0.069


64
0.256
0.175
0.137
0.111
0.090
0.076

4. References

If you find our code useful or use the toolkit in your work, please consider citing:

@inproceedings{Lian25_AutoSSVH,
  author={Lian, Niu and Li, Jun and Wang, Jinpeng and Luo, Ruisheng and Wang, Yaowei and Xia, Shu-Tao and Chen, Bin},
  title={AutoSSVH: Exploring Automated Frame Sampling for Efficient Self-Supervised Video Hashing},
  booktitle={Proceedings of the IEEE/CVF conference on computer vision and pattern recognition},
  year={2025}
}

5. Acknowledgements

This code is based on our previous work ConMH at AAAI'23.
We are also grateful for other teams for open-sourcing codes that inspire our work, including
S5VH,
SSVH,
BTH,
MCMSH,
BerVAE,
DKPH,
and SHC-IR.

6. Contact

If you have any question, you can raise an issue or email Jinpeng Wang ([email protected]).