PersonalizedRouter

# PersonalizedRouter: Personalized LLM Routing via Graph-based User Preference Modeling

<p align="center">
<a href="https://ulab-uiuc.github.io/PersonalizedRouter/">🌐 Project Page</a> |
<a href="https://arxiv.org/abs/2511.16883">📜 arXiv</a> |
<a href="#">📮 Twitter Post</a>
<p>

📝 Message

[2025.02] 🧩 PersonalizedRouter has been integrated into LLMRouter. You can now experience PersonalizedRouter directly in the unified LLMRouter framework. We welcome everyone to try PersonalizedRouter in LLMRouter!

[2025.12] 🎉 We open-sourced LLMRouter, a unified and extensible framework for training and evaluating single-round / multi-round / agentic / personalized LLM routers. LLMRouter aims to reduce duplicated engineering effort and enable fair comparison across different routing methods. We warmly welcome the community to integrate and benchmark their own routers!

[2025.01] ✅ PersonalizedRouter is accepted for TMLR 2025.

🛠️Preliminary

Environment Setup

# create a new conda environment
conda create -n personalizedrouter python=3.10
conda activate personalizedrouter

# install pytorch. Modify the command to align with your own CUDA version.
pip3 install torch  --index-url https://download.pytorch.org/whl/cu118

# install related libraries
pip install -r requirements.txt

# install pyg
pip install torch_geometric
pip install pyg_lib torch_scatter torch_sparse torch_cluster torch_spline_conv -f https://data.pyg.org/whl/torch-2.1.0+cu118.html

Notation

In the project files, the suffix v1 refers to the Multi-cost-efficiency Simulation Strategy described in the paper, while v2 refers to the LLM-as-a-Judge Strategy, and large denotes the large-scale setting.

Dataset Preparation

First, generate data/unified_qa_data.csv.

python data_processing/multidata_unify.py

Then, generate data/router_data_v1(v2).csv and configs/llm_description_embedding_v1(v2).pkl by setting your api_key in configs/config.yaml.

python data_processing/construct_router_data.py

For the LLM-as-a-Judge setting, generate llm_judge_results.csv

python data_processing/construct_llm_judge.py

In addition, we need to introduce user nodes to generate router_user_data_v1(v2).csv

python data_processing/add_user_data_v1.py

or

python data_processing/add_user_data_v2.py

For your convenience, we have provided the pre-generated files in the data folder, including most files mentioned above.
We also provide a Hugging Face dataset page, where you can directly download the data.

📝Experiments

Training and Evaluation

Run experiments and print/save evaluation results on metrics Performance, Cost, and Reward. You can edit the hyperparameters in configs/config.yaml or using your own config_file.

python run_exp.py --config_file [config]

Citation

@inproceedings{dai2025personalizedrouter,
  title={PersonalizedRouter: Personalized LLM Routing via Graph-based User Preference Modeling},
  author={Zhongjie Dai and Tao Feng and Jiaxuan You},
  journal={Transactions on Machine Learning Research},
  year={2025}
}