Xiangyu Qi

I am a Member of Technical Staff at OpenAI, where I work to make LLMs robust. Previously, I obtained my Ph.D. from Princeton University, advised by Prof. Prateek Mittal and Prof. Peter Henderson.

Selected Research

Safety Alignment Should Be Made More Than Just a Few Tokens Deep

ICLR, 2025. Outstanding Paper Award (3 Out Of 11,672 Submissions)

Xiangyu Qi, Ashwinee Panda, Kaifeng Lyu, Xiao Ma, Subhrajit Roy, Ahmad Beirami, Prateek Mittal, Peter Henderson
Paper Code

Coverage: Yannic Kilcher’s Paper Explainer Center for Democracy and Technology letter to NIST

Fine-tuning Aligned Language Models Compromises Safety,
Even When Users Do Not Intend To!

ICLR, 2024. Oral Presentation, 1.2%
Covered by The New York Times

Xiangyu Qi $^*$ , Yi Zeng $^*$ , Tinghao Xie $^*$ , Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, Peter Henderson
Paper Policy Brief Code Website

Coverage: The New York Times PCMag The Register VentureBeat

Visual Adversarial Examples Jailbreak Aligned Large Language Models

AAAI, 2024. Oral Presentation, 4.6%
GPT-4V(ision) system card cited this paper to underscore the emerging threat vector of multimodal jailbreaking.

Xiangyu Qi $^*$ , Kaixuan Huang $^*$ , Ashwinee Panda, Peter Henderson, Mengdi Wang, Prateek Mittal
Paper Code

Xiangyu Qi (漆翔宇)

Selected Research