From Failures to Fixes: LLM-Driven Scenario Repair for Self-Evolving Autonomous Driving

Xinyu Xia1, Xingjun Ma2, Yunfeng Hu1, Ting Qu1,
Hong Chen3, Xun Gong1,*
1Jilin University 2Fudan University 3Tongji University
Paper Code arXiv

Abstract

Ensuring robust and generalizable autonomous driving requires not only broad scenario coverage but also efficient repair of failure cases, particularly those related to challenging and safety-critical scenarios. However, existing scenario generation and selection methods often lack adaptivity and semantic relevance, limiting their impact on performance improvement. In this paper, we propose SERA, an LLM-powered framework that enables autonomous driving systems to self-evolve by repairing failure cases through targeted scenario recommendation. By analyzing performance logs, SERA identifies failure patterns and dynamically retrieves semantically aligned scenarios from a structured bank. An LLM-based reflection mechanism further refines these recommendations to maximize relevance and diversity. The selected scenarios are used for few-shot fine-tuning, enabling targeted adaptation with minimal data. Experiments on the benchmark show that SERA consistently improves key metrics across multiple autonomous driving baselines, demonstrating its effectiveness and generalizability under safety-critical conditions.

Method Overview:

LLM-Driven Scenario Repair for Self-Evolving Autonomous Driving

Given the performance logs generated from a rigorous pre-evaluation process, SERA utilizes a large language model (LLM) to analyze and interpret failure patterns. These insights guide a Failure-Aware Scenario Recommendation pipeline that retrieves semantically relevant scenarios from a structured Scenario Bank. The recommended scenarios are further refined by an LLM-based reflection module to ensure semantic alignment and diversity, producing high-quality, efficient scenarios for subsequent Self-Evolving Scenario Repair. This iterative repair process significantly enhances the robustness and safety of autonomous driving systems, particularly under challenging, rare, and safety-critical conditions.

BibTeX

@article{xia2025failures,
  title={From Failures to Fixes: LLM-Driven Scenario Repair for Self-Evolving Autonomous Driving},
  author={Xia, Xinyu and Ma, Xingjun and Hu, Yunfeng and Qu, Ting and Chen, Hong and Gong, Xun},
  journal={arXiv preprint arXiv:2505.22067},
  year={2025}
}