Papers

Filtered to on-policy distillation · clear filter

Browse by term

continual learning 86reinforcement learning 48benchmarking 13large language models 12benchmarks 11vision-language models 10language models 9robotics 7world models 7natural language processing 6recursive self-improvement 6generative models 5on-policy distillation 5video generation 5attention mechanisms 4coding agents 4LLMs 4multi-agent systems 4multimodal learning 4multimodal models 4self-distillation 4self-supervised learning 4transformers 4vision-language-action models 4world modeling 4agent-based systems 3agentic models 3agentic search 3agents 3autonomous systems 3

Matching papers

When EOS Tokens Disagree: Understanding Length Inflation in On-Policy Distillation

96 upvotes · 17 SEP 2026 · Yuxiao Yang, Tianrun Yu, Shangzhe Li et al.

This paper investigates why some AI models can generate excessively long responses and proposes a solution to mitigate this issue by aligning the models' termination tokens. Practitioners might care about this problem because it can lead to wasted generation budgets and inefficient AI applications.

on-policy distillationlength inflationEOS tokenstermination-token mismatchcontinual learning

Rethinking On-Policy Distillation of Large Language Models II: One Training Example

81 upvotes · 3 SEP 2026 · Zixuan Fu, Bingxiang He, Yuxin Zuo et al.

This paper investigates the role of training data in on-policy distillation, a technique used to improve large language models, and finds that even a single query can lead to significant improvements, but the process is slow and algorithm-starved.

on-policy distillationlarge language modelssingle-query trainingdata-minimal limitstate coverageteacher-student alignment

Self-OPD: On-Policy Distillation for Flow Matching Models without Teacher

73 upvotes · 27 AUG 2026 · Shiyi Zhang, Mushui Liu, Yunze Tong et al.

This paper introduces a new method for training flow matching models called Self-OPD, which uses the model's own exploration to generate supervisory signals without needing a separate teacher model. Practitioners may care because it could lead to more efficient and effective training of flow matching models.

flow matching modelson-policy distillationcontinual learningreinforcement learningnormalization

Rethinking Classifier-Free Guidance in On-Policy Diffusion Distillation

70 upvotes · 27 JUL 2026 · Bingnan Li, Haozhe Wang, Haozhong Xiong et al.

This paper investigates how to improve the adaptation of diffusion models in a way that doesn't rely on a classifier, and how to address a problem where the model can't accurately learn from its teacher. Practitioners might care about this because it could lead to more effective knowledge transfer in machine learning applications.

on-policy distillationcontinual learningdiffusion modelsguidancevelocity matching

Weak-to-Strong On-Policy Distillation

49 upvotes · 28 JUL 2026 · Fangxu Yu, Zinan Lin, Xiaodong Liu et al.

This paper introduces a new method for transferring capabilities from multiple weak models to a strong student model in reinforcement learning, allowing the student to surpass the domain teacher. Practitioners may care about this approach for improving the performance of their own reinforcement learning models.

on-policy distillationweak-to-strong transferlogit spacereverse KL divergencecontrast pairsreinforcement learning models