Papers

Filtered to data-minimal limit · clear filter

Browse by term

continual learning 86reinforcement learning 48benchmarking 13large language models 12benchmarks 11vision-language models 10language models 9robotics 7world models 7natural language processing 6recursive self-improvement 6generative models 5on-policy distillation 5video generation 5attention mechanisms 4coding agents 4LLMs 4multi-agent systems 4multimodal learning 4multimodal models 4self-distillation 4self-supervised learning 4transformers 4vision-language-action models 4world modeling 4agent-based systems 3agentic models 3agentic search 3agents 3autonomous systems 3

Matching papers

Rethinking On-Policy Distillation of Large Language Models II: One Training Example

81 upvotes · 3 SEP 2026 · Zixuan Fu, Bingxiang He, Yuxin Zuo et al.

This paper investigates the role of training data in on-policy distillation, a technique used to improve large language models, and finds that even a single query can lead to significant improvements, but the process is slow and algorithm-starved.

on-policy distillationlarge language modelssingle-query trainingdata-minimal limitstate coverageteacher-student alignment