Papers

Filtered to distillation · clear filter

Browse by term

continual learning 86reinforcement learning 48benchmarking 13large language models 12benchmarks 11vision-language models 10language models 9robotics 7world models 7natural language processing 6recursive self-improvement 6generative models 5on-policy distillation 5video generation 5attention mechanisms 4coding agents 4LLMs 4multi-agent systems 4multimodal learning 4multimodal models 4self-distillation 4self-supervised learning 4transformers 4vision-language-action models 4world modeling 4agent-based systems 3agentic models 3agentic search 3agents 3autonomous systems 3

Matching papers

Self-Supervised Visual On-Policy Distillation

158 upvotes · 14 AUG 2026 · Yijiang Li, Yijun Liang, Yunjie Tian et al.

This paper proposes a new method called Self-Supervised Visual On-Policy Distillation (S^2VOPD) that generates learning signals from asymmetric augmented views of images, allowing for effective on-policy learning without privileged information. Practitioners might care about this paper because it presents a simple yet effective way to improve performance on various perception benchmarks.

visual augmentationon-policy learningself-supervised learningdistillationreinforcement learning

Eliciting Weak-to-Strong Generalization with On-Policy Reverse Distillation

78 upvotes · 8 SEP 2026 · Youngrok Park, Sangmin Bae, Hojung Jung et al.

This paper introduces a method called On-Policy Reverse Distillation that helps stronger models learn from weaker supervisors by selectively amplifying the parts of the weaker model's guidance that are most useful to the stronger model. This can lead to faster and more efficient learning, especially in situations where it's expensive to retrain models from scratch.

reinforcement learningpolicy optimizationdistillationtransfer learningmodel selection