Firehose

This paper introduces Visual Contrastive Self-Distillation, a method that removes the need for external teacher information and privileged answers in on-policy self-distillation, allowing for simpler and more efficient learning. Practitioners might care about this approach because it can lead to better performance in language models.

This paper introduces H^2SD, a hybrid hindsight self-distillation framework for reinforcement learning with verifiable rewards, which combines the strengths of different methods to improve large language models' reasoning capabilities. Practitioners may care about this work because it addresses limitations of existing methods and shows promising results on challenging reasoning benchmarks.