This paper proposes a new method for aligning large language models with human preferences, called Comparison-based Preference Optimization (ComPO), which is more efficient than existing methods and can mitigate a problem called likelihood displacement. Practitioners might care about this paper because it offers a new approach to aligning LLMs with human preferences, which is essential for developing more reliable and trustworthy AI models.
Firehose
Filtered to tagged “zeroth-order optimization” · clear filters
Browse: People · Companies · Papers · Podcasts · Hacker News · Deep dives
Browse by tag
artificial intelligence 82continual learning 29AI 26agentic coding 17reinforcement learning 16open-weight models 11AI agents 10AI safety 9AI ethics 8cybersecurity 8machine learning 7open-source 7language models 6natural language processing 6Reinforcement learning 6artificial general intelligence 5deep learning 5Diffusion models 5Agentic AI 4computer vision 4conversational AI 4ethics 4existential risk 4large language models 4LLMs 4Recursive self-improvement 4robotics 4security 4vision-language models 4ai 3