Filtered to coding agents · clear filter
This paper introduces LoopArena, a benchmark for evaluating how well a model can guide a coding agent through a long-running task, and finds room for improvement in long-horizon loop control. Practitioners working on AI development might care because it could lead to more efficient and reliable development processes.
This paper investigates how different components of coding harnesses, such as planning, action space, and context management, impact the performance of autonomous coding agents in software engineering tasks. Practitioners might care about understanding how to design harnesses that effectively utilize these components to improve agent performance.
This paper proposes a new pruning method for coding agents that prunes tool outputs directly inside the agent, rather than relying on a separate code classifier, and shows it can save up to 39% of tokens while preserving task quality.
This paper introduces SWE-bench Science, a benchmark for evaluating coding agents' performance in repairing scientific software, and identifies common failure mechanisms that hinder their success. Practitioners may care about this research as it can help improve the reliability and reproducibility of scientific findings by developing more effective coding agents.