Skip to content
All tags

#swe-bench

2 posts
aiguideReading Harvard CS2680 Modern AI Systems: Agents and System Optimizations

Building a Coding Agent Harness by Hand: Four Tools and a Loop | Harvard CS2680 Assignment 2

CS2680 Assignment 2 (Mads-Loop) has you write your own coding agent harness: with a fixed small model, qwen3.6-35b-a3b, it fixes real SWE-bench Pro issues inside Docker containers and logs every step in a required format. The assignment page says plainly not to trust the design it hands you, and some of the 10 tasks can't be solved. This post covers the spec, the places on the page worth doubting, the five places a harness most often breaks and how to handle each (with a snippet that has actually been run), how to build your own four-task eval set, and how to tell whether a task really is unsolvable. No official solution.

#cs2680#ai-course#harvard#coding-agent#agent-harness#swe-bench#course-guide
aiguideReading Harvard CS2680 Modern AI Systems: Agents and System Optimizations

Saving Money and Time Without Losing Tasks: A Cost-Optimization Competition | Harvard CS2680 Assignment 3

CS2680 Assignment 3 (Mads-Opt) is a harness competition: on the same SWE-bench Pro tasks, who solves the most, spends the least per solved task, and waits the least per solved task. The three metrics aren't weighted into one score. Instead submissions are ranked in skyline (Pareto) layers: if nobody is at least as good as you on all three and better on at least one, you're rank 1. This post covers the container setup and hard limits, how ranking and scoring work (with a worked example), a first estimate of what one task costs using the model prices on the assignment page, and how Lecture 8's cost and speed levers map onto the assignment. No official solution and no specific strategies.

#cs2680#ai-course#harvard#coding-agent#agent-harness#swe-bench#course-guide