CS2680 Assignment 1 has you build a web page that drives Claude Code: submit a task, watch live what the agent reads, what it runs, and how it turns out, then show how much it cost, how many seconds, and how many turns. The key is reading the event stream from `claude -p --output-format stream-json --verbose`: tool_use and tool_result pair up by the same id, and subagent events hang back under the step that dispatched them via parent_tool_use_id. This post breaks down the seven requirements, shows 40 lines of Python that build the events into a tree and tell success from failure, then covers submission, grading, and the places development most often gets stuck.
CS2680 Assignment 2 (Mads-Loop) has you write your own coding agent harness: with a fixed small model, qwen3.6-35b-a3b, it fixes real SWE-bench Pro issues inside Docker containers and logs every step in a required format. The assignment page says plainly not to trust the design it hands you, and some of the 10 tasks can't be solved. This post covers the spec, the places on the page worth doubting, the five places a harness most often breaks and how to handle each (with a snippet that has actually been run), how to build your own four-task eval set, and how to tell whether a task really is unsolvable. No official solution.
CS2680 Assignment 3 (Mads-Opt) is a harness competition: on the same SWE-bench Pro tasks, who solves the most, spends the least per solved task, and waits the least per solved task. The three metrics aren't weighted into one score. Instead submissions are ranked in skyline (Pareto) layers: if nobody is at least as good as you on all three and better on at least one, you're rank 1. This post covers the container setup and hard limits, how ranking and scoring work (with a worked example), a first estimate of what one task costs using the model prices on the assignment page, and how Lecture 8's cost and speed levers map onto the assignment. No official solution and no specific strategies.
Harvard CS2680 follows one coding agent and takes it apart from the top down. You start by watching Claude Code run, then write your own loop, pick tools and manage context, grow it into multiple agents, add memory, learn to evaluate it, and finally cut its cost and latency. In the second half of the term you go further down and serve the model yourself. This post covers what Part I's eight lectures and three assignments teach, works one example by hand to show why every extra agent step costs more, and ends with a prerequisite check and a reading order.
CS2680 Lecture 1 (2026-09-02) starts with how AI systems differ from traditional systems: computation is split across CPUs and GPUs, floating-point math doesn't give the same answer every time, and state is large enough that it has to move between GPUs. It then compares an agent to a new kind of computer (the LLM is the CPU, the context window is memory, the harness is the operating system) and shows that one agent request becomes a long chain of inference → tool → inference. That makes tokens per second the wrong target; what matters is task success rate, end-to-end latency, and cost per success. This post works through two agents' numbers to show how those metrics are used.
CS2680 Lecture 2 (2026-09-09) takes Claude Code apart. You type one request and it calls the model many times. Each time, the model only proposes a tool to run; the harness checks permissions, runs it, and puts the result back into context, and when the context fills up it gets compacted. The second half covers memory (the CLAUDE.md you write and the auto memory Claude keeps) and four extensions: skills cover how to do things, MCP reaches outside systems, hooks cover what must always happen, and plugins handle packaging. This post first shows what one API call looks like, then walks the whole loop through fixing one test.
The first half of CS2680 Lecture 3 (2026-09-14) is about scaling yourself up. It uses two plugins, Superpowers and Caveman, to show how skills, hooks, and a proxy fit together, then covers three directions to scale: run agents in parallel to buy wall-clock time, delegate to subagents to buy focus, and let one agent run long to buy scope. The second half is about the cost of scaling: when you have an agent write code from natural language, uncertainty seeps in from four layers (meaning, context, model sampling, the tool environment), and the remedies are to define clearly, separate proposal from execution, and tighten verification. This post works through "three bugs in three worktrees" to show what parallelism actually saves.
CS2680 Lecture 4 (2026-09-16) switches from using agents to designing them. An agent is a model plus a loop, and you decide which tools it gets and what it sees at each step. The tools half covers four ways to find code, four ways to edit it, the power and risk of a shell, and a set of design principles: few orthogonal tools, purpose-built over generic, token-cheap output, errors that teach. The context half covers why context is a finite resource, four ways to manage it (from guarding the door to summarizing as a last resort), and five ways context fails. It closes with a paper showing that when context is tight, the same model with only a different harness can solve 70% to more than twice as many tasks.
CS2680 Lecture 5 (2026-09-23) is about designing multi-agent systems. There are four reasons to split work across agents: specialization, isolation, parallelism, and context management. Any architecture can be described by three questions: who coordinates, how the work is split, and how messages move. The lecture lays out eight architectures, then covers six ways to decompose a task, a granularity formula for deciding whether to split, six common decomposition failures, and why communication is really a distributed systems problem: messages get lost or resent, and two agents end up changing the same thing at once. It closes by contrasting A2A with MCP, with "Is multi-agent really necessary?" as the first design principle.
The first half of CS2680 Lecture 6 (2026-09-28) is about agent memory. Keeping one fact means deciding, in order, whether to write it, what form it takes, where it lives, when to retrieve it, whether to generalize from it, and whether it still holds, plus who owns the memory. Don't simply delete outdated facts; record when they were true and where they came from. The second half is about reliability: at 95% success per step, a 20-step run is fully correct only about 36% of the time. Failures can be traced to five layers (task spec, harness, model, tool, environment), each with its own fixes. The two halves meet at one harness-layer failure: treating a stale note as fact.
CS2680 Lecture 7 (2026-09-30) is about measuring whether an agent is any good. First, be clear whether you are measuring a component or the whole task, and whether "success" means a correct answer, working code, or an environment left in the right state. On metrics, pass@k asks whether at least one of k tries succeeds, while pass^k asks whether all k succeed; from the same set of trials the two can come out as 0.9 and 0.3. Then come eight representative benchmarks and what each one measures, and finally when a score can't be trusted: the answer may be in the training data, the agent may look it up online, it may be sitting in the environment's git history, and the grader itself can be gamed.
CS2680 Lecture 8 (2026-10-07) has two halves. The first takes apart an agent's bill: cost is a product of users × sessions per user × turns per session × steps per turn × tokens per step × price per token, and the last four are the ones you can cut. The students' own Claude Code logs show 97.6% of tokens are cached re-reads, so keeping the prefix stable matters more than keeping the prompt short. Speed is set by the critical path, not the sum of every step. The second half compares three harness architectures (monolithic, modular, microkernel) and the design choices of eight open-source and commercial harnesses. This post works one bill by hand: put a timestamp at the start of the system prompt and the same task goes from 23.6 units back to 65 or more.