Skip to content
All tags

#red-teaming

3 posts

Agent Safety — Four Threat Models and Three Limits: Sandboxes, Credential Brokering, Monitoring | CMU 11-768 Lecture 13

Lecture 13 is about limiting what agents can access and do. The slides open with five real incidents, sort them into four threat models (malicious user, prompt injection, agent error, improper means), then give four layers of defense: red teaming and safety training, sandboxes and credential brokering, monitoring, and safety evaluation. The easiest point to miss: hiding the API key does not stop an agent from misusing the access behind it.

Reproduce and Extend One Headline Figure | Harvard CS2881R Topic Guide

The CS2881R midterm isn't an exam. Teams of 2–4 pick one of four AI safety papers, redo its central figure or table, add one or two extensions, and hand in a 3–5 page report plus a GitHub repo. The spec slides and rubric are public, so an outside reader can do the whole thing. The rubric puts most points on reproduction and extensions, and reserves one point for reflecting on how fragile the result is. That one point is the research habit the assignment is really training.

aideep-dive

Promptfoo Deep Dive: Local-First LLM Evaluation and Red Teaming

Promptfoo combines prompts, providers, test cases, and assertions in YAML to produce repeatable local and CI evaluation matrices, with red teaming against the same targets. It lowers the testing barrier but does not remove output variance, LLM-judge bias, or hosted data-flow concerns.