Promptfoo combines prompts, providers, test cases, and assertions in YAML to produce repeatable local and CI evaluation matrices, with red teaming against the same targets. It lowers the testing barrier but does not remove output variance, LLM-judge bias, or hosted data-flow concerns.
Model Armor can inspect prompt injection, jailbreaks, sensitive data, malicious URLs, and unsafe content at runtime; it is a probabilistic detector, not an authorization or sandbox boundary.
Promptfoo plugins generate risk probes, strategies transform attacks, targets execute the system, and graders judge outcomes; useful red teams exercise the full agent application rather than only a foundation model.
Volcengine (ByteDance) open-sources OpenViking, replacing black-box vector search with a viking:// virtual filesystem for agent memory — benchmarks show 80%+ accuracy while saving 34-91% tokens. munder-difflin wraps multiple coding CLIs into a desktop office with shared memory; ai-memory solves cross-CLI amnesia with a Rust MCP server; mukul975's cybersecurity skill pack rockets to ~28K stars in a day. pydantic-ai v2.32.0 adds OpenRouter/xAI attachment search and instrumentation improvements.