Skip to content
All tags

#promptfoo

3 posts
ai deep-dive

Promptfoo Deep Dive: Local-First LLM Evaluation and Red Teaming

Promptfoo combines prompts, providers, test cases, and assertions in YAML to produce repeatable local and CI evaluation matrices, with red teaming against the same targets. It lowers the testing barrier but does not remove output variance, LLM-judge bias, or hosted data-flow concerns.

#promptfoo#llm-evaluation#red-teaming#ai-security#ci-cd#llm
tech deep-dive Technology Choices in the AI Era

Promptfoo Red Team: Turning Prompt Injection, Tool Misuse, and Data Leaks into Regression Tests

Promptfoo plugins generate risk probes, strategies transform attacks, targets execute the system, and graders judge outcomes; useful red teams exercise the full agent application rather than only a foundation model.

#promptfoo#red-team#ai-security#prompt-injection#evaluation
ai guide The RAG Techniques Compendium

RAG Evaluation Frameworks and Tool Selection: Promptfoo, RAGAS, DeepEval, and TruLens

No industry standard mandates one RAG evaluation tool. Measure retrieval, generation, and operations separately, then choose Promptfoo, RAGAS, DeepEval, or TruLens for the actual stack.

#rag#evaluation#promptfoo#ragas#deepeval#trulens#metrics#quality