Skip to content
All categories

ai

1274 posts

Origins, Core Concepts, and Suggested Methods | Stanford CS224U Topic Guide

The Spring 2023 edition of CS224U defines in-context learning as a frozen language model performing a task only by conditioning on the prompt, and warns that the second condition of few-shot learning (no examples of the behavior seen in training) is almost impossible to verify. Potts's 38-page deck runs from GPT-2's TL;DR trick through choosing demonstrations, chain of thought, self-consistency, and DSP, and ends with four recommendations: build dev/test sets first, learn your target model's instruction format, and treat prompt writing as AI system design. Mina Lee's guest lecture asks the reverse question: who should learn to read prompts, people or models?

From Classical IR and IR Metrics to Neural IR | Stanford CS224U Topic Guide

The Spring 2023 edition of CS224U spends a whole unit on retrieval, because OpenQA gives you only the question and you have to find the evidence yourself, while large language models fabricate sources. The slides by Potts and Omar Khattab go from TF-IDF and BM25 through Success@K, MRR, and average precision to four neural IR designs (cross-encoder, DPR, ColBERT, SPLADE) and how each trades expressiveness against scale, and they end by asking you to count latency and cost as metrics too.

One Question Asked for Forty Years, and How a 2023 NLU Course Defines Understanding | Stanford CS224U Topic Guide

The first CS224U lecture of Spring 2023 asks "Which U.S. states border no U.S. states?" of every system from Chat-80 (1980) to text-davinci-001. The answers show that the progress is real. The lecture then questions whether that progress counts as understanding, using Levesque's "cheap tricks," models that invent links, and benchmarks that saturate within a year or two. That splits the course map in two: the first half teaches you to build systems with Transformers and retrieval-augmented in-context learning, and the second half teaches you to test them with harder benchmarks, behavioral evaluation, and causal explanation methods.

Lit Review and Experiment Protocol | Stanford CS224U Topic Guide

The first two deliverables of the CS224U final project are a literature review and an experiment protocol. The lit review covers 5, 7, or 9 papers depending on team size, under five suggested sections. The protocol has seven required sections, and its core is a hypothesis you can state. The course supplies a six-step paper-search loop, a rule that AI-assistant output must be quoted, and a worked example: a student's final project that became a Findings of EMNLP paper. The Gradescope format and rubric slides, and past exemplary papers, are behind a login.

A Classifier with 0.81 Accuracy and 0.43 Macro F1 — What Classifier and Generation Metrics Each Encode | Stanford CS224U Topic Guide

The CS224U slides compute two numbers from one three-class confusion matrix: accuracy 0.81 and macro F1 0.43. One says the system is good; the other says it gets the two small classes almost entirely wrong. The unit's claim is that different metrics encode different values, and it goes through the bounds, values, and weaknesses of accuracy, the three F-score averages, perplexity, word error rate, and BLEU. Final projects are graded on whether the metrics fit, not on how high the scores are.

Writing NLP Papers, Submitting, and Giving Talks | Stanford CS224U Topic Guide

CS224U's 'Presenting your research' lecture has four parts: the course-specific rules for the final paper, how to write an NLP paper, how conference submission works, and how to give a talk. Three things matter most. The final paper must include Known project limitations and an Authorship statement. Write as a Shieber-style 'rational reconstruction,' not a chronological tour of your dead ends. At submission, your title largely decides reviewer bidding. The slides, four videos, and projects.md are public; past example papers need a Stanford login.

aideep-dive

Learn Inference: Inference Engineering, Rebuilt with Dials You Can Turn

learn-inference.com is an unofficial interactive companion to Philip Kiely's Inference Engineering (256 pages, Baseten Books, free PDF). It follows the book's 8 chapters and 42 sections with rewritten explanations, turns intuition-heavy ideas like TTFT, P99, speculative decoding, and prefix-cache routing into slider-driven simulators, and ships a keyless JSON API and MCP server.

aideep-dive

Obscura: A Headless Browser for AI Agents, Rewritten in Rust

Obscura is a headless browser engine written from scratch in Rust. It embeds V8 and speaks the Chrome DevTools Protocol, so Puppeteer and Playwright scripts connect by changing one endpoint, and it ships an MCP server with 37 tools. The headline numbers (about 30 MB per process, about 85 ms per page load) come from the project's own benchmarks; it passes 83.3% of its core WPT subtests, and long-tail CSS and some Web APIs still differ from Chromium.

aideep-dive

Making the Invisible Visible: Component Design Philosophy for Agent Chat UI

Traditional chat only needs text bubbles, but AI Agent conversations must surface thinking, tool calls, citations, and progress — we solved this with 12 Vue components, three DisplayModes, and a unified chatBlocks rendering pipeline.

Where Agent Memory Is Heading in 2026: Files Beat Vectors, Forgetting Just Started

In H1 2026, OpenAI, Anthropic, Letta, and LangChain independently chose Markdown files + indexes over vector databases; write permissions shifted back to humans; forgetting mechanisms appeared but nobody implemented Ebbinghaus; Penfield Labs caught 6.4% wrong answers in LoCoMo; three vendors simultaneously adopted 'Dreaming' for offline memory consolidation. Five trends, one conclusion: memory is not a feature — it is an architecture decision.

The Attack Surface of Agent Memory: When Memories Become Persistent Backdoors

Memory turns prompt injection from a one-shot nuisance into a persistent backdoor: MINJA shows conversation-only injection succeeds >95% of the time, and SpAIware demonstrated continuous data exfiltration via planted memories. The industry's two defensive lines — citation-based verification (Copilot) and human approval inboxes (Gemini CLI / Devin) — each have blind spots.

Series Guide: AI Agent Memory Engineering

Agent memory is not one feature — it is at least four distinct engineering problems: working, episodic, semantic, and procedural. This ten-part series walks through the full design space, from taxonomy to coding agent implementations, platform APIs, open-source frameworks, security attack surfaces, and 2026 trend analysis.

Four Types of Memory and Six Design Axes: The Design Space of Agent Memory Systems

CoALA splits agent memory into working, episodic, semantic, and procedural — but the four-cell taxonomy alone doesn't explain why Claude Code uses Markdown files while Mem0 uses vectors. This post adds six independent design axes (read mode, write timing, fidelity, write authority, forgetting, scope) and a file-to-graph spectrum to map the full design space of agent memory systems in 2026.

Test-Time Scaling: BrowseConf and Confidence-Guided Reasoning

The previous articles covered evaluation. This one covers another dimension: how to dynamically allocate compute during reasoning. BrowseConf's core insight is that an agent's self-declared 'confidence' can predict answer accuracy. High confidence uses fewer resources; low confidence searches more rounds.

Five Clouds, Five Memory APIs: How OpenAI, Anthropic, Google, AWS, and Microsoft Let Agents Remember

All five major cloud platforms shipped agent memory APIs in 2025–2026, but their design philosophies diverge sharply: OpenAI writes memory as files, Anthropic mounts memory as a directory, Google uses vectors with topic classification, AWS combines events with pluggable strategy pipelines, and Microsoft abstracts memory behind context providers. Pricing ranges from free to $0.75/1K records/month; tenant isolation spans from 'your app handles it' to IAM as a first-class citizen.

How Nine Coding Agents Handle Long-Term Memory: From CLAUDE.md to MemFS

Nine coding agents have taken at least four different paths for long-term memory: Claude Code and Codex use Markdown files (agent-written, human-readable), Antigravity CLI inherits Gemini CLI's approval inbox (agent proposes, human decides), and Copilot uses citations with JIT verification (auto-deleted after 28 days unverified). Cursor removed its Memories feature and fell back to human-written Rules. Hermes Agent, OpenClaw, and Letta Code treat memory as a core harness component, not a plugin. Their choices on write timing, forgetting, and cross-team sharing are completely different — and none has published a controlled experiment on whether their memory system actually helps.

Commercial Landscape: OpenAI, Perplexity, Gemini, Claude, Grok

By 2026, the deep research commercial market has differentiated: OpenAI is comprehensive, Perplexity is fast, Gemini integrates ecosystems, Claude reasons deeply, Grok is real-time. This article compares each product's differences—not who is best, but who fits your scenario.

Deep Research Landscape: Taxonomy of 80+ Implementations, Roadmap, and Trade-offs

The entire Deep Research field has 80+ implementations, but the core structure is just three-stage roadmap × four components × three optimization methods. This article maps the full landscape: from Agentic Search to Full-stack AI Scientist, from query planning to answer generation, from workflow prompting to end-to-end RL.

Benchmark Deep Dive: DeepResearch Bench II and the Evaluation Landscape

DeepResearch Bench II uses 9,430 expert rubrics covering 132 tasks, and finds that even the strongest agents satisfy less than 50% of criteria. This article breaks down the benchmark architecture, scoring methodology, leaders, and the overall evaluation landscape.

aidebug

One Missing 'Write a Script': How Skill Instructions Determine LLM Success or Failure

An AI assistant platform running Opus 4.6 hit stream_stall (90s timeout) twice consecutively when generating docx. Root cause: skill instructions lacked one sentence — 'Write a script' — causing the model to output JS code inline instead of writing a file and executing with node. Claude.ai's official SKILL.md has that sentence, and the model consistently takes the safe path.