🌏 中文版
Today's Focus
Saturday's rotation is Technical PM. The trap in this category is that the interviewer says "design a system," which sounds like a test of engineering chops — but what they're actually checking is whether you can turn a vague ask into a scoped problem, then use system attributes (speed, accuracy, privacy) to make the trade-offs explicit, instead of rushing to draw an impressive-looking architecture diagram. The extra wrinkle in 2026: AI questions aren't a separate category anymore. They're folded into ordinary system-design prompts — when someone asks you to design a system for Gemini to answer a query, you need both the classic system-design vocabulary (caching, latency, scaling) and the AI-specific trade-offs (RAG, context windows, hallucination).
Core Framework
The five-step system-design sequence (from Aced/Exponent's 2026 Technical PM guide): most candidates lose points on sequence, not on knowledge — jumping straight from a one-line prompt to an architecture diagram usually produces a system that looks plausible but answers the wrong question.
| Step | What to do | Easy to skip |
|---|---|---|
| 1. Clarify and scope | Ask who the users are, what they're doing, and constraints like scale, latency, and data sensitivity — then summarize it back before drawing anything | Skipping straight to assumptions about the prompt |
| 2. Derive requirements from the user journey | Walk every step the user takes and map each to a system requirement; if a component doesn't trace to one, it probably doesn't belong | Adding components that look thorough but serve no requirement |
| 3. Name the system attributes | State out loud which of speed, accuracy, reliability, or privacy this system optimizes for — the same requirements with different attributes produce completely different designs | Thinking it but never saying it; the interviewer can't score what they don't hear |
| 4. Draw the architecture | Components, what each does, how data flows — stay at boxes-and-arrows altitude, don't drop into implementation detail | Over-drawing and burning time on a corner that doesn't matter |
| 5. Volunteer the trade-offs | Don't wait to be asked — say "if scale doubled, here's what I'd worry about and what I'd change" | Waiting to be asked makes it look reactive instead of judgment |
This is the same discipline as last week's Technical PM drill — "RFC → decision matrix → ADR" — applied to a different setting: that framework resolves disagreement inside an engineering team, while this one designs a system from scratch in an interview room. Both come down to the same move: state your criteria before you state your conclusion, so the conclusion can hold up.
Today's Practice Question
The Question
"Design a high-level system for Gemini to respond to a user query."
(Source: Aced/Exponent's Technical PM question bank, listed as a system-design question that has come up in Google interviews)
Simplified; based on Aced/Exponent's suggested answer for "design a system for Gemini to respond to a user query." Logging/evals and retry paths omitted.
How to Break It Down
- Clarify the question: Ask the interviewer what kind of query is typical — does it usually need fresh or private data, or does the model's own knowledge usually suffice? What's the latency tolerance? The fastest way to get this wrong is to skip confirming whether retrieval even applies and just draw one fixed pipeline for every case.
- Define the user journey: The user types a query → waits for a response → reads the answer → (possibly) follows up or flags the answer as wrong. Every step should map to a component later in the architecture; a component that doesn't map to a step should come out.
- Name the system attributes: For this question, the attribute ranking is usually "factual correctness first, then latency, then cost" — a confidently wrong answer does more damage than a slightly slow one, and that ranking is what decides whether you add retrieval or add citations.
- Draw the architecture: The query passes through a safety-and-policy gateway first. If the answer depends on fresh or private information, it runs through RAG retrieval to pull in relevant context. The retrieved content (or the raw query) goes into the model along with the prompt. The output passes through guardrails and post-processing — citations, content filtering — and streams back to the user token by token so the response feels fast. Logging and offline evals wrap around the whole pipeline to track quality and catch regressions.
- Volunteer the trade-offs: Name them yourself — retrieval adds latency and cost but cuts hallucination significantly; streaming only improves perceived speed, not total time; and if scale went up another order of magnitude, the first thing you'd worry about is retrieval throughput, not the inference call itself.
Sample Answer (how to actually say this in the interview)
Scope it first: "Before I design anything, I want to confirm a couple of things — are these mostly general-knowledge queries, or do a meaningful share need the user's own data or something that happened today? If it's the latter, I'd treat 'does this query need retrieval, and when' as the first fork in the design, not something I bolt on afterward."
Name the attribute, then draw: "Assuming a meaningful share of queries need fresh or private information, I'd prioritize 'correct over fast.' Architecturally: the query hits a gateway that runs safety and policy checks and blocks anything clearly abusive or high-risk. Depending on the content, it may run through a retrieval step that pulls relevant documents or data into context. The retrieved content and the original query go into the model together. The output passes through post-processing — citations, content filtering — and streams back token by token, so the user sees something happening immediately instead of waiting in silence."
Close on trade-offs, unprompted: "Three trade-offs I'd flag myself: retrieval adds latency and cost, but it buys traceable, citable answers and a lower hallucination rate — worth it under a correctness-first priority. Streaming only improves how fast the response feels, not the system's actual total time, so it's not a substitute for real performance work. And if traffic grew tenfold, the first thing I'd worry about is retrieval throughput, since it depends on an external data source and will bottleneck before inference does."
Self-Check
Use this table to check whether your answer covered the key points:
| Checkpoint | Covered? |
|---|---|
| Asked about query type and latency tolerance before designing | |
| Explicitly named the priority system attribute (correctness / latency / cost) | |
| Every component in the architecture maps to a step in the user journey | |
| Clearly explained how the "retrieve or not" fork gets decided | |
| Volunteered trade-offs instead of waiting to be asked | |
| Bonus: named what would worry you first at 10x scale |
Today's Case
Discord: migrating the messages database from Cassandra to ScyllaDB
Discord engineer Bo Ingram documented this migration in detail on the company's engineering blog in 2023. In 2017, Discord ran 12 Cassandra nodes storing billions of messages. By early 2022, the same database had grown to 177 nodes holding trillions of messages — and had become a high-toil system: on-call was paged frequently for latency spikes, compaction backlogs caused cascading latency, and the team had a routine nicknamed the "gossip dance" — pulling a node out of rotation to let it compact, bringing it back to take traffic, and repeating until the backlog cleared. What actually triggered the migration wasn't a newer technology looking more appealing — it was operational pain accumulating past a threshold the team could quantify, which is what made the risk of migrating trillions of messages worth taking on.
The migration itself is also a demonstration of stating trade-offs before acting: the team's first plan used ScyllaDB's Spark migrator on the old data, with an estimated completion time of three months. Judging that too slow, the team rewrote the data migrator in Rust instead, dropping the estimate to nine days — and the actual migration finished in just over nine days with zero downtime. After the cutover, the cluster shrank from 177 Cassandra nodes to 72 ScyllaDB nodes; p99 latency for fetching historical messages dropped from Cassandra's 40–125ms range to a steady 15ms on ScyllaDB, and p99 write latency dropped from 5–70ms to a steady 5ms.
Interview angle: this case fits any Technical PM question about "when do you justify ripping out a system that's held up for five years" — the point isn't which database won, but how the team turned operational pain into numbers they could take to engineering leadership to justify the migration, and how they treated the migration plan itself like a product: estimate the timeline, and when it's too slow, change the approach instead of grinding through the original three-month plan.
Further Reading
- 30+ Technical Product Manager Interview Questions (2026 Guide) — source for today's question and the five-step system-design framework, covering AI/ML, system design, technical fundamentals, and technical product sense
- How Discord Stores Trillions of Messages with High Performance — ByteByteGo's system-design-interview retelling of this migration, a good pairing with today's question
- How Discord Moved Trillions of Messages to ScyllaDB — Hello Interview's write-up of this case as a recurring "real-world example" in system-design interviews
References
- 30+ Technical Product Manager Interview Questions (2026 Guide) — source for today's question, the five-step framework, and the suggested answer for the Gemini query system
- How Discord Stores Trillions of Messages — Discord's own engineering blog, the primary source for today's node counts, latency numbers, and migration timeline