Table of Contents
- Agenda: language, compiler, and two evaluations
- answer and summary define the boundary
- The compiler makes the language practical
- The gap left by Lecture 6
- Semantics of summary and answer
- Parsing a new target language
- Three compiler optimizations
- Optimization must preserve semantics
- Restaurant and HybridQA evaluation
- Restaurant and HybridQA test different worlds
- Build a minimal prototype
- Evaluation spans two worlds
- A practical reading exercise
- Material gaps
- References
🌏 中文版
This guide reconstructs the lecture from the official Fall 2025 deck; system descriptions and reported results below are attributed to that historical course material unless a paper is linked at the claim.
Lecture 7 continues the hybrid-data problem. SQL handles types, filters, ordering, and aggregation; text retrieval extracts meaning from reviews and descriptions. Answering separately and asking an LLM to merge the results loses optimization and traceability. SUQL instead gives both operations one high-level language.
Agenda: language, compiler, and two evaluations
The deck covers SUQL's rationale and free-text support, its semantic parser and compiler, evaluation in a Yelp-style restaurant application, and HybridQA questions that hop between tables and passages. (lecture source)
answer and summary define the boundary
SUQL retains SQL structure and lets a query call answer(text, question) or summary(text) over textual columns. “Find Japanese restaurants in Palo Alto whose reviews describe them as good for a date, then summarize the reviews” retains structured constraints and textual judgment in one query. (lecture source)
That creates a contract absent from direct answering. Parser output is syntactically inspectable; database fields and free-text questions remain explicit. A failure can be localized to parsing, row selection, or the answer function.
The compiler makes the language practical
Author extension: The following is an implementation or review method derived from the lecture, not a result reported by the deck.
Naive execution would call an LLM for every row. The optimizing compiler first applies cheap SQL predicates, sends only necessary top results to textual functions, and evaluates lazily. Unused values are never computed. Temporary tables preserve intermediate results while an ordinary SQL engine performs most work.
The gap left by Lecture 6
Routing can choose SQL for fixed fields and IR for text, but one question can require both. Flattening SQL rows into prose hides optimization; searching text first loses exact filters and ordering. SUQL treats this as language design: one program contains relational operators and free-text functions. (lecture source)
The parser expresses a plan and the compiler schedules it. This gives up arbitrary tool exploration in exchange for replay, inspection, and optimization.
Semantics of summary and answer
summary(text_column) condenses text over selected rows. answer(text_column, question) extracts a question-specific value that can appear in projection, filtering, ordering, and comparison. HybridQA examples extract a location or birth date from passages and then apply relational operations. (lecture source)
Output types matter. Dates and numbers need explicit casts and unknown handling. Placing model output inside SQL does not make that output deterministic.
Parsing a new target language
The parser must decide which conditions belong in SQL, which require reading text, and how they depend on one another. Sending everything to answer loses precision and optimization; translating semantics into LIKE loses paraphrases. (lecture source)
Evaluate operator placement, columns, free-text questions, casts, and aggregation in addition to syntax. Restaurant errors show that information may live in reviews rather than an intuitively named field, making schema documentation and no-result diagnosis essential.
Three compiler optimizations
Predicate pushdown applies city, price, and rating filters before expensive model calls. Projection and top-k send only needed rows and columns. Lazy evaluation invokes a text function only when downstream conditions or output require its value. (lecture source)
An ordinary SQL engine can execute rewritten fragments while temporary tables materialize model-derived results. This reduces cost without requiring the database itself to understand an LLM.
Optimization must preserve semantics
Moving a limit before a summary can change its coverage; limiting before ordering by an extracted value can remove the true maximum. The compiler needs dependencies rather than a universal “SQL first” rule. (lecture source)
Caching requires text, model, and prompt versions. Evaluate quality, calls, tokens, latency, and processed rows together. Fewer calls are not a win if evidence was discarded.
Restaurant and HybridQA evaluation
The restaurant application measures parsing, result precision, end-to-end responses, and false empty results on a fixed real schema. Fair baselines need the same data and model, with disclosure of information lost by flattened records. (lecture source)
HybridQA tests table-to-passage, passage-to-table, and multi-hop composition across per-question schemas. Separate language coverage, gold-program execution, and predicted-program parsing. A failure of predicted SUQL is not evidence that the language itself cannot express the question.
Restaurant and HybridQA test different worlds
The restaurant setting stresses executable user constraints, empty results, and textual fields in a live schema. HybridQA stresses multi-hop composition between tables and associated text. Reporting them separately prevents a compiler improvement on one workload from becoming a claim of universal hybrid-query competence. (lecture source)
Build a minimal prototype
Author extension: The following is an implementation or review method derived from the lecture, not a result reported by the deck.
Start with SQLite and deterministic mock answer and summary functions. Hand-write hybrid programs to test filtering, dependencies, casts, and ordering without model noise. Add a parser and compare execution to gold programs; then add a real text model and evaluate function accuracy separately.
Author extension: Finally add compiler rewrites and record rows before and after pushdown, function calls, latency, and result equivalence. Production also needs query sandboxing, PII-column policy, timeouts, budgets, and provenance at the compiler boundary.
Evaluation spans two worlds
The restaurant application tests parsing, result precision, and end-to-end responses on a real schema, including false empty results caused by parser errors. HybridQA includes table-to-passage, passage-to-table, and multi-hop compositions, where extracted text answers become values for filtering, ordering, or comparison. (lecture source)
Language design is not judged by final-answer accuracy alone. Can the query be expressed clearly? Does the compiler avoid unnecessary model calls? Can failures be located? These are the central tradeoffs against letting an agent improvise its next tool.
A practical reading exercise
Author extension: The following is an implementation or review method derived from the lecture, not a result reported by the deck.
Use a product table with specification and review columns. Write one SQL-only, one text-only, and one hybrid question. Hand-write SUQL, estimate naive text-function calls, then recalculate after SQL pre-filtering and lazy evaluation.
Material gaps
Author extension: The following is an implementation or review method derived from the lecture, not a result reported by the deck.
The slides provide syntax examples and compiler strategies, not a complete grammar or stable API contract. Their experimental summaries do not replace the paper and code. Unpublished classroom quiz answers are not reconstructed here.
References
Loading...