A few COGS generalization splits score 0 for nearly every model. CS224U uses its own ReCOGS work to explain why: the zeros on the recursion splits are mostly a length-generalization problem, and the zeros on the prepositional-phrase split come from training data that only ever put PPs in certain variables and positions. Assignment 3, hw_recogs.ipynb, uses 135K ReCOGS training pairs. It first has you find Charlie and Lina, two names whose train and test roles are exact opposites, then shows a trained model stumbling on them.
The lecture parses patient records and trial criteria into SMT, retrieves candidates through a weaker propositional projection, and runs a solver on the reduced set. Reasoning is inspectable, but NL-to-SMT remains the main error boundary.
SUQL adds answer and summary functions over text to SQL. A semantic parser emits one hybrid query, while an optimizing compiler applies predicate pushdown, top-k pruning, and lazy evaluation.
Genie Worksheets declare task capability as a form-like specification. A contextual semantic parser updates formal dialogue state while the runtime controls queries, actions, and responses.