Table of Contents
🌏 中文版
DuckDB is an in-process analytical database. It embeds as a library in Python, R, Node.js, or a CLI instead of running a separate server. Columnar, vectorized execution targets scans, aggregations, and joins. Its practical advantage is querying Parquet, CSV, JSON, DataFrames, and object storage without first building a warehouse ingestion pipeline.
Query files before moving data
SELECT region, sum(amount) AS revenue
FROM read_parquet('s3://bucket/orders/*.parquet')
WHERE ordered_at >= DATE '2026-08-01'
GROUP BY region
ORDER BY revenue DESC;Parquet projection and filter pushdown can reduce reads, but performance still depends on file sizes, partitions, networking, compression, and row-group statistics. Measure real files with EXPLAIN ANALYZE; direct querying does not make every layout fast. Materialize a DuckDB table for repeated work when appropriate.
Embedded strength defines the concurrency boundary
In-process operation removes a network hop, service operations, and unnecessary copies. It fits notebooks, CI data checks, local analytics, ETL, feature engineering, and single-machine applications. The documented concurrency model focuses on multiple threads in one process; multiple processes writing one file is not a general primary-server architecture. Web applications with many clients, tiny concurrent writes, HA, and tenant authorization should use MySQL, PostgreSQL, or a managed warehouse.
SQLite is also embedded but emphasizes transactional application storage; DuckDB emphasizes analytical scans. Warehouses such as BigQuery or Snowflake add centralized governance, elastic multi-user compute, and managed operations at greater cost and data movement. DuckDB can complement them as preprocessing, a local cache, or a small-to-medium alternative.
AI workflows can scan evaluation results, token logs, Parquet datasets, and embedding metadata with DuckDB. Restrict untrusted SQL, extensions, files, and network access; embedded does not mean sandboxed.
References
Loading...