OMP Internals 12: Rust Native Crates & FFI Contracts

TL;DR

OMP pushes grep, AST matching, shell/PTY, file walking, isolation, and voice into 6 independent Rust crates (pi-ast, pi-natives, pi-walker, pi-shell, pi-iso, pi-voice), then exposes a unified JavaScript interface through pi-natives — a single N-API cdylib.

• Why Rust: grep/AST need native throughput; shell/PTY need correct process control and signal handling; hashline/isolation need deterministic cross-platform behavior.
• N-API contract: Rust #[napi] annotations → bun run build:bindings runs napi-rs to emit index.d.ts + .node → gen-enums.ts rewrites const enum to runtime objects and emits explicit ESM exports → Bazel cross-compiles multi-platform .node files, packaged as optional dependency leaf packages.
• WASM fallback: pi-iso falls back to recursive copy (Rcopy) when no native backend exists; the same Rust code compiles to WASM for browser/edge runtimes.

Context

OMP (oh-my-pi) is the core toolchain for an AI coding agent, providing:

• Text search: ripgrep-class regex search, fuzzy find, glob
• Code structure: tree-sitter / ast-grep syntactic search, edit, summarization
• Terminal ops: embedded shell (brush), PTY sessions, process management
• Filesystem: parallel walking, ignore rules, scan cache
• Isolation: APFS clone / Linux overlayfs / Windows ProjFS / git worktree fallback
• Voice: microphone capture, playback, WebRTC live conversation

These capabilities are invoked from Node.js / Bun main threads, but JavaScript is fundamentally unsuited to implement any of the above. OMP's solution: sink core subsystems into Rust crates, bridge via N-API.

Problem

1. Performance: JS regex engine, sync FS APIs, tree walking are too slow; large-project grep/AST scans must complete in seconds.
2. Correctness: Shell needs precise signal propagation, process group management, PTY master/slave coordination; JS child_process abstraction is too thin and inconsistent.
3. Determinism: hashline (content-addressing), isolation (copy-on-write) need bit-identical cross-platform behavior.
4. Distribution UX: Users npm install and it works — no Rust toolchain, no compilation, support Linux/macOS/Windows x64/arm64, with AVX2/baseline CPU variants.

Attempts

Early: Everything in One Crate

Initially pi-natives contained both N-API bindings and all algorithms. Drawbacks:

• Extreme compile times (single monolithic crate)
• Subsystems couldn't be tested independently (grep, walker, shell entangled)
• Cross-platform cfg scattered everywhere, hard to maintain

Post-Split Architecture

@oh-my-pi/pi-natives (JS entrypoint)
       │
       ▼
crates/pi-natives (N-API cdylib, ONLY binding conversion & platform adaptation)
       │
       ├─► pi-ast         # tree-sitter registry, ast-grep matching/edit/summary
       ├─► pi-walker      # parallel file walking, ignore, cache, glob/fuzzy candidate source
       ├─► pi-shell       # brush shell execution, process plumbing, minimizer, in-process builtins
       ├─► pi-builtins    # builtin commands: grep/rg/fd/ls/find/jq/sed/... (uutils ports)
       ├─► pi-iso         # isolation backends: APFS/overlayfs/ProjFS/Rcopy + diff
       └─► pi-voice       # audio capture/playback, Opus/WebRTC peer

Each crate is an independent rlib exporting only Rust APIs, zero N-API dependency. pi-natives assembles them into the cdylib.

Solution

6 Crates: Responsibilities & Surfaces

Key design: pi-natives contains zero algorithms — only N-API conversion, platform adaptation, cancellation bridging, runtime installation. This lets Rust crates be consumed by non-N-API users (CLI, WASM, tests) directly.

How the N-API Binding Contract Is Written

1. Rust Side: #[napi] Annotations

// crates/pi-natives/src/grep.rs
#[napi]
pub struct GrepOptions {
    pub pattern: String,
    pub path: Option<String>,
    pub ignore_case: Option<bool>,
    pub max_count: Option<u32>,
    pub max_count_per_file: Option<u32>,
    pub context_before: Option<u32>,
    pub context_after: Option<u32>,
    pub timeout_ms: Option<u32>,
    pub signal: Option<AbortSignal>,
}

#[napi]
pub async fn grep(options: GrepOptions, on_match: Option<ThreadsafeFunction<GrepMatch>>) -> GrepResult { ... }

• Naming: Rust snake_case → JS camelCase (automatic)
• Type mapping: Option<T> → T \| undefined, Vec<T> → T[], String → string, u32 → number
• Async: async fn → returns Promise<T> (napi-rs uses Task on libuv workers)
• Streaming callbacks: ThreadsafeFunction<T> → JS callback (match) => void

2. Generate Bindings: bun run build:bindings

# packages/natives/scripts/build-bindings.ts
napi build \
  --manifest-path crates/pi-natives/Cargo.toml \
  --package-json-path packages/natives/package.json \
  --platform --no-js --dts index.d.ts \
  -o <build-output-dir> \
  --profile local

Outputs:

• index.d.ts: TypeScript declarations (with const enum, interfaces, classes)
• .node: Native addon (placed in packages/natives/native/)

3. gen-enums.ts: Patching Runtime Enums & ESM Exports

napi-rs emits const enum — a TypeScript-only construct with no JS runtime value. gen-enums.ts does two things:

1. Rewrite const enum to runtime objects:

   // index.d.ts originally: export declare const enum GrepOutputMode { Content = "content", ... }
   // Rewritten:        export declare enum GrepOutputMode { Content = "content", ... }

2. Emit explicit ESM exports (inside native/index.js marker block):

   // --- generated native exports (do not edit) ---
   // classes
   export const GrepResult = nativeBindings.GrepResult;
   export const Shell = nativeBindings.Shell;
   // functions
   export const grep = nativeBindings.grep;
   export const executeShell = nativeBindings.executeShell;
   // string/numeric enums
   export const GrepOutputMode = { Content: "content", Count: "count", ... };
   // --- end generated native exports ---

Why explicit ESM exports? Consumers import { grep } from '@oh-my-pi/pi-natives' expect named exports, but napi-rs's dynamic loader returns an object that must be manually bound.

4. Release Pipeline: Bazel Cross-Compile + Leaf Packages

scripts/bazel-natives.ts linux-x64-modern linux-x64-baseline darwin-arm64 ...
       │
       ▼
//:natives-<target> (native_addon rule)
//   └─ rust_shared_library + config transition (LTO, target-cpu, strip)
       │
       ▼
pi_natives.<platform>-<arch>[-variant].node
       │
       ▼
gen-npm-packages.ts → @oh-my-pi/pi-natives-<platform>-<arch> (optional dependency)

• x64 variants: modern (AVX2, x86-64-v3) / baseline (x86-64-v2)
• musl/gnu share filenames: CI builds separately, installs to separate dirs
• Windows MSVC static CRT: +crt-static + static_link_msvc, no VC++ Redistributable dependency
• Loader (loader-state.js): platform tag → variant detection → candidate ordering → sentinel version validation → load → __ompInstallTokioRuntime() installs bounded Tokio/Rayon pools

Why These 6 Subsystems Must Be Rust

grep / AST / fuzzy find: Native Throughput + Determinism

• Regex engine: Rust regex + pcre2 fallback, controllable JIT (OMP_PCRE2_JIT)
• Parallel walking: pi-walker uses Rayon, custom heartbeat cancellation, globset compiled down to walker level
• Memory control: Large files read only first 4 MiB (read_owned_prefix), no mmap, avoids page faults
• AST: tree-sitter grammars statically linked, parse cache reused across requests, ast-grep pattern matching zero-copy

JS alternative: shell out to ripgrep CLI, parse stdout → high startup overhead, hard to stream, can't integrate cancellation tokens.

shell / PTY: Correct Process Control

• brush shell: Pure Rust POSIX shell (parser, expansion, interpreter), controllable builtin registration
• In-process builtins: grep/rg/fd/ls/find/jq... run inside the shell process, no fork/exec — stdio piped, cwd synced, env isolated, cancellation token reaches directly
• PTY: portable_pty abstracts cross-platform differences, ConPTY (Windows) / pseudotty (Unix), dedicated reader thread + incremental UTF-8 decode, resize/kill via control channel
• Cancellation semantics: timeoutMs + AbortSignal → CancelToken → shell triggers Tokio cancellation token → sends TERM/KILL signal waves → 2s grace window → forced abort

JS child_process.spawn cannot: precise process group control, in-process builtins, unified cancellation model, PTY size sync.

walker / isolation / hashline: Cross-Platform Determinism

• pi-walker cache: key = canonical root + full WalkOptions (including follow_links, detail, gitignore...), TTL 1s, max 16 entries, stale-empty recheck
• pi-iso backend priority: macOS Apfs → Zfs → Rcopy; Linux Btrfs → Zfs → LinuxReflink → Overlayfs → Rcopy; Windows WindowsBlockClone → Projfs → Rcopy
• Diff delegates to git: When merged is a git repo, uses git diff directly — byte-identical output for git apply; non-git falls back to (size, mtime) → content compare
• WASM fallback: pi-iso's Rcopy backend is pure Rust, no platform syscalls, compiles to WASM — browser/edge runtimes can run isolation logic

Why This Architecture

Rationale Behind Decisions

Pitfalls Avoided

1. Windows thread commit limit: module_init cannot spawn threads; loader post-load installs Tokio/Rayon pools, with std::thread::Builder::spawn pre-probing how many threads the OS allows
2. musl/gnu basename collision: Deliberately share filenames, CI builds separately, loader never sees both simultaneously
3. const enum has no runtime value: gen-enums.ts forces literal object emission
4. Over-partitioned cache keys: Full WalkOptions (except cache flag) participates in key, preventing different policies from sharing stale scans
5. In-process builtins shadowing system commands: Controllable via env vars (PI_DISABLE_UUTILS_BUILTINS etc.), preserving user expectations

Lessons Learned

1. N-API contracts should be generated, not hand-written: #[napi] + napi-rs guarantees type alignment, correct async bridging, safe ThreadsafeFunction. Hand-written bindings drift with versions.
2. Separate "algorithms" from "boundaries": Rust crates export pure Rust APIs; N-API layer only does conversion. This lets crates be used by WASM, CLI, tests directly; binding changes don't touch core logic.
3. Cross-platform native behavior needs a "testable fallback": pi-iso's Rcopy backend is both the fallback, the WASM target, and the CI verification baseline.
4. Release pipeline must handle "multi-platform × multi-variant": Bazel transitions + leaf package pattern solves the optionalDependencies combinatorial explosion with CPU variants.
5. Cancellation semantics belong in the API contract: Blocking uses exceptions; async commands use typed flags. Don't mix — consumers need to handle them correctly.

References

• OMP Source: crates/pi-natives
• OMP Source: packages/natives
• native-crates.md
• natives-architecture.md
• natives-binding-contract.md
• natives-build-release-debugging.md
• natives-shell-pty-process.md
• natives-text-search-pipeline.md
• fs-scan-cache-architecture.md
• natives-rust-task-cancellation.md
• napi-rs Documentation
• Bazel rules_rust