Skip to content
All tags

#state-forking

1 posts
aideep-dive

Serving Decision Models Fast: Flide's State Forking, Where It Wins, and Its Limits

When several questions share one context, Flide computes the shared state once and forks it per question with answers identical to asking alone; up to 9.5x faster than vLLM and SGLang at best, roughly 1.3x to 3.3x on average for shared-state workloads, and only about 1x to 3x with one question per input.