Page 75 of 129
A reproducible checklist of the most common operational pitfalls: truncated results from fixed caps, selector errors tied to DOM stability, render-mode cost/determinism tradeoffs, version-change verification (v0.1.0 preview), and the non-negotiable security boundaries (URL policy, auth, concurrency, budget).
US-stock LLM agents have attracted nearly 100,000 GitHub stars, yet no Taiwan-stock project has even passed 10. I consolidated three side projects into a Taiwan-stock research agent where every conclusion must first survive a backtest; this article explains why.
Five analysts fan out in parallel within one superstep, so latency is max rather than sum; backtesting and reflection stand before synthesis, restricting the LLM to explaining evidence that already exists.
Only two roles call an LLM; every other analyst remains fully programmatic. Each call follows an Anthropic API → local Claude CLI → rules-based degradation chain, and cost accounting trusts only provider-reported values—unknown cost is never treated as $0.
This project has one core rule: every LLM conclusion must first pass a historical backtest of the same signals. When expectancy is negative, synthesis cannot issue an optimistic verdict. Each of the four traps that make backtests lie has a programmatic countermeasure.
I do not measure whether the agent ‘feels accurate.’ I freeze parameters in walk-forward OOS tests, record a hash of every input in run cards, and keep the honest 5/10 = 50% golden-eval baseline so the agent has to admit that it is not accurate yet.
Numbers are the easiest part of an LLM report to hallucinate. I therefore put every trusted number into a SHA-256-addressed evidence manifest and let the LLM cite only {{fact.id}} placeholders. If it writes a bare number, the entire output is discarded and replaced with a deterministic template.
A research request first becomes a ResearchPlan that requires human approval. External documents must be fetched in full, and verbatim quotes must be verified before they can enter a report. Quant review is always append-only, and free-text feedback never flows back into a prompt. This is the complete M5 Copilot loop.
Three frozen Pydantic contracts weld the boundary between a research artifact and order-placement authority shut: content addressing, eight hard gates, and paper-only execution, while the agent never touches credentials.
The full deployment path for a Python agent, from local uv run to Docker to a public API on Cloudflare Containers: the Worker enforces authentication, the Container runs FastAPI, secrets never enter the image, and the service sleeps automatically after 10 idle minutes—the right way for a side project to save money.
An academic-search pipeline cannot simply concatenate five APIs: use arXiv or PubMed for domain discovery, align OpenAlex and Semantic Scholar records through DOI, PMID, and arXiv IDs, then use Crossref and PubMed relationships to check the version of record, corrections, and retractions.
AG2 continues AutoGen's ConversableAgent model: agents collaborate through messages, while GroupChatManager selects the next speaker by round robin, manual choice, randomness, or an LLM.
These seven tools are not one product category: LangGraph, MAF, and Mastra emphasize durable workflows; CrewAI and AG2 emphasize multi-agent collaboration; Pydantic AI emphasizes typed Python agents; DSPy optimizes AI programs against data and metrics. Choose the control model first.
An agent should not send the user's sentence unchanged to every search service. Classify the need as exact lookup, keyword, semantic, or fielded search; move source, date, language, and field constraints into native provider parameters; then rewrite according to zero-result, overbroad, stale, or source-mismatch symptoms.
Amazon Bedrock is more than a reseller for multiple model APIs. It brings model invocation, IAM, Regions, Knowledge Bases, Guardrails, and CloudWatch into one AWS control plane. It fits teams already on AWS that value governance overhead more than the lowest token price.
Phoenix is an MIT-licensed open-source LLM observability and evaluation platform. It collects traces with OpenTelemetry and OpenInference, turns production failures into versioned datasets, compares prompt, model, or RAG changes in experiments, then writes code, human, and LLM evaluator scores back as annotations. It is not Arize AX, and self-hosting defaults require security work.
Authenticated browser state is not a convenience setting; it is a credential that can impersonate its owner. Use a dedicated low-privilege account and isolated profile, separate reading from reversible writes and high-risk transactions, and leave MFA plus final submission to a human.
Baseten puts custom-model packaging, GPU deployment, inference engines, autoscaling, and release workflows on one platform. Its value is not another OpenAI API, but retaining runtime control while operating less GPU orchestration.
Braintrust connects versioned datasets, immutable experiments, scorers, and production traces into one evaluation loop. Its value is not another score but the ability to turn production failures into offline tests. The company announced an $80 million Series B in February 2026; its customer list is company-reported.
Brave Search API exposes five endpoint families—Web, News, Images, Videos, and LLM Context—backed by Brave's own Web index and ranking models. Its core search is not merely a Google SERP wrapper.