The OpenFactoryAI blog
Stop worrying about models. Read about the work.
Practical notes on coding agents, model-agnostic inference, and shipping software you can prove.
Complete technical seriesThe Inference and Automation Field GuideInference, Loop Engineering, agent training, swarms, automation, and outcome economics.21 guides →Three Factories. One Missing Layer.
AI factory can mean a GPU plant, a model-production system, or a software-delivery system. Here is the map, the current software-factory landscape, and why OpenFactoryAI sits outside the line as proof.
The Last Seat
The rise and fall of software GCCs and ODCs is not the disappearance of global engineering. It is the end of the seat, person-month, and billable pyramid as the default unit of software capacity.
Swarms Multiply Errors
Multi-agent systems create useful search diversity only when roles, evidence, state ownership, communication, merge authority, cancellation, and verification are engineered explicitly.
The Last Step Gets Too Much Credit
Long agent workflows need temporal, structural, and multi-agent credit assignment that distinguishes decisive work from visible final actions without rewarding plausible narration.
Reward Is a Bug Report
Reinforcement learning for agents succeeds or fails on environment state, action contracts, reset fidelity, verifiers, reward timing, and the shortcuts the reward accidentally permits.
The Benchmark Ends Too Soon
Agent evaluation must measure the production task after generation: execution, verification, rework, review, latency, cost, recovery, and value under the real workload.
No Trace. No Truth.
Agent observability must connect prompts, tokens, retrieval, tools, state transitions, costs, policy decisions, artifacts, and verification to one terminal outcome.
After the Demo
AI-native development begins when an agent task must survive timeouts, retries, process death, duplicate delivery, changing repositories, and partial side effects without losing the truth.
TokenMaxing
TokenMaxing is not asking every model to think longer. It allocates retries, branches, critiques, context, and verification where the next unit of inference has positive marginal verified-outcome value.
The Loop Is the Runtime
An agent is a stateful runtime around inference. Define observe, decide, act, verify, stop, authority, budgets, event logs, recovery, and a complete eight-call workflow trace before tuning the prompt.
Make Invalid Tokens Impossible
Constrained decoding masks tokens that cannot lead to valid JSON, SQL, or tool calls. Learn the syntax/semantics boundary, tokenizer alignment, retry economics, schema design, validation, authorization, and safe execution.
Seven Places RAG Can Lie
RAG is a control system across ingestion, retrieval, reranking, context, generation, citations, and feedback. Trace each failure, measure conditional stage quality, and stop blaming the vector database for every wrong answer.
The Window Is Not the Memory
A large context window is capacity, not a mandate to fill it. Allocate tokens across policy, evidence, tools, history, and output; test position, distractors, compression, latency, and verified outcome quality.
Route the Work
An LLM router is a policy under uncertainty. Compare fixed models, cascades, and learned routing using expected outcome cost, calibration, privacy, latency, availability, and a reproducible 1,000-request scenario.
Four Bits. Full Consequences.
Quantization changes the precision of weights, activations, or KV state. Calculate the real memory budget, understand kernels and calibration, and validate task-specific quality before claiming faster inference.
The Draft Is Allowed to Be Wrong
Speculative decoding uses cheap draft work and parallel target verification to advance several output tokens per target step. Derive the breakeven, preserve the target distribution, and test acceptance, memory, latency, and throughput.
The Batch Never Waits
Continuous batching schedules LLM inference one token iteration at a time. See how requests enter and leave a live batch, why prefills stall decodes, and how fairness and memory admission become product policy.
The Cache Ladder
LLM caching is four different architectural decisions. Learn what exact, semantic, prefix, and KV caches reuse, how to key and invalidate them, and when a hit costs more than a miss.
Capacity Is a Queue
Fast models can serve slow products when bursts, long prompts, agent fan-out, batching, memory pressure, and high utilization create queues. Use queueing math, goodput, and admission control to plan inference capacity.
The Token Bill Lies
Token prices are an input cost, not an automation business case. Build the full unit economics across model calls, tools, retries, review, recovery, false acceptance, and fixed integration cost.
From Prompt to Proof
Follow one automated request through gateway, routing, context, queueing, prefill, decoding, tools, verification, and commit. The important latency is not first token. It is time to a verified outcome.
Who signs off on AI written code?
When an agent writes the change, a human still has to vouch for it. Today that human never gets to go home. Here is what replaces them.
What a factory must prove
The five things any agent built artifact has to carry before you trust it, and why each one ships as evidence rather than as a promise.
Code got free. Trust didn't.
Models generate fluent software cheaply, so the bottleneck moved. The unsolved problem is proving the output is safe to ship.