04

Field notes / Engineering judgment

Technical opinions with consequences.

Our public thinking focuses on the decisions that separate a compelling demonstration from a system worth operating.

Original notes · no generic AI trend summaries
01Evaluation

A model metric is not an operating result.

The useful metric is attached to a decision, an error cost, and an accountable owner.

Accuracy can rise while the exception queue gets worse. Forecast error can fall while the planning horizon remains unusable. Evaluation starts by naming the action that changes when the output is right or wrong.

02Architecture

The cheaper model can be the stronger system.

Reliability, latency, privacy, and observability often matter more than leaderboard distance.

A smaller pretrained model with constrained inputs and explicit review can outperform a larger dependency once production cost, failure containment, and data boundaries are included.

03Product

A good prototype is designed to die.

Its job is to answer the expensive question before production architecture hardens around an assumption.

Prototype code may survive, but that is not the success condition. A useful prototype produces evidence, exposes missing data and integrations, and creates a defensible go, change, or stop decision.