AI Systems

The hard part of building with LLMs is almost never the model. It is deciding which parts of a problem deserve one.

A model is the right tool for judgment under ambiguity: what is this message about, what is this photo of, what is this person asking for. It is the wrong tool for anything with a correct answer. Routing, pricing, permissions, arithmetic. Handing those to a model buys unpredictability and pays nothing for it.

Most systems that feel unreliable are unreliable in exactly that way. Someone let the model decide something that had a right answer.

What I keep finding

Draw the line early and write it down. Classify with the model, act with code. In the Alpha Apex engine the classifier picks a category and a confidence score, and a plain routing layer decides what happens next. That split is why it can be trusted with real inbound.

Confidence scores are load-bearing. A classifier that says “consulting, 0.94” and one that says “consulting, 0.51” should not produce the same behaviour. Most of the value of a classifier is knowing when to stop trusting it.

Add the human where the risk is, not everywhere. A confirmation step on the irreversible actions gets a system adopted. A confirmation step on everything gets it switched off.

Errors must be loud. The worst failure is silent. A dropped lead nobody hears about costs more than a crash, so every pipeline gets global error handling that alerts someone.

Measure the thing you are billed for. Building an inference API, I found that the model with the best tokenizer for a language still produced worse bills, because it burned eight times the output tokens to say the same thing. Cost per token is not cost per answer, and only one of those is on the invoice.

Providers are not load-bearing. They change constantly. Put them behind a factory on day one and swapping is an afternoon instead of a rewrite.

Where I have done this

Inference infrastructure, generation pipelines, classification systems in production, and the approval layer that makes teams comfortable turning them on.