Home / Writing and talks / AI systems
ML decides, the LLM writes, code checks
Most of what gets called AI in a product is three different kinds of work. Knowing which part is which is how you keep a regulated customer's trust, and how you avoid reaching for an agent framework when a for-loop would do.
I spent some time playing with a bank's public product data, pulled through its API, to reimagine how AI could work in a regulated environment. I worked through five use cases and built three of them as working prototypes. I labelled every step by the kind of work it was, because the first question a risk team asks is which part is AI, and what happens when it is wrong. The labels turned into a rule I now apply to everything:
Machine learning decides who or what. The language model decides how it is said, never who gets it. Code checks both. A person approves.
Four kinds of work
Plain code
Data, business rules, search, calculations, checks, the interface. Deterministic and testable.
Decides what is trueMachine learning
Learns patterns from data to predict or rank, such as who is likely to respond. Measurable and explainable.
Decides who or whatLanguage model
Reads or writes language: messages, summaries, answers. Flexible, and the hardest part to verify.
Decides how it is saidA person
Approves, edits or rejects, and is accountable for the outcome to their own customers and board.
Decides whether it happensThree ways to wire it
Most of the time the honest answer is plain code plus one model call. Frameworks help in two specific places: LangChain when you need standard plumbing for documents at scale, and LangGraph when the flow has to branch, loop, keep state or wait for a person.
Your code does everything except the writing. Every prompt sits in your own files, so it is the easiest version to explain and debug.
Still a straight line, but each step is a swappable component. The value is not rewriting loaders and retrievers, and changing models or stores without touching the rest.
Steps share one saved state. Edges can branch, loop back to redraft, call tools and pause for a person. The graph shows every path the system can take, and tracing records the one it did.
Five banking use cases, one pattern
The same split holds across very different jobs. In four of the five, the decision is made by a model or by rules, and the language model only writes. The one that needs an orchestration framework is the one with a loop and an approval in the middle. The three I built are on their own page.
| Use case | What decides | The language model's job | What code checks | In production |
|---|---|---|---|---|
| Product answers for staff | Code retrieves the relevant product sections (search is a scoring formula, not ML) | Answers only from those sections, cites them, or refuses | Every citation points to a real section; a freshness note is added | Plain SDK. LangChain's loaders and retrievers save plumbing at scale |
| A copilot for relationship managers | Rules and a simple model rank which clients need attention | Writes talking points only from facts computed in code | Every number and claim matches a source fact | LangGraph, for the redraft loop, a safe fallback and an approval that can wait hours |
| Campaign targeting | An uplift model picks who the offer changes behaviour for, after consent filtering | Writes one variant of the offer per segment | Rates, terms and banned phrases; one retry, then a template | No framework. A scheduled ML pipeline and a loop |
| Next best product | A propensity model ranks; eligibility rules decide what is allowed | Writes a one-line opener for the relationship manager | It names the right product and invents no figures | No framework. Batch scoring, an eligibility service and a cached API |
| Re-engaging dormant investors | A model picks who might return; rules pick an education-first nudge | Writes the nudge | Blocks tickers, buy or sell language and past returns | No framework for nudges. LangGraph only for a research brief with tools and a compliance sign-off |
When a framework earns its place
The copilot is the clearest case. Every brief needs the same data, so gathering it is fixed steps in code, not an agent choosing tools. What a graph adds is the loop and the pause: a failed fact-check sends the draft back at most twice, then falls back to a template flagged for review, and the whole thing waits, with its state saved, until the relationship manager approves. Every step is named and traced, which is exactly what a bank's risk review wants to see.
Questions I get asked
Retrieval or fine-tuning?
They solve different problems. Retrieval leaves the model alone and looks documents up when the question is asked, so it suits knowledge that changes, such as product terms and policies, and gives you citations, an audit trail and per-user access control.
Fine-tuning changes the model's weights, so it suits behaviour: a format, a tone, a narrow classification, or a smaller model doing one job cheaply. It costs more to update and can make the model worse at things it used to do well. Techniques like LoRA, which train small add-on weights and leave the base model frozen, make it cheaper. In a bank I default to retrieval, and fine-tune only a high-volume, narrow task.
Why not let a second model check the first?
For exact facts like balances, rates and dates, code is cheaper, repeatable, and doesn't need checking itself. I add a model as a judge only for what code can't check, such as whether a claim is supported by its source, and only after comparing its verdicts with human-labelled examples.
How do you evaluate an agent?
Check the final output, and also the path: did it call the right tools, in a sensible order, in a reasonable number of steps? Track cost and latency per run, and replay a set of recorded scenarios before every release. Every failure becomes a new test case.
What are the security risks?
Prompt injection: retrieved documents or news can carry instructions aimed at the model. So tools are read-only and allowlisted, any action that writes waits for a person, and outputs are checked. Saved state holds customer data, so it needs encryption, masking and a retention policy.
Won't loops make it slow and expensive?
Cap the retries, use a smaller model for drafting where evaluation shows it's good enough, cache what repeats, and measure the cost of each brief against the time it saves the person using it.
Workflow or agent?
If every run needs the same information, gather it in fixed steps. That is a workflow: fewer surprises, easier to test. An agent that chooses its own tools earns its place only where what needs looking up genuinely varies, such as free-form follow-up questions.
The rule underneath
Facts live in code. The model only writes. Code checks what it wrote. A person approves anything that matters. Everything else, including which framework to use, follows from where a decision sits and what it costs to get it wrong.
None of this depends on a particular model or vendor.