Agentic AI, Without the Mystique

On August 1 we ran a 90-minute webinar called Agentic AI essentials + Hands-on: $0 AI agent in 15 mins. Around a hundred people signed up, we built two working agents live, and the Q&A ran to the buzzer.

The recording is worth more than the event was, so we've cut it into three segments and we're publishing them one at a time. This is the first: the 20-minute theory talk that opens the session.

It is a teaching segment, not a pitch. Fernfly appears once, as one row of six in a comparison table, disclosed as ours. Everything else is vendor-neutral, and it should stay that way. If you leave with a working mental model and go build on somebody else's stack, the segment did its job.

TL;DR. An agent is a goal, a set of tools, and a loop. The model's entire output is a function name and some arguments, so the model never touches your database, and your code stays the trust boundary it always was. Agents rarely fail at thinking. They fail at typing. That reframes the model choice: on bounded, repeating, structured work, a small specialized model beats a frontier one on speed, cost, privacy, and above all control. RAG is for what the model doesn't know. Fine-tuning is for how it should behave.

What follows is the written version, for people who'd rather read than watch.

Ask a chatbot to do something

Here is the whole argument in one customer message:

"the mug in order 4821 arrived cracked, I want a refund"

There are two useful shapes of answer.

The chat answer is a polite paragraph: sorry to hear that, head to your orders page, find order 4821, select "Return or replace items"... It is not wrong. It is the wrong shape. Your refunds system cannot consume a paragraph, so you write a parser, and you hope the next reply is phrased the same way. It usually isn't.

The intent-to-action answer is this:

create_refund({
  order_id: 4821,
  reason: "damaged"
})

Nothing to interpret, nothing to clean up. And it cannot refund an order that doesn't exist, because the schema won't let it.

Same sentence, two different jobs. Ask yourself which one your current chatbot does. For almost everybody, it's the first.

An agent is a model that can act

Three parts, and only one of them is the model:

1. A goal, in plain language. "Refund this order."
2. Tools: a finite list of things the agent is allowed to do, each with a name and typed arguments.
3. A loop: call a tool, look at the result, decide whether it's done.

Emphasis on allowed. The tool list is a permission boundary as much as a capability list. Whatever isn't in it cannot happen, which is a security property, not just a design one.

Take the tools away and you're left with a chatbot.

Where the model actually sits

The scariest version of "AI agent" is a model with its hands on production. That is not what this is.

The model never touches your database. Its entire output is a name and a bag of arguments: a string. Your code decides whether to honour it.

So the trust boundary is exactly where it has always been, at your own function. You validate, you authorise, you rate-limit, you log. There is nothing new to invent, and everything after the model is ordinary software you already know how to write, test, and roll back.

A "tool" is a function you already wrote

This is the part that gets over-mystified. A tool is a JSON description of a function: name, arguments, types. If you have ever written an OpenAPI spec, you have already done this.

{
  "name": "create_refund",
  "parameters": {
    "order_id": "integer",
    "reason": "damaged | late | wrong_item"
  }
}

The hard part isn't the schema. It's the input side:

• "mug in 4821 came cracked, refund pls"
• "order 4821 arrived broken, money back?"
• "need a refund on 4821, item was damaged"

Same intent, three phrasings, and there are a hundred more. The model's only job is to collapse all of them onto one call.

Note the enum on reason. The model cannot invent a fourth reason, because the schema doesn't contain one. Constraint is a feature, and we come back to it below.

You are not writing prompts. You are publishing an API.

"Agentic" is a dial, not a badge

The word has been claimed by the deepest version of the loop, which is why it sounds intimidating and expensive. Depth is a choice, and it has a cost curve:

• One turn. One request, one call. A command bar. A website chatbot. Most useful work lives here.
• A few turns. Call, read the result, call again. Book a slot, then confirm it.
• Open-ended. Plans, retries, sub-agents. Powerful, expensive, hard to debug.

Every extra turn is another model call, another chance to drift, another thing to debug at 2am. The chatbot we build in segment two is a one-turn agent, and it is genuinely useful.

Multi-agent frameworks are real and they are for the top rung. You should be able to justify why you aren't on the bottom one.

Where agents break

Not where you expect.

A frontier model understands the refund request perfectly well. Comprehension was never the hard part. The hard part is emitting exactly the right structured object, every time, at volume:

create_refund({
  order_id: "4821-ish",
  resaon: "seems damaged?"
})

Wrong tool. Malformed arguments. Off-schema drift. A cloud round-trip on every call.

The misspelled key is the realistic one. It is not a comprehension failure, it is a formatting failure, and it takes your endpoint down just as effectively.

Agents rarely fail at thinking. They fail at typing. Which reframes the entire model-choice question.

So what belongs in the model box?

Everything else in that pipeline is settled engineering. The one open decision is the model, and it sets your cost, your latency, your privacy posture, and your reliability for the life of the product.

The default answer is "the biggest model you can afford." It is usually wrong.

"Small" isn't one thing. It's a ladder.

Nobody is going to hand you a parameter count where a large model becomes a small one, so let's not pretend there is one. What matters is what changes as you descend:

Two things change together. Capability narrows, and the deployment story gets radically easier.

The bottom rung is the one people find hardest to believe, so: Gemma 3 at 270M, SmolLM2 at 135M, and encoder models like MiniLM well under 100M that still do serious classification work.

Somewhere below a few billion parameters, models stop being assistants and start being components. A 100M model is not a bad assistant. It is not an assistant at all. It is a part you drop into a pipeline the way you would a regex or a parser, except it tolerates the mess of real language.

If you want a boundary: around 1B to 3B is where general instruction-following starts holding up unaided. Below that, fine-tuning is not optional.

Most AI work isn't reasoning

Call it 80/20. It is a rule of thumb, not a measurement. Gaurav made the long version of this argument in A Lot of AI Work Isn't Reasoning, and it is the premise the rest of this segment rests on.

The 80% is bounded and repeatable: routing and triage, extraction, slot-filling, function calling.

The 20% is genuinely hard: open-ended reasoning, novel and ambiguous problems, long-context synthesis, work with no fixed schema.

Both are real. Only one of them needs a frontier model.

Be scrupulous about that 20%. This is not an anti-frontier-model argument. That is where the big models earn every penny, and nothing small replaces them. The mistake was never using a big model. It was using a 20% tool for 80% of the work, paying the genius rate for jobs that are, by construction, not hard.

The three-question test

No framework, no decision tree, no maturity model. Three questions about your task:

1. Does the same intent → action repeat at volume?
2. Is the output structured: a call, a field, a route?
3. Is the domain bounded?

Three yeses, a small specialized model wins. Three nos, reach for the frontier model. That's what it's for.

Mixed answers are common and fine. It usually means the task hasn't been decomposed yet, and the two halves want different models.

Function calling is one member of a family

Most teams have three or four of these buried in their stack right now, running on a frontier model because that was the easiest thing to wire up:

• Routing. Which of twelve queues does this ticket belong in?
• Extraction. Invoice number, date, and total, out of a forwarded email.
• Classification. Spam, abusive, off-policy, churn risk. One label per input.
• Redaction. Find the personal data and strip it before anything reaches your logs.
• Reranking. Given fifty search hits, put the right three at the top.
• Normalisation. Messy address, OCR sludge, free-text date, into one canonical form.
• Short summarisation. A subject line for a thread. A one-line note from a call.
• Function calling. The one this whole post is about.

Same shape every time: bounded input, structured output, and the job repeats.

Worth saying plainly: several of these do not need a generative model at all. Classification, extraction, and reranking were solved by encoder models (the BERT family) before generative AI arrived, and per unit of compute they still win. So the honest framing is not "use a small language model." It is use the smallest thing that solves your problem, and sometimes that is not a language model.

Why small wins on the 80%

• Speed. Milliseconds on your own hardware, not a round-trip to somebody else's datacentre, queued behind everybody else's requests.
• Cost. Pennies, or nothing at all if you host it. The difference between AI as a line item and AI as a margin problem.
• Privacy. It runs where your data already is. The compliance story writes itself, because there is nothing to disclose.
• Control. The underrated one.

Speed, cost, and privacy are the reasons people expect. Control is the one that changes minds, and nobody arrives with it. The instinct is that bigger is safer. On a bounded task it is backwards: bigger means more capability you are not using, and more surface area for the output to drift.

A small model on a bounded task can't wander. Lock it to your schema and you get a guarantee, instead of flexibility you have to police.

Prompt, RAG, or fine-tune?

Reach for these in this order:

1. Prompt it. Free, instant, nothing to run. Most problems genuinely stop here, and you should want them to.
2. Show examples. Few-shot. Still just a prompt, except now you pay for those examples on every single call, forever.
3. Retrieve. RAG. Look the facts up, paste them in.
4. Fine-tune. Change the weights.

This is the slide people most often get backwards, so here it is as one line:

RAG is for what the model doesn't know. Fine-tuning is for how it should behave.

Teams reach for fine-tuning hoping to teach the model facts. That mostly fails. Weights are a lossy, expensive, un-updateable place to put a fact you could have looked up. If the answer changes when your database changes, you want retrieval.

And teams reach for RAG hoping to fix behaviour. Also mostly fails. No amount of retrieved context reliably makes a model stop writing prose when you needed a schema-valid object.

Rung 2 has a cost nobody prices in. Every few-shot example is input tokens on every call, at volume, forever. Fine-tuning is partly just moving those examples out of the prompt and into the weights, where you pay for them once.

What fine-tuning actually is

It sounds like research. It is mostly data work.

What it is. Show the model many input → wanted output pairs and nudge its weights until it reproduces them. That is the whole idea.

What you need. Examples, not prose. Hundreds for one narrow task, thousands for a broad one. Out of your logs, your API spec, or generated and then spot-checked. This is the step everyone underestimates: ninety per cent of the effort is assembling examples that cover the hundred sloppy ways somebody asks for the same thing.

Why it got cheap. LoRA. Train a small adapter instead of every weight, on the order of 1% of them. A job that needed a cluster in 2023 runs on one rented GPU for the price of a coffee.

For a sense of scale, one real run from crawling a public site:

25 actions → 1,369 example pairs → 860 training steps → a working model

Your numbers will differ. Treat it as a sense of scale, not a spec.

What it will not do: teach reliable new facts, or buy reasoning the base model never had. That boundary is worth not soft-pedalling.

What you can actually reach for today

No row here wins everything. Pick the one whose shape matches your problem, and be suspicious of anyone who tells you their row is the answer. Some honest readings:

• Row one is right far more often than this crowd expects. If you make a thousand calls a month, a frontier API is cheaper than any infrastructure you could stand up, and you should not be fine-tuning anything.
• Encoder classifiers are the most underused row. If the output is a label rather than a sentence, you probably do not need a generative model.
• Do-it-yourself is where you end up if the data cannot leave your building, and the tooling is genuinely good now. Unsloth and Axolotl are free.
• Our row is deliberately the narrowest. Fernfly does intent to action and nothing else. If your problem is classification or summarisation, you want a different row.

There is no benchmark scoreboard in this segment, on purpose. A vendor's own numbers are a digression in a teaching talk, and they invite you to argue with the table instead of the argument. The transferable version: ask any vendor for the row where their model loses. If there isn't one, it isn't a measurement. Ours are published, including a domain where Fern Bud clearly trails GPT-4o, in the Playground and in the post that launched it.

The line

Small and specialized where the work repeats.

Frontier where the work is genuinely hard.

It is a both/and, not an either/or. Most production systems need both. The trouble is that most are using the chat tool for the action job, and paying for it on every call, forever.

The mistake was never using a big model. It was using a 20% tool for 80% of the work.

Coming next

Two more segments from the same session, publishing over the next few days:

• Workshop: building a website chatbot. The five steps behind the one-field version, then the same thing by hand.
• Workshop: a Chart.js agent. Twelve plain JavaScript functions, a dozen sentences, and a chart that redraws. Including the bit where we break it on purpose and don't rescue it.

If you'd rather not wait, Hoobert is this whole argument shipped as a product. It is a ⌘K command bar for WooCommerce merchants: type "refund order 1042" and it picks the right store action, fills the arguments, and runs it, in well under a second and a half on a small specialized model. There is a one-click browser demo, and the model behind it is one you can train yourself in a few minutes.

For the side-by-side version, the Playground runs the same request through a small specialized model and GPT-4o and scores both.

And if the three-question test came back three yeses for something on your own stack, build the model and tell us where it lands and where it slips.