Playground

The same intent→action request, run through a 1.5B Fern model and through GPT-4o, side by side. Watch the structured call, the latency, and the cost for each.

Fern model

Turn scheduling requests into calendar actions. Book events, check your agenda, cancel by id, and find a free slot.

add a kickoff Thursday at 1pm, invite Wei

book a retro next Friday at 5pm with Sofia

set up a check-in with Marcus on Tuesday 12:30

what's on my calendar Saturday

show my schedule for next Thursday

do I have anything 2026-08-15

delete event 612

cancel the meeting with id 925

find a free 75 minute slot Saturday

any 105 min gap next Friday?

find time for a 50 minute meeting 2026-08-15

Methodology

Every model works from the identical tool schema, and each Fern size was trained on it and is scored on the same held-out prompts. GPT-4o runs via the OpenAI API at published rates ($2.50 / $10 per 1M input / output tokens) - its cost includes re-sending the whole tool spec on every call. Fern runs on a single RTX PRO 6000 with no per-call API fee and never re-sends the schema: the 12M model has it built in, and the larger sizes receive a registered tool_set_id. Latency is wall-clock end-to-end. “Match” means the predicted call exactly matched tool name and arguments; “right tool” means the correct tool was chosen but argument(s) differs. Fern always runs live; because GPT-4o is deterministic here (temperature 0) and returns the same answer for a given preset, its result is cached and refreshed periodically. The cost shown is a representative per-call figure.