Call your model
Send an utterance to your project's endpoint from your own code and act on the tool calls it returns.
Every deployed project has an endpoint. Send it an utterance, and it returns the tool calls your model chose. Your code then runs them against your API, with your own authentication and permissions.
The endpoint
POST https://fernfly.com/api/p/<project-id>/inferThis is the project's stable endpoint. It always answers with the newest deployment, so deploying a retrained model needs no change to your code. Copy the exact URL from Active endpoint on the project's Overview. The project id is the identifier in that URL.
Each deployment also has a versioned endpoint, /api/deployments/<deployment-id>/infer, which keeps answering with that one model. See Train and deploy.
To call your model from the OpenAI SDK, the Vercel AI SDK or LangChain, use the project's OpenAI-compatible endpoint instead. It answers with the same model and counts toward the same limits.
Authentication
Send the project API key in an X-Api-Key header:
X-Api-Key: <your project API key>An Authorization: Bearer <key> header works too. The key is shown once, at the project's first deployment. Keep it on your server: anyone with it can call your model and use your daily allowance.
To call the endpoint from a browser, as the intent bar does, turn on keyless calls instead of shipping the key. See API keys and keyless calls.
The request
{
"utterance": "show unpaid invoices from this quarter",
"meta": { "user_id": "u_1042" }
}| Field | Required | Meaning |
|---|---|---|
utterance | Yes | The sentence the user typed. |
meta | No | Any JSON object. It is returned unchanged in the response and stored with the request, so you can match responses to your own records. |
Responses
A request the model answered:
{
"calls": [
{ "name": "list_invoices", "arguments": { "status": "unpaid", "issued": "this_quarter" } }
],
"latency_ms": 41,
"log_ref": "…",
"meta": { "user_id": "u_1042" }
}| Field | Meaning |
|---|---|
calls | The tool calls to run, in order. Usually one. |
latency_ms | How long the model took, in milliseconds. |
log_ref | An opaque reference to this request. The intent bar uses it to send a user's correction. |
meta | Your meta, if you sent one. |
Two other shapes tell you the model didn't produce a call:
| Response | Meaning | What to do |
|---|---|---|
calls: [] with out_of_scope: true and a reply | The request has nothing to do with your tools. | Show the reply ("That's outside what I can help with here.") or your own message. |
needs_fallback: true | The request relates to your tools, but the model couldn't produce a call. | Run calls if there are any: they came from the AI fallback. If calls is empty, tell the user the request wasn't understood. |
A response with needs_fallback: true and quota_exceeded: true means the AI fallback would have tried, but today's fallback allowance is used up.
Always check a call before running it: that the tool exists in your app, and that the user is allowed to do what it asks.
AI fallback
When your model can't produce a call for a request that relates to your tools, Studio can ask Gemini, a general-purpose model, to try. The AI fallback switch under Settings on the project's Overview controls this.
- On: the utterance is sent to Gemini, with your tool definitions, and any calls it returns come back with
needs_fallback: true. Each one counts toward your daily AI fallbacks. - Off: nothing is sent to a third party. The response has
needs_fallback: trueand no calls.
Either way, the request is recorded in the project's History, so you can add it to the training data later. The Free plan includes no AI fallbacks. See Usage limits.
Errors
| Status | error | Cause |
|---|---|---|
| 400 | Invalid JSON body | The body isn't valid JSON. Send Content-Type: application/json. |
| 401 | Invalid or missing API key | The key is missing or wrong, and keyless calls are off. |
| 403 | This origin is not allowed to call this endpoint. | A browser on a site that isn't in the project's allowed sites made the call. |
| 404 | Project not found | The project id in the URL is wrong. |
| 404 | No active deployment for this project | The project has no deployed model yet. |
| 429 | Too many requests. Please slow down and try again shortly. | A keyless caller went over 20 requests a minute. The body has rate_limited: true. |
| 429 | Daily request limit reached for this project. | Your plan's daily inference requests are used up. The body has quota_exceeded: true. The limit resets at 00:00 UTC. |
| 502 | Inference service unavailable | The model couldn't be reached. Retry after a short wait. |
Examples
The project's Overview has these under Show code, with your endpoint filled in.
curl -X POST 'https://fernfly.com/api/p/<project-id>/infer' \
-H 'Content-Type: application/json' \
-H 'X-Api-Key: <your project API key>' \
-d '{"utterance": "show unpaid invoices from this quarter"}'Running the calls
Fernfly returns the call. Running it is up to your code, which keeps your API's authentication and permissions in your hands. A simple pattern is a map from tool name to a function:
const handlers = {
list_invoices: (args) => api.invoices.list(args),
refund_order: (args) => api.orders.refund(args.order_id, args),
}
for (const call of await toCalls(utterance)) {
const run = handlers[call.name]
if (!run) continue // a tool your app doesn't have
await run(call.arguments)
}For actions that change data, show the user what will happen and ask them to confirm before running it. The intent bar does this for you.
Revised