Docs

Call your model

Send an utterance to your project's endpoint from your own code and act on the tool calls it returns.

Every deployed project has an endpoint. Send it an utterance, and it returns the tool calls your model chose. Your code then runs them against your API, with your own authentication and permissions.

The endpoint

POST https://fernfly.com/api/p/<project-id>/infer

This is the project's stable endpoint. It always answers with the newest deployment, so deploying a retrained model needs no change to your code. Copy the exact URL from Active endpoint on the project's Overview. The project id is the identifier in that URL.

Each deployment also has a versioned endpoint, /api/deployments/<deployment-id>/infer, which keeps answering with that one model. See Train and deploy.

To call your model from the OpenAI SDK, the Vercel AI SDK or LangChain, use the project's OpenAI-compatible endpoint instead. It answers with the same model and counts toward the same limits.

Authentication

Send the project API key in an X-Api-Key header:

X-Api-Key: <your project API key>

An Authorization: Bearer <key> header works too. The key is shown once, at the project's first deployment. Keep it on your server: anyone with it can call your model and use your daily allowance.

To call the endpoint from a browser, as the intent bar does, turn on keyless calls instead of shipping the key. See API keys and keyless calls.

The request

{
  "utterance": "show unpaid invoices from this quarter",
  "meta": { "user_id": "u_1042" }
}
FieldRequiredMeaning
utteranceYesThe sentence the user typed.
metaNoAny JSON object. It is returned unchanged in the response and stored with the request, so you can match responses to your own records.

Responses

A request the model answered:

{
  "calls": [
    { "name": "list_invoices", "arguments": { "status": "unpaid", "issued": "this_quarter" } }
  ],
  "latency_ms": 41,
  "log_ref": "…",
  "meta": { "user_id": "u_1042" }
}
FieldMeaning
callsThe tool calls to run, in order. Usually one.
latency_msHow long the model took, in milliseconds.
log_refAn opaque reference to this request. The intent bar uses it to send a user's correction.
metaYour meta, if you sent one.

Two other shapes tell you the model didn't produce a call:

ResponseMeaningWhat to do
calls: [] with out_of_scope: true and a replyThe request has nothing to do with your tools.Show the reply ("That's outside what I can help with here.") or your own message.
needs_fallback: trueThe request relates to your tools, but the model couldn't produce a call.Run calls if there are any: they came from the AI fallback. If calls is empty, tell the user the request wasn't understood.

A response with needs_fallback: true and quota_exceeded: true means the AI fallback would have tried, but today's fallback allowance is used up.

Always check a call before running it: that the tool exists in your app, and that the user is allowed to do what it asks.

AI fallback

When your model can't produce a call for a request that relates to your tools, Studio can ask Gemini, a general-purpose model, to try. The AI fallback switch under Settings on the project's Overview controls this.

  • On: the utterance is sent to Gemini, with your tool definitions, and any calls it returns come back with needs_fallback: true. Each one counts toward your daily AI fallbacks.
  • Off: nothing is sent to a third party. The response has needs_fallback: true and no calls.

Either way, the request is recorded in the project's History, so you can add it to the training data later. The Free plan includes no AI fallbacks. See Usage limits.

Errors

StatuserrorCause
400Invalid JSON bodyThe body isn't valid JSON. Send Content-Type: application/json.
401Invalid or missing API keyThe key is missing or wrong, and keyless calls are off.
403This origin is not allowed to call this endpoint.A browser on a site that isn't in the project's allowed sites made the call.
404Project not foundThe project id in the URL is wrong.
404No active deployment for this projectThe project has no deployed model yet.
429Too many requests. Please slow down and try again shortly.A keyless caller went over 20 requests a minute. The body has rate_limited: true.
429Daily request limit reached for this project.Your plan's daily inference requests are used up. The body has quota_exceeded: true. The limit resets at 00:00 UTC.
502Inference service unavailableThe model couldn't be reached. Retry after a short wait.

Examples

The project's Overview has these under Show code, with your endpoint filled in.

curl -X POST 'https://fernfly.com/api/p/<project-id>/infer' \
  -H 'Content-Type: application/json' \
  -H 'X-Api-Key: <your project API key>' \
  -d '{"utterance": "show unpaid invoices from this quarter"}'

Running the calls

Fernfly returns the call. Running it is up to your code, which keeps your API's authentication and permissions in your hands. A simple pattern is a map from tool name to a function:

const handlers = {
  list_invoices: (args) => api.invoices.list(args),
  refund_order: (args) => api.orders.refund(args.order_id, args),
}

for (const call of await toCalls(utterance)) {
  const run = handlers[call.name]
  if (!run) continue // a tool your app doesn't have
  await run(call.arguments)
}

For actions that change data, show the user what will happen and ask them to confirm before running it. The intent bar does this for you.

Revised

Was this page helpful?