Custom model connections

Connect the native fx CLI to a local model server or a remote OpenAI-compatible Chat Completions endpoint. A named connection defines the endpoint, authentication, and optional model limits. Select that connection for the interactive shell, fx ask, or native ACP.

Before you start

• Use an endpoint that accepts streaming requests at /chat/completions and a model that supports function tools for coding tasks. fx cannot enable a server's missing tool-call or reasoning parser.
• Put connection definitions in the top level of your private ~/.fx/settings.json, not a repository's .fx.json or a nested workspace definition.
• Merge the examples into your existing settings. Keep unrelated preferences and other entries under providers and models.
• Restart fx after editing connection definitions. Before sending a prompt, check fx status --json and confirm that provider_endpoint is the endpoint you intended.

Ollama

Start Ollama if it is not already running, and make the model available locally:

In another terminal:

Add this connection to ~/.fx/settings.json:

{
  "provider": "local",
  "providers": {
    "local": {
      "protocol": "openai-chat-completions",
      "base_url": "http://localhost:11434/v1",
      "auth": { "type": "none" },
      "model_metadata": {
        "qwen2.5:1.5b": {
          "context_window": 32768,
          "max_output_tokens": 2048,
          "supports_tool_use": true
        }
      }
    }
  },
  "models": {
    "local": "qwen2.5:1.5b"
  }
}

The model limits above are example values. Match them to your server configuration; they control fx's request and compaction budgets, not the server's configuration. A small model can demonstrate connectivity without being reliable enough for every coding task.

Select the connection and inspect it before sending a request:

Run fx to use the same saved connection interactively. Anonymous connections send no Authorization header and do not require a Gateway account.

OpenRouter

Make OPENROUTER_API_KEY available in the environment that launches fx. Load it from your secret manager; in CI, inject it through the job's secret settings. The JSON below names the variable, not its value.

{
  "provider": "openrouter",
  "providers": {
    "openrouter": {
      "protocol": "openai-chat-completions",
      "base_url": "https://openrouter.ai/api/v1",
      "auth": { "type": "bearer", "env": "OPENROUTER_API_KEY" }
    }
  },
  "models": {
    "openrouter": "openai/gpt-4.1-mini"
  }
}

Use a model ID available to your OpenRouter account and ensure that the account has enough credit for the request. fx does not substitute a Gateway credential if this key is missing or rejected.

vLLM

Start your vLLM server with the chat template, tool-call parser, and reasoning settings required by its model. Those are server settings, not fx connection options.

For an anonymous local server listening on port 8000:

{
  "provider": "vllm",
  "providers": {
    "vllm": {
      "protocol": "openai-chat-completions",
      "base_url": "http://localhost:8000/v1",
      "auth": { "type": "none" }
    }
  },
  "models": {
    "vllm": "your-served-model-id"
  }
}

Replace your-served-model-id with the exact ID your server exposes. If the server requires a bearer key, replace auth with:

{
  "type": "bearer",
  "env": "VLLM_API_KEY"
}

Provide VLLM_API_KEY in fx's environment. For a non-loopback server, use an HTTPS base URL rather than plain HTTP.

Bring your own key

BYOK at the connection level means fx sends your key to the endpoint you configured, using Authorization: Bearer <key>. The connection reads only the environment variable named by auth.env; it does not store the key in the settings file.

For example, a direct OpenAI-compatible connection to OpenAI would use https://api.openai.com/v1, the environment variable OPENAI_API_KEY, and a model ID such as gpt-4.1-mini. That ID differs from OpenRouter's openai/gpt-4.1-mini: use the model namespace expected by each endpoint.

fx setup configures a Gateway API key, not a key for a custom connection. Native APIs or authentication schemes that need something other than Chat Completions with anonymous or bearer authentication are outside this adapter. Router-managed BYOK, where a gateway uses your vendor credentials for upstream requests, is separate from the key fx uses to authenticate to that gateway. This guide does not configure Gateway or OpenRouter's upstream BYOK settings.

Connection reference

providers is an object keyed by connection name. Names are case-sensitive, start with an ASCII letter, and contain only letters, digits, underscores, or hyphens, up to 64 characters. gateway, codex, and grok are reserved names, regardless of case.

Base URLs require HTTPS, except for loopback HTTP. Do not include credentials, query strings, or fragments in the URL. Unknown fields inside a connection definition are rejected rather than forwarded to the server.

Model metadata

Each key under model_metadata is an exact model ID. Missing fields remain unknown.

Without a known context window, fx cannot determine an automatic compaction threshold. Supply accurate metadata for the models used by the main conversation, subagents, and any reviewer_model.

fx models and fx models --json list the IDs you supply in model_metadata; they do not discover models from a custom endpoint. An explicit model ID does not need a catalog entry. The server still has to recognize it.

Select a connection for one process

fx provider <name> saves your selection. FX_PROVIDER and FX_MODEL override one process without rewriting profile settings:

A private workspace entry in settings.json can select a connection or model, but connection definitions under providers belong at the top level. Project .fx.json cannot define or select connections. See Configuration.

The interactive sign-in picker lists the built-in providers. Use the CLI or profile settings to select custom connections; do not use fx login local or fx setup to configure them.

Permissions and saved sessions

Normal permission rules still apply. In auto mode, reviews use the selected connection and its optional reviewer_model. An invalid or unavailable review leaves the action unapproved; fx does not switch to a cloud reviewer or disable permission checks.

Saved sessions retain the connection name and a non-secret endpoint/authentication-slot fingerprint. Changing or removing the endpoint or credential variable blocks implicit resume against that changed connection. Rotating a key's value in the same variable does not change the slot. Restore the original definition or explicitly select the connection you intend to use before continuing.

History remains readable without sending it to a provider:

Reasoning metadata is preserved through tool calls and saved-session continuation on the same connection and model. Signed and encrypted details remain opaque. fx omits incompatible reasoning when the connection or model changes.

Compatibility and troubleshooting

This adapter supports text, reasoning, and function-tool streaming. A stream must include a finish reason followed by [DONE]; incomplete tool calls never become executable. Cumulative usage updates replace previous observations rather than being added together.

This is not complete Gateway feature parity. Gateway-only search, credit checks, and vision fallback are unavailable. Token usage is reported when supplied, but unknown dollar cost is not shown as free. Native image input, Responses/Anthropic/Gemini APIs, custom authentication headers, and provider-specific reasoning controls are not implemented here.

Use fx status --json to inspect the selected endpoint and model. Keep provider-specific credentials and traces private when following the broader troubleshooting guide.