OpenClaw local models sound simple: point the Gateway at a model running on your own hardware and stop sending prompts to a hosted provider. That is the headline. The real work starts after that.
A local model setup changes where inference happens, who owns the hardware, how much context you can afford, how tool calls behave, and what kind of fallback story your agents have when the local server is slow, unloaded, or just not good enough for the turn. It can be the right move. It can also turn a working agent into a slow, brittle one if you treat “local” as a magic safety switch.
This guide walks through the practical path: when to use local models in OpenClaw, which backend to pick, what config matters, how to test the route, and where the security risks move after you bring inference onto your own GPU box.
What OpenClaw local models actually change
In OpenClaw, a local model is still just a model provider from the agent’s point of view. The Gateway routes a turn to a provider, sends the model request, receives the response, and continues the agent loop. The difference is that the provider endpoint may live on loopback, a LAN host, a tailnet machine, or a private GPU server instead of a public API.
That gives you three useful things.
- Data control: prompts, tool results, memory snippets, and file context can stay inside your own machine or private network.
- Cost control: once you own the hardware, marginal token cost can be treated as zero in OpenClaw config, though electricity and hardware are very real costs.
- Provider independence: you are less exposed to hosted provider outages, account limits, policy changes, and rate caps.
But the trade is not free. Hosted models usually bring large context windows, tuned tool calling, provider-side safety filtering, fast cold starts, and managed capacity. With local inference, you own the rough edges. If the model emits tool JSON as plain text, drops context, stalls on a long run, or misunderstands a tool schema, that is now your operating problem.
That is why OpenClaw’s docs are blunt about local models: they work, but they raise the bar on hardware, context size, and prompt-injection defense. Small or aggressively quantized models are not just weaker at writing. They can be worse at following tool boundaries and easier to steer with malicious instructions in retrieved content.
The hardware question nobody should skip
The OpenClaw local models docs give a useful floor: aim for 2+ maxed-out Mac Studios or an equivalent GPU rig, roughly $30k+, for a comfortable agent loop. A single 24 GB GPU can handle lighter prompts, but expect more latency and tighter model choices.
That number is not there to scare hobbyists away. It is there because agents are harder on models than one-off chat. A normal OpenClaw turn may include session history, workspace instructions, memory, tool schemas, fetched web content, file snippets, and hidden control text. The model needs enough context to read that material and enough reasoning ability to use tools without turning every instruction into mush.
If you only want a local model for short drafting, summarizing, or a private side task, modest hardware may be fine. If you want your main agent to handle real tools, long sessions, cron jobs, and mixed user requests all day, under-sizing the box will show up quickly.
Use the largest, least-compromised model you can host. The fastest tiny checkpoint is rarely the best agent model. Speed is nice until a tool call goes sideways and the agent spends three turns recovering from a bad assumption.
Pick the backend before editing config
OpenClaw supports several local paths. The best one depends on how much you want the backend to manage for you.
LM Studio for the cleanest first setup
LM Studio is the easiest first stop for many operators. It gives you a GUI model loader, a local server, and OpenAI-style endpoints. OpenClaw’s docs call it the best current local stack when paired with a large model and the Responses API.
The default local server is usually:
http://127.0.0.1:1234/v1
In OpenClaw config, use `api: “openai-responses”` when the loaded LM Studio build supports it. That keeps reasoning separate from final text, which matters for channels such as WhatsApp where you only want the final answer sent back to the user. If Responses is not available, use the completions mode OpenClaw documents for LM Studio.
The guided path is still the better path for most setups:
openclaw onboard
Choose LM Studio, let OpenClaw inspect the local server, and pick the model it discovers. The provider docs say OpenClaw only suggests an installed LM Studio model when the server reports tool training and at least 16K of effective context, then verifies the route with a real completion before saving it. That one check catches a lot of bad local setups before they reach a live agent.
Ollama when you want a CLI-first local daemon
Ollama is better if you prefer a CLI workflow, model pulls, and a daemon that can run without a GUI. OpenClaw supports local-only, cloud-only, and cloud-plus-local Ollama modes.
One detail matters more than people expect: OpenClaw uses Ollama’s native API, not Ollama’s OpenAI-compatible `/v1` endpoint. The docs warn against `http://host:11434/v1` because it can break tool calling and make models emit raw tool-call JSON as plain text. Use the native base URL instead:
http://host:11434
For local and LAN Ollama hosts, OpenClaw can use a non-secret local marker such as `ollama-local`. Public remote hosts and Ollama Cloud need a real credential. If you run daily agents or cron jobs on an Ollama model, also pay attention to daemon availability. OpenClaw can skip isolated cron runs when a selected local/private Ollama host is unreachable, which is better than launching a doomed full agent turn.
vLLM, MLX, SGLang, LiteLLM, and custom proxies
For higher-throughput serving, you can point OpenClaw at an OpenAI-compatible `/v1` backend such as vLLM, MLX, SGLang, LiteLLM, OAI-proxy, or your own gateway. This is the grown-up path when you already know which server stack you trust.
The config shape is straightforward: a provider id, a `baseUrl`, an API marker or credential, an API mode, model metadata, context window, and token limit. For most custom `/v1` servers, use `api: “openai-completions”` unless the backend explicitly supports the Responses API.
LiteLLM is worth calling out because its docs describe a self-hosted OpenAI-compatible proxy that can route to 100+ model providers. That can be useful when OpenClaw should talk to one internal gateway while the gateway handles provider-specific routing. Just remember that another proxy layer does not make a weak local model better at tool use. It mostly standardizes transport and policy.
OpenClaw local models config that matters
A minimal custom provider can work, but the important fields are the ones that prevent confusion later.
{
models: {
mode: "merge",
providers: {
local: {
baseUrl: "http://127.0.0.1:8000/v1",
apiKey: "sk-local",
api: "openai-completions",
timeoutSeconds: 300,
models: [
{
id: "my-local-model",
name: "My Local Model",
reasoning: false,
input: ["text"],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 120000,
maxTokens: 8192
}
]
}
}
}
}
Keep `models.mode: “merge”` unless you are deliberately building a local-only environment. Merge mode lets your existing hosted models stay available as fallbacks. That is the sane default for most teams.
Also keep the model id rules straight. Inside `models.providers.
Set `input: [“text”, “image”]` only for a real vision-capable local model. If you mark a text-only model as vision-capable, OpenClaw may pass image attachments into turns that the backend cannot handle.
Raise `timeoutSeconds` on the provider before raising the whole agent timeout. A slow local model server needs room for connection, headers, body streaming, and the full model response. If the full agent run timeout is lower, raise that too, but start with the provider setting so the problem stays scoped.
Keep hosted fallbacks unless you have a hard local-only rule
A common mistake is switching the default model to local and removing every hosted fallback on day one. That feels clean. It is also how you turn one unloaded model into a whole-agent outage.
OpenClaw’s model failover behavior is built for this. Configured defaults and cron-selected primaries can use fallback chains. Explicit user session selections stay strict, which is right: if a user chooses a specific model, OpenClaw should not silently answer from a different one.
A practical production setup often looks like this:
- Hosted primary with local fallback while you test the local route.
- Local primary with hosted fallback once the model has passed real sessions.
- Local-only only for workloads with a hard privacy or network rule.
That middle option is the sweet spot for many self-hosted operators. You get local inference when it works and a hosted safety net when the local model is unloaded, overloaded, or out of its depth.
For more on the fallback side, read the related wcblog.in guide on OpenClaw model failover. If you are still choosing your primary provider policy, the OpenClaw model providers guide pairs well with this one.
Test transport before blaming the agent
When a local setup fails, do not start by debugging the whole agent loop. Split the problem.
First, confirm the model responds with no tools, no transcript, and no agent context:
openclaw infer model run --local --model <provider/model> --prompt "Reply with exactly: pong" --json
Then confirm Gateway routing:
openclaw infer model run --gateway --model <provider/model> --prompt "Reply with exactly: pong" --json
If both probes pass but real turns fail, the issue is usually not basic connectivity. It is probably context size, tool support, chat template behavior, timeout pressure, or model quality.
That is when OpenClaw’s local model lean mode can help. It trims the heavy tool surface for local models, keeping the turn smaller and easier for the model to follow. If the backend still emits malformed tool calls, declare stricter compatibility. Some local servers need string-only message content. Some reject extra message keys. Some models simply do not support tools well enough for agent work. In that case, set `supportsTools: false` for that model and use it for tool-free tasks.
Local model services for cold-start control
If you do not want a local server running all day, OpenClaw can start it only when a selected provider needs it. The `models.providers.
The flow is simple: OpenClaw probes the health endpoint, starts the process if it is down, waits until it is ready, sends the model request, and later stops the process if `idleStopMs` is set. Startup is serialized per provider and command, so concurrent requests do not spawn duplicate servers for the same configured service.
This is useful for expensive GPU processes, but do not hide cold-start pain from yourself. If a model takes minutes to load, set a realistic readiness timeout and test a user-facing turn after idle shutdown. A setup that looks fine when the model is already warm may feel broken during the first morning request.
Security tradeoffs: privacy does not erase prompt-injection risk
Local inference helps with data routing. It does not automatically make the agent safer.
Prompt injection is still a problem because the hostile instruction may come from a webpage, email, file, ticket, or chat message that the agent is supposed to process. The model still has to separate user intent, tool policy, source text, and untrusted content. Smaller or heavily quantized models often struggle more with those boundaries.
For serious OpenClaw local models, keep these rules:
- Run the largest model your hardware can support at acceptable latency.
- Keep Gateway sandboxing and tool approval policies tight.
- Use hosted fallbacks for high-risk or high-complexity turns unless local-only is mandatory.
- Do not expose local model endpoints casually on public networks.
- Prefer loopback, LAN, tailnet, or private DNS origins with clear trust boundaries.
OpenClaw already treats exact configured local origins differently from random private origins, and metadata or link-local style targets stay blocked unless explicitly opted in. That is good plumbing. Still, the security line is the full agent system, not the model endpoint alone.
Setup checklist
Before moving a daily-use OpenClaw agent to local inference, run this checklist.
- Pick the backend: LM Studio for easiest setup, Ollama for CLI daemon workflow, custom `/v1` for advanced serving.
- Load the largest model you can run with enough context for agent turns.
- Use `openclaw onboard` when possible so OpenClaw can discover and verify the route.
- Keep `models.mode: “merge”` while testing.
- Set provider-local model ids correctly.
- Declare text or vision input honestly.
- Set provider `timeoutSeconds` for slow local responses.
- Run the local and Gateway `pong` probes.
- Test a real tool-using agent turn.
- Add compatibility declarations only after you see the actual failure mode.
- Keep a hosted fallback unless the workload must be local-only.
- Document how the model server starts, warms, and restarts after a host reboot.
FAQ about OpenClaw local models
Can OpenClaw run fully local?
Yes, if your configured model providers, embeddings, tools, and external integrations allow it. For the model layer, OpenClaw can route to LM Studio, Ollama, and custom local OpenAI-compatible endpoints. A fully local agent is still limited by the model’s context, tool ability, and your hardware.
Should I start with LM Studio or Ollama?
Start with LM Studio if you want the lowest-friction GUI setup and a clean local server. Start with Ollama if you prefer CLI model management and a daemon-style workflow. Use a custom OpenAI-compatible server when you already have a serving stack such as vLLM, MLX, SGLang, LiteLLM, or a proxy gateway.
Why does OpenClaw warn against Ollama’s `/v1` endpoint?
Because OpenClaw’s Ollama provider is built for the native Ollama API. The docs warn that the `/v1` OpenAI-compatible URL can break tool calling and cause raw tool-call JSON to appear as normal assistant text. Use `http://host:11434`, not `http://host:11434/v1`.
Can I use local models only as fallback?
Yes. That is a good way to test local inference without betting the whole agent on it. You can keep a hosted primary, add a local fallback, and watch how the local model performs before making it the primary.
What is the first thing to test when a local model fails?
Run a plain `pong` probe with `openclaw infer model run –local`, then run the same probe through `–gateway`. If both pass, move on to context, tools, compatibility, and timeout settings. If either fails, fix transport first.
Final take
OpenClaw local models are worth running when you need stronger data control, provider independence, or hands-on model experimentation. They are not a shortcut around operations work.
Start with LM Studio or Ollama. Keep hosted fallbacks while you learn the failure modes. Test plain inference before full agent turns. Use local model services when cold starts need management. Most of all, do not confuse “runs on my GPU” with “ready to run my agent.”
If you are building a self-hosted AI stack around OpenClaw, local inference should be one layer in the design, not the whole design. Pair it with sane model failover, strict tool policy, and a model large enough to respect the boundaries your agent depends on.
