Using LM Studio as a local provider

[bruwy]

I'm trying to find a way to use LM Studio as a local AI provider instead of relying on paid OpenAI/Anthropic APIs.

Has anyone here managed to make this work with Omi, or found a workaround? I searched through older posts and docs but couldn’t find anything conclusive.

Any tips, examples, or guidance would be really appreciated 🙏

[Charlie284]

Short answer: this is not supported by the released Omi app at the moment.

LM Studio does expose an OpenAI-compatible API, normally at http://localhost:1234/v1, but Omi's BYOK flow accepts provider keys rather than a custom base URL and model. That missing setting is tracked in #6878. The implementation attempt in #8466 is still open, so putting a dummy value in Omi's OpenAI-key field will not redirect requests to LM Studio.

There is also an architecture issue: the normal app sends chat work to Omi's hosted backend. localhost:1234 from that backend means the backend host, not the PC running LM Studio.

The current workaround is therefore a developer/self-host route:

1. Run LM Studio's server and load a model. Its base URL for OpenAI clients is http://localhost:1234/v1.
2. Run your own Omi backend and point your app's BASE_API_URL at it, following Omi's backend setup.
3. Patch the backend's ChatOpenAI construction in backend/utils/llm/clients.py to pass LM Studio's base_url, the exact loaded model ID, and an API token/dummy key as required by the client.
4. If Omi's backend is not running on the same machine, enable LM Studio's Serve on Local Network option and use the PC's LAN address. Enable LM Studio authentication and keep the port behind your firewall; do not expose an unauthenticated inference server to the internet.

That is not yet a complete drop-in replacement. Omi's main agent chat currently follows a separate Anthropic path, and the open custom-provider PR itself says it only covers the get_llm features such as extraction, structuring, and summaries. LM Studio also does not replace Omi's speech-to-text provider.

So: it is possible with a locally hosted backend plus source changes, but there is no supported app-only configuration today. I would subscribe to #6878/#8466 rather than expecting the existing BYOK fields to work with LM Studio.