Vapi supports using any OpenAI-compatible endpoint as the LLM. This includes services like OpenRouter, AnyScale, Together AI, or your own server.

​
Using an LLM provider

You’ll first want to POST your API key via the /credential endpoint: 
{
  "provider": "openrouter",
  "apiKey": "<YOUR OPENROUTER KEY>"
}
Then, you can create an assistant with the model provider: 
{
  "name": "My Assistant",
  "model": {
    "provider": "openrouter",
    "model": "cognitivecomputations/dolphin-mixtral-8x7b",
    "messages": [
      {
        "role": "system",
        "content": "You are an assistant."
      }
    ],
    "temperature": 0.7
  }
}

​
Using your server

To set up your server to act as the LLM, you’ll need to create an endpoint that is compatible with the OpenAI Client. For best results, your endpoint should also support streaming completions. If your server is making calls to an OpenAI compatble API, you can pipe the requests directly back in your response to Vapi. If you’d like your OpenAI-compatible endpoint to be authenticated, you can POST your server’s API key and URL via the /credential endpoint: 
{
  "provider": "custom-llm",
  "apiKey": "<YOUR SERVER API KEY>"
}
If your server isn’t authenticated, you can skip this step. Then, you can create an assistant with the custom-llm model provider: 
{
  "name": "My Assistant",
  "model": {
    "provider": "custom-llm",
    "url": "<YOUR OPENAI COMPATIBLE ENDPOINT BASE URL>",
    "model": "my-cool-model",
    "messages": [
      {
        "role": "system",
        "content": "You are an assistant."
      }
    ],
    "temperature": 0.7
  }
}