Skip to main content

​
OpenAI-Compatible Models

Many LLM providers expose an OpenAI-compatible Chat Completions API. RadarOS lets you connect to any of them by simply providing a baseURL and apiKey — no custom provider code required. This works for providers like Together AI, Groq, Fireworks, OpenRouter, NVIDIA, DeepInfra, Cerebras, SambaNova, LM Studio, vLLM, and any other service that implements the OpenAI format.

​
Two Ways to Connect

​
Option 1: openai() Factory with Custom baseURL

The simplest approach — use the existing openai() factory: 
import { openai } from "@radaros/core";

const model = openai("mistralai/Mixtral-8x7B-Instruct-v0.1", {
  apiKey: process.env.TOGETHER_API_KEY,
  baseURL: "https://api.together.xyz/v1",
});
This works because these providers use the same API format as OpenAI.

​
Option 2: OpenAICompatibleProvider Class

For more control or when building reusable providers, use the class directly: 
import { OpenAICompatibleProvider } from "@radaros/core";

const model = new OpenAICompatibleProvider(
  "together",                                    // providerId
  "mistralai/Mixtral-8x7B-Instruct-v0.1",       // modelId
  { baseURL: "https://api.together.xyz/v1", apiKeyEnvVar: "TOGETHER_API_KEY" },
);

​
Provider Examples

​
Together AI

Access 100+ open-source models including Llama, Mistral, Qwen, and more. 
import { Agent, openai } from "@radaros/core";

const agent = new Agent({
  name: "together-agent",
  model: openai("meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo", {
    apiKey: process.env.TOGETHER_API_KEY,
    baseURL: "https://api.together.xyz/v1",
  }),
  instructions: "You are a helpful assistant.",
});

const result = await agent.run("Explain how transformers work in 3 sentences.");
console.log(result.text);
Environment: 
export TOGETHER_API_KEY="..."

​
Groq

Ultra-fast inference for open models. 
import { Agent, openai } from "@radaros/core";

const agent = new Agent({
  name: "groq-agent",
  model: openai("llama-3.3-70b-versatile", {
    apiKey: process.env.GROQ_API_KEY,
    baseURL: "https://api.groq.com/openai/v1",
  }),
  instructions: "You are a fast, helpful assistant.",
});

const result = await agent.run("What is the capital of France?");
console.log(result.text);
Environment: 
export GROQ_API_KEY="..."

​
Fireworks AI

Optimized inference for open-source models. 
import { Agent, openai } from "@radaros/core";

const agent = new Agent({
  name: "fireworks-agent",
  model: openai("accounts/fireworks/models/llama-v3p1-70b-instruct", {
    apiKey: process.env.FIREWORKS_API_KEY,
    baseURL: "https://api.fireworks.ai/inference/v1",
  }),
  instructions: "You are a helpful assistant.",
});

const result = await agent.run("Write a haiku about programming.");
console.log(result.text);
Environment: 
export FIREWORKS_API_KEY="..."

​
OpenRouter

Access models from multiple providers through a single API. 
import { Agent, openai } from "@radaros/core";

const agent = new Agent({
  name: "openrouter-agent",
  model: openai("anthropic/claude-sonnet-4-20250514", {
    apiKey: process.env.OPENROUTER_API_KEY,
    baseURL: "https://openrouter.ai/api/v1",
  }),
  instructions: "You are a helpful assistant.",
});

const result = await agent.run("What are the pros and cons of microservices?");
console.log(result.text);
Environment: 
export OPENROUTER_API_KEY="..."

​
NVIDIA NIM

NVIDIA-hosted inference for optimized models. 
import { Agent, openai } from "@radaros/core";

const agent = new Agent({
  name: "nvidia-agent",
  model: openai("meta/llama-3.1-70b-instruct", {
    apiKey: process.env.NVIDIA_API_KEY,
    baseURL: "https://integrate.api.nvidia.com/v1",
  }),
  instructions: "You are a helpful assistant.",
});

const result = await agent.run("Explain GPU parallelism.");
console.log(result.text);
Environment: 
export NVIDIA_API_KEY="..."

​
DeepInfra

Serverless inference at low cost. 
import { Agent, openai } from "@radaros/core";

const agent = new Agent({
  name: "deepinfra-agent",
  model: openai("meta-llama/Meta-Llama-3.1-70B-Instruct", {
    apiKey: process.env.DEEPINFRA_API_KEY,
    baseURL: "https://api.deepinfra.com/v1/openai",
  }),
  instructions: "You are a helpful assistant.",
});

const result = await agent.run("What is retrieval-augmented generation?");
console.log(result.text);
Environment: 
export DEEPINFRA_API_KEY="..."

​
LM Studio (Local)

Connect to a locally-running LM Studio server. 
import { Agent, openai } from "@radaros/core";

const agent = new Agent({
  name: "local-agent",
  model: openai("local-model", {
    apiKey: "lm-studio",  // LM Studio doesn't validate keys
    baseURL: "http://localhost:1234/v1",
  }),
  instructions: "You are a helpful assistant.",
});

const result = await agent.run("Hello! How are you?");
console.log(result.text);

​
vLLM (Self-Hosted)

Connect to a self-hosted vLLM server. 
import { Agent, openai } from "@radaros/core";

const agent = new Agent({
  name: "vllm-agent",
  model: openai("meta-llama/Meta-Llama-3.1-8B-Instruct", {
    apiKey: "token-abc123",  // your vLLM auth token
    baseURL: "http://localhost:8000/v1",
  }),
  instructions: "You are a helpful assistant.",
});

const result = await agent.run("What is vLLM?");
console.log(result.text);

​
Tool Calling

Tool calling works with any OpenAI-compatible provider that supports function calling: 
import { Agent, openai, defineTool } from "@radaros/core";
import { z } from "zod";

const agent = new Agent({
  name: "tool-agent",
  model: openai("meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo", {
    apiKey: process.env.TOGETHER_API_KEY,
    baseURL: "https://api.together.xyz/v1",
  }),
  tools: [
    defineTool({
      name: "getWeather",
      description: "Get current weather for a city",
      parameters: z.object({ city: z.string() }),
      execute: async ({ city }) => `${city}: 22°C, Sunny`,
    }),
  ],
});

const result = await agent.run("What's the weather in Paris?");
console.log(result.text);
Not all models or providers support tool calling. Check your provider’s documentation for function calling support. Larger models (70B+) generally have better tool calling reliability.

​
Registering a Custom Provider

If you use a specific endpoint frequently, register it as a named provider: 
import { modelRegistry, OpenAICompatibleProvider } from "@radaros/core";

modelRegistry.register(
  "together",
  (modelId) =>
    new OpenAICompatibleProvider("together", modelId, {
      baseURL: "https://api.together.xyz/v1",
      apiKeyEnvVar: "TOGETHER_API_KEY",
    }),
);

// Now use it like a built-in provider
const model = modelRegistry.resolve("together", "meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo");

​
Supported Provider Endpoints

ProviderBase URLEnv VarTool Calling
Together AIhttps://api.together.xyz/v1TOGETHER_API_KEYYes (most models)
Groqhttps://api.groq.com/openai/v1GROQ_API_KEYYes
Fireworkshttps://api.fireworks.ai/inference/v1FIREWORKS_API_KEYYes
OpenRouterhttps://openrouter.ai/api/v1OPENROUTER_API_KEYModel-dependent
NVIDIA NIMhttps://integrate.api.nvidia.com/v1NVIDIA_API_KEYYes
DeepInfrahttps://api.deepinfra.com/v1/openaiDEEPINFRA_API_KEYYes
Cerebrashttps://api.cerebras.ai/v1CEREBRAS_API_KEYYes
SambaNovahttps://api.sambanova.ai/v1SAMBANOVA_API_KEYModel-dependent
LM Studiohttp://localhost:1234/v1N/AModel-dependent
vLLMhttp://localhost:8000/v1CustomModel-dependent

​
Full Example

import { Agent, openai, CostTracker, defineTool } from "@radaros/core";
import { z } from "zod";

const costTracker = new CostTracker({
  pricing: {
    "meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo": { promptPer1k: 0.00088, completionPer1k: 0.00088 },
  },
});

const agent = new Agent({
  name: "together-assistant",
  model: openai("meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo", {
    apiKey: process.env.TOGETHER_API_KEY,
    baseURL: "https://api.together.xyz/v1",
  }),
  instructions: "You are a helpful coding assistant.",
  tools: [
    defineTool({
      name: "searchDocs",
      description: "Search documentation",
      parameters: z.object({ query: z.string() }),
      execute: async ({ query }) => `Results for "${query}": ...`,
    }),
  ],
  costTracker,
  maxToolRoundtrips: 3,
});

const result = await agent.run("How do I set up a Next.js project with TypeScript?");
console.log(result.text);
console.log(`Cost: $${costTracker.getSummary().totalCost.toFixed(4)}`);

​
Cross-References