Run Any Open-Source Model With Two Lines of Code: Together AI's OpenAI-Compatible Inference API

TL;DR: Together AI's inference API is an OpenAI-compatible endpoint for 200+ open models — two lines of code to switch from any closed-model API to Llama, DeepSeek, or Qwen without touching the rest of your pipeline.

What you'll be able to do after this:

• Replace closed-model calls with open-weight inference without rewriting your SDK code
• Run Llama 3.3 70B, DeepSeek R1, and Qwen 2.5 on managed infrastructure at open-model pricing
• Deploy fine-tuned LoRA adapters to serverless inference using the same endpoint

Resource: Together AI inference quickstart · Sentdex Together API Basics notebook (GitHub)

The two-line swap

If your code uses the OpenAI Python SDK, switching to Together AI is a URL and API key change:

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.together.xyz/v1",
    api_key=os.environ["TOGETHER_API_KEY"],
)

response = client.chat.completions.create(
    model="meta-llama/Llama-3.3-70B-Instruct-Turbo",
    messages=[{"role": "user", "content": "Classify this support ticket: ..."}],
    max_tokens=512,
)
print(response.choices[0].message.content)

The same tools=, response_format=, and streaming patterns work identically — Together's API surface matches OpenAI's completion spec.

Model naming

Together uses <provider>/<model-id> format. Key models as of September 2026:

• Llama 3.3 70B Instruct: meta-llama/Llama-3.3-70B-Instruct-Turbo
• DeepSeek R1: deepseek-ai/DeepSeek-R1
• Qwen 2.5 72B: Qwen/Qwen2.5-72B-Instruct-Turbo
• Mistral 7B: mistralai/Mistral-7B-Instruct-v0.3

Browse the full catalog at api.together.xyz/models.

Streaming

stream = client.chat.completions.create(
    model="meta-llama/Llama-3.3-70B-Instruct-Turbo",
    messages=[{"role": "user", "content": "Explain reranking in three sentences."}],
    stream=True,
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="", flush=True)

Deploying a fine-tuned adapter

Together's native fine-tuning lets you train a LoRA adapter on the platform and serve it at the same per-token rate as the base model — no additional hosting cost per adapter slot. Once training completes, your custom model ID appears in the catalog and you call it identically to any base model. If you've trained an adapter with Unsloth or HuggingFace TRL, Together is a direct path from adapter weights to a serverless endpoint without managing vLLM infrastructure yourself.

Together vs OpenRouter: when to use which

Together AI is a single provider with its own GPU infrastructure. OpenRouter is a routing proxy across 70+ providers. Use OpenRouter to compare models during development (one string change switches the model); migrate to Together when you've committed to a specific open model and want the lowest per-token rate and no routing overhead. Together wins when you also need native fine-tuning — OpenRouter doesn't handle adapters.

Real limits

• Together's catalog (200+ models) is smaller than OpenRouter's 400+. Check api.together.xyz/models before assuming your model is available.
• No automatic provider fallback. If Together has an outage, your retry logic is your responsibility — OpenRouter handles this for you.
• The free tier allows limited daily calls; production traffic requires API credits. Per Together's pricing pages, serverless rates vary by model — verify current prices before building cost estimates.
• Together claims top time-to-first-token on many open models; independent latency comparisons shift as providers upgrade hardware. Measure in your own region and context window size before depending on published rankings.

Sources: Together AI inference quickstart · Sentdex Together API Basics notebook — GitHub · Top 5 AI Inference Platforms comparison 2026 — Deepak Gupta · Together AI on HuggingFace inference providers