> ## Documentation Index
> Fetch the complete documentation index at: https://tensorfuse.io/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# How to compare Groq, Cerebras, and SambaNova inference APIs

> Compare Groq, Cerebras, and SambaNova with the same task. Record API latency, token usage, errors, and correctness before comparing cost per accepted request.

By [Samagra Sharma](/docs/authors/samagra-sharma) · Reviewed September 6, 2026

A fast inference API only saves money if its available models complete your task correctly and within the deadline. Compare Groq, Cerebras, and SambaNova by sending the same task to each, retaining usage and errors, and checking the answer separately from speed.

This guide compares hosted services. Different models, precision, scheduling, and network paths make it an application comparison; the results cannot isolate chip performance. The example follows current vendor documentation and has not been executed against paid APIs for this guide.

## 1. Select models available to your accounts

Create an account with each provider and load its API key into the environment variable below using your secret manager. Each API uses `Authorization: Bearer` authentication. The script reads keys from the environment and does not print headers or keys.

| Provider  | API base URL                     | Key variable        | Choose an exact model ID                                                                   |
| --------- | -------------------------------- | ------------------- | ------------------------------------------------------------------------------------------ |
| Groq      | `https://api.groq.com/openai/v1` | `GROQ_API_KEY`      | [Groq model catalog](https://console.groq.com/docs/models)                                 |
| Cerebras  | `https://api.cerebras.ai/v1`     | `CEREBRAS_API_KEY`  | [Cerebras models API](https://inference-docs.cerebras.ai/api-reference/models/list-models) |
| SambaNova | `https://api.sambanova.ai/v1`    | `SAMBANOVA_API_KEY` | [SambaCloud model catalog](https://docs.sambanova.ai/docs/en/models/sambacloud-models)     |

Select a text chat model accessible on your account's service tier. Record its model family, context limit, reasoning behavior, and preview status. Where providers offer equivalent checkpoints, use them; otherwise compare both against the same task rubric. Model names can differ across providers.

In Bash, enter the selected IDs:

```bash theme={null}
read -r -p 'Groq model ID: ' GROQ_MODEL
read -r -p 'Cerebras model ID: ' CEREBRAS_MODEL
read -r -p 'SambaNova model ID: ' SAMBANOVA_MODEL
export GROQ_MODEL CEREBRAS_MODEL SAMBANOVA_MODEL
```

## 2. Send one bounded request to each API

Save this as `compare_apis.py` and run it with Python 3.10 or later. It sends three completion requests total, without automatic retries. Running it may consume your API credits.

```python theme={null}
import http.client
import json
import os
import time
import urllib.error
import urllib.request
from datetime import datetime, timezone

providers = {
    "GROQ": "https://api.groq.com/openai/v1",
    "CEREBRAS": "https://api.cerebras.ai/v1",
    "SAMBANOVA": "https://api.sambanova.ai/v1",
}
required = [f"{name}_{suffix}" for name in providers
            for suffix in ("API_KEY", "MODEL")]
missing = [name for name in required if not os.environ.get(name, "").strip()]
if missing:
    raise SystemExit("Set these environment variables: " + ", ".join(missing))

prompt = "Extract the order ID. Reply only with the ID: Order TF-1042 ships tomorrow."
for name, base in providers.items():
    model = os.environ[f"{name}_MODEL"].strip()
    payload = {
        "model": model,
        "messages": [{"role": "user", "content": prompt}],
        "max_completion_tokens": 256,
        "stream": False,
    }
    request = urllib.request.Request(
        base + "/chat/completions",
        data=json.dumps(payload).encode(),
        headers={
            "Authorization": "Bearer " + os.environ[f"{name}_API_KEY"],
            "Content-Type": "application/json",
        },
        method="POST",
    )
    result = {"provider": name, "requested_model": model,
              "started_at": datetime.now(timezone.utc).isoformat()}
    started = time.perf_counter()
    try:
        with urllib.request.urlopen(request, timeout=60) as response:
            result["http_status"] = response.status
            body = json.load(response)
        if not isinstance(body, dict):
            raise ValueError("Expected a response object")
        result.update(returned_model=body.get("model"), usage=body.get("usage"))
        choices = body.get("choices")
        if not isinstance(choices, list) or not choices or not isinstance(choices[0], dict):
            raise ValueError("Expected a completion choice")
        choice = choices[0]
        message = choice.get("message")
        if not isinstance(message, dict):
            raise ValueError("Expected a message object")
        output = message.get("content")
        if output is not None and not isinstance(output, str):
            raise ValueError("Expected text content")
        result.update({
            "output": output,
            "finish_reason": choice.get("finish_reason"),
        })
    except urllib.error.HTTPError as error:
        result.update(http_status=error.code, error="HTTPError")
    except (http.client.HTTPException, OSError, ValueError, TypeError) as error:
        result["error"] = type(error).__name__
    result["elapsed_seconds"] = round(time.perf_counter() - started, 4)
    print(json.dumps(result))
```

```bash theme={null}
python3 compare_apis.py > comparison.jsonl
```

The common request fields and usage responses are documented by [Groq](https://console.groq.com/docs/api-reference), [Cerebras](https://inference-docs.cerebras.ai/api-reference/chat-completions), and [SambaNova](https://docs.sambanova.ai/docs/api-reference/chat-completions/create-chat-based-completion). Check model-specific restrictions before adding sampling or reasoning settings. This initial script uses each model's defaults and records the complete usage object when supplied.

## 3. Score correctness before accepting a latency result

For this task, require the output to equal `TF-1042` after trimming whitespace. Record a separate correctness pass and whether the response met your application deadline. HTTP 200 alone is insufficient. Treat empty, malformed, truncated, and incorrect answers as failures; record missing usage as unknown rather than zero.

If `finish_reason` is `length`, the output budget may have been consumed before a final answer. Reasoning defaults differ between models. Choose supported settings or raise the shared budget, then label and rerun the comparison. For 401/403, check account access; for 429, inspect the provider's quota and rate limits before retrying.

`elapsed_seconds` measures the complete non-streaming HTTP operation from this client, including connection setup, network travel, scheduling, generation, and response reading. It does not measure time to first token. Three requests establish connectivity and a smoke test, not reliable latency percentiles. A larger evaluation should use representative inputs, repeated runs, controlled request rates, and the same client region.

## 4. Compare the actual cost of accepted requests

```text theme={null}
Cost per accepted request = attributable billed cost / accepted logical requests
```

Reconcile the returned input/output token counts with each provider's usage dashboard and bill. Account for cached input, reasoning tokens, retries, service tiers, and other billed features according to that provider's rules. Different tokenizers can produce different token counts for identical text. Compare the final bill for useful tasks instead of assuming equal token counts imply equal work or cost.

Use the [specialized accelerator explainer](/docs/guides/inference/hardware/specialized-accelerators) for the architecture differences and the [inference cost workflow](/docs/guides/inference/how-to/reduce-cost) to compare the shortlisted API with your self-hosted endpoint.