> ## Documentation Index
> Fetch the complete documentation index at: https://tensorfuse.io/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Choose an open model for self-hosted inference

> Evaluate Qwen3.8, DeepSeek V4, Kimi K3, and GPT-OSS for self-hosted inference: check engine support, GPU memory, task quality, and deployment readiness.

A new model release can look attractive before you know whether it fits your GPUs or works with your application. Choose an open-weight model by checking task quality, licensing, exact engine support, memory at your target context length, and measured serving cost. Treat release announcements as candidates for evaluation, not deployment guarantees.

## Recent upstream changes to evaluate

Reviewed September 6, 2026. This is an engineering shortlist based on primary release sources, not a ranking of search volume or model quality.

| Model family       | Recent evidence                                                                   | What to evaluate next                                                       |
| ------------------ | --------------------------------------------------------------------------------- | --------------------------------------------------------------------------- |
| Qwen3.8-Flash-Next | The Qwen team announced open weights and architectural changes on August 26, 2026 | Exact checkpoint support, multimodal inputs, tool behavior, and memory use  |
| DeepSeek V4        | vLLM 0.28.0 includes sparse-attention and speculative-decoding changes            | Supported hardware, quantization, kernel requirements, and multi-GPU cost   |
| Kimi K3            | vLLM 0.28.0 includes model-specific kernel and parallelism improvements           | Required topology, memory headroom, and performance at useful concurrency   |
| GPT-OSS            | vLLM maintains a serving recipe for the 20b and 120b models                       | A reproducible baseline for reasoning workloads and application integration |

Sources: [Qwen3.8-Flash-Next announcement](https://qwen.ai/blog?id=qwen3.8-flash-next), [vLLM 0.28.0 release notes](https://github.com/vllm-project/vllm/releases/tag/v0.28.0), and [upstream GPT-OSS recipe](https://docs.vllm.ai/projects/recipes/en/latest/OpenAI/GPT-OSS.html). These upstream developments do not certify a Tensorfuse deployment configuration for every model in the table.

## Estimate memory before choosing GPUs

Start with the stored weights, then budget for runtime state, attention caches, activations, and execution buffers. A mixture-of-experts model's active parameter count describes work per token; it does not represent all the weights that must be stored or made available to the runtime.

Quantization changes weight storage and can change kernel requirements and quality. A weights-only estimate therefore cannot establish how many concurrent long-context requests will fit. Test the intended context length and concurrency with the engine's startup and memory logs.

For non-NVIDIA deployments, use the [alternate silicon explainers](/docs/guides/inference/hardware/overview) to check runtime, compiler, and kernel requirements alongside memory capacity.

## Check the application contract

Run a small evaluation set from real tasks before a large throughput benchmark. For a coding agent, test tool selection, argument validity, multi-turn state, and completion of the task. For extraction, test schema compliance and factual accuracy. For multimodal tasks, test the actual image, audio, or video inputs your application accepts.

Also verify the model license, chat template, stop tokens, tokenizer revision, reasoning output format, and parser configuration. An HTTP response alone does not establish a working agent or reliable structured output.

## Turn a candidate into a deployment recipe

Record the exact checkpoint revision, engine image digest, GPU configuration, launch arguments, and environment. Confirm startup, health checks, authenticated streaming requests, and behavior under load. Publish performance with prompt lengths, output lengths, concurrency, cache state, and failures.

Use [vLLM vs SGLang](/docs/guides/inference/vllm-vs-sglang) to choose an engine and [benchmark latency and cost](/docs/guides/inference/benchmarking) to compare candidates. The [GPT-OSS guide](/docs/guides/modality/text/openai_oss) provides a concrete Tensorfuse configuration to review and validate on your cluster.

## Related guides

<CardGroup cols={2}>
  <Card title="Deploy GPT-OSS on AWS" href="/docs/guides/modality/text/openai_oss">Configure the server, secrets, ports, and client requests.</Card>
  <Card title="Plan serverless inference" href="/docs/guides/inference/overview">Choose between warm serving, scale-to-zero, and batch jobs.</Card>
</CardGroup>