Skip to main content
Tensorfuse serves an application as an autoscaling API endpoint. After the image is built and stored, Tensorfuse provisions the requested hardware, deploys the container, connects it to the autoscaler, and exposes an HTTPS endpoint.

​
Readiness comes before serving

Starting the container is only one part of making the endpoint available. A model server may still need to load weights and initialize its runtime. Configure a readiness endpoint so the deployment can check whether the application is ready to receive traffic. The probe port must match the port on which the server listens.

​
Capacity follows the workload

The deployment configuration sets the GPU resources per replica, minimum and maximum replicas, and concurrency target. A minimum of zero permits scale-to-zero. Returning from zero can require provisioning and application startup, so benchmark both warm traffic and the first request after an idle period. Use the configuration reference for exact fields, the benchmarking guide to measure service latency, and the deployment guide to create the endpoint.
Tensorfuse serve architecture diagram