Backed by
Trusted by
Connect
Connect your cloud account (AWS, GCP or Azure) and Tensorfuse will automatically provision the resources to manage your infra.
Deploy
Deploy ML models to your own cloud via the Tensorfuse SDK.
Data never leaves your cloud and you can start using an OpenAI compatible API.
import tensorkube
image = tensorkube.Image.from_registry(
"nvidia/cuda" ).add_python(version='3.9')
.apt_install([ 'git','git-lfs' ])
.pip_install([ 'transformers', 'torch', 'torchvision', 'tensorrt', ])
.env( { 'SOME-RANDOM-SECRET-KEY': 'xxx-xyz-1234-abc-5678', } )
.run_custom_function( download_and_quantize_model, )
@tensorkube.entrypoint(image, gpu = 'A10G')
def load_model_on_gpu():
import transformers
model = transformers.BertModel.from_pretrained('bert-base-uncased')
model.to('cuda')
tensorkube.pass_reference(model, 'model')
@tensorkube.function(image)
def infer(input: str):
model = tensorkube.get_reference('model')
# test the model on input
response = model(input)
return response
Scale
Tensorfuse automatically scales in response to the amount of traffic your app receives.
Fast cold boots with our optimized container system
Customize your environment
Describe container images and hardware specifications in simple Python. No YAML
import tensorkube
image = tensorkube.Image.from_registry(
"nvidia/cuda" ).add_python(version='3.9')
.apt_install([ 'git','git-lfs' ])
.pip_install([ 'transformers', 'torch', 'torchvision', 'tensorrt', ])
.env( { 'SOME-RANDOM-SECRET-KEY': 'xxx-xyz-1234-abc-5678', } )
.run_custom_function( download_and_quantize_model, )
@tensorkube.use_image(image)
def infer():
print('Your inference code goes Here!')
Private by default
Your model and data live within your private cloud
Scale at will
Meet user demand in real time by scaling GPU workers from zero to hundreds in seconds.
Cost effective
Reduce egress charges by using model inference within your cloud environment.
OpenAI compatible
Start using your deployment on an OpenAI compatible endpoint.
Compute utilization
Easily utilize compute resources across multiple cloud providers.
Insights and inspirations.
Explore our blogs
Pricing for every team's size
© 2024. All rights reserved.
Product
social
Privacy Policy