Prerequisites
Before you begin, ensure you have the configured Tensorkube on your AWS account. If you haven’t done that yet, follow the Getting Started guide.Deploying SpeechT5 on Tensorfuse
Each tensorkube deployment requires two things - your code and your environment (as a Dockerfile). While deploying machine learning models, it is beneficial if your model is also a part of your container image. This reduces cold-start times by a significant margin.Code files
We will write a small FastAPI app that loads the model and serves predictions. The FastAPI app will have two endpoints -/readiness
and /tts
. Remember that the /readiness
endpoint is used by Tensorkube to check the health of your deployments.
tts_deploy.py
Environment files (Dockerfile)
Next, create yourrequirements.txt
file
requirements.txt
Dockerfile