Add models in deployments

This page provides a reference guide to adding a model on a Gradient Deployments.

Leveraging the Gradient Model Registry, you can pull any stored models into your deployment. To do this, add your model’s model-id to the models parameter in the deployment spec.

Optionally, you can specify the path where you want your model mounted on the deployment container. The default path for mounting models is /opt/models.

Learn how to add a model to the model registry here.

enabled: true
image: tensorflow/serving
port: 8501
models:
  - id: model-id
    path: /opt/<model-id> #Use the model-id in the path to ensure a unique model directory
resources:
  replicas: 1
  instanceType: C4

Once your model is mounted you can load it into your application from the path it is mounted to (e.g. /opt/<model-id>/model-name ).

Note: Certain paths do not work for model mounting locations. Models must be mounted to absolute paths outside of root. Therefore model paths like / or . will cause an error in the deployment.