Managing Data in Gradient

Persistent Storage

Persistent Storage is accessible across Paperspace VM instances and Gradient Jobs and Notebooks. Uploading and downloading data is possible from any of these endpoints.

Once you have set up either a Gradient Notebook or a Job you will have access to Persistent Storage in /storage.

Uploading

To upload data into /storage , you can either SCP data from a local laptop/desktop to a Paperspace VM instance or use the upload functionality from within a Gradient Jupyter Notebook.

Downloading

Downloading data to /storage is as simple as using curl or wget from a Gradient Job or Notebook or Paperspace VM instance. Alternatively, you can use a browser from within a Paperspace VM instance to download files.

See a full tutorial here.

Artifacts

Get

Get the artifacts files for the job with the given id. The name of a particular file, or directory can be specified, and can include a wildcard character at the end, e.g., "myfiles"*. If no specifc file or directory is specified all artifact files will be retrieved.

Example Use

$ paperspace jobs artifactsGet --jobId "j123abc"

Properties

Name

Type

Attributes

Description

jobId

string

Id of the job to get artifacts for

files

string

<optional>

Optional; if getting only certain files, a wildcard pattern to match against, e.g., "myfiles*". Note: if you include a wildcard you must double-quote the files argument.

dest

string

<optional>

Optional; an existing directory to copy the artifacts files to.

json

boolean

<optional>

Optional; return JSON object instead of writing to standard out. '--json' with no value is equivalent to true.

List

List job artifact files for the specified job.

Example Use

$ paperspace jobs artifactsList --jobId "j123abc" --size true

Properties

Name

Type

Attributes

Description

jobId

string

Id of the job to list artifacts for

files

string

<optional>

Optional; wildcard expression of file(s) to list, e.g., "myfiles*". Note: if you include a wildcard you must double-quote the files argument.

size

boolean

<optional>

Optional; include file size in bytes. '--size' with no value is equivalent to true.

links

boolean

<optional>

Optional; include https links to artifacts. Note: links are only valid for 8 hours. '--links' with no value is equivalent to true.

json

boolean

<optional>

Optional; return JSON object instead of writing to standard out. '--json' with no value is equivalent to true.

Destroy

Destroy artifact files of the job with the given id. The name of a particular file, or directory can be specified, and can include a wildcard character at the end, e.g., "myfiles*". If no specifc file or directory is specified all artifact files will be destroyed.

Example Use

$ paperspace jobs artifactsDestroy --jobId "j123abc" --files "myfiles*"

Properties

Name

Type

Attributes

Description

jobId

string

The id of the job to destroy artifacts for

files

string

<optional>

Optional; if destroying only certain files, a wildcard pattern to match against, e.g., "myfiles*". Note: if you include a wildcard you must double-quote the files argument.