Dataset Management

Dataset management provides the capabilities of managing dataset resources such as create, delete, edit datasets and of permission-control so that datasets can be accessed only by specific groups accordingly.

Creating New Datasets

Click Add to add a Dataset and it will pop up the edit screen of Datasets.

You need to fill in these fields:

• Name (required): Only lowercase letters, numbers, dash - and the bottom line _ can be filled in.

• Display name

• Description

• Mount Root This field is not editable. It displays the path to datasets.

• Launch Group Only If enabled, the dataset is only visible to users who select the specific groups.

• Global If enabled, everyone can use this dataset. If disabled, linking groups with permission by connecting existed Groups is required.

• Type git, env and pv.

• connect existed Groups Set accessible groups, when Global is disabled.

There are three type:

git

Fill the URL of git repo (can be https or git). You can use #branch to specify the branch or tag name.

Click Change to select a secret from the list if a pull-secret is required.

env

If dataset is an environment variable, not a file, you can use env type. Please fill the key and value in the Variables. If there are more than one variable, you can click + Add field to add more field.

pv

You can specify the volume size, once it is confirmed, there is a fixed-size volume created and the volume size is not able to be changed via Admin Dashboard.

Click Confirm to complete the addition.

Upload Server

There is a new feature, Upload Server, introduced in type pv dataset from v2.0 above. A user can upload files to a type pv dataset volume via the upload server.

When editing a created type pv dataset, Enable Upload Server toggle and Regenerate Secret button appear.

Toggle Enable Upload Server on, and click Confirm. There is a pop-up showing the credential (Username / Password) for the uploader access.

Since the credential shows once only, you must keep it in a memo before clicking OK.

Note: if credential is lost you can go back to dataset editing page and click Regenerate Secret button again to have a new pair of credential.

You can see a Link created in Upload Server field of the dataset.

Clicking the link, and input the credential you keep in the memo.

After login the upload server, it displays a file list and click Upload Data.

A upload dialogue appears.

Dragging files for uploading, files are not uploaded yet at this moment.

Click Upload n files to trigger the upload. It shows Complete at bottom of the uploader once finished.

Close the uploader and back to the file list, uploaded files are listed.

You can find these uploaded files in the dataset volume which is mounted on the hub in your jupyter notebook. Currently these files can be removed only via notebook.

Deleting Datasets

Click Delete in the Datasets list, the confirmation dialog will pop up, and the Dataset will be deleted when you click OK.

Click Edit to enter the edit page of the Dataset.

If Global is disabled, please click connect existed Groups under the edit Dataset page to set accessible groups that have permission to use the Dataset.