Skip to main content

6. Work with Training Data

Download Data​

You can download a dataset from the Gallery by clicking on the Download button on a specific dataset.

Dataset Zoo

Once a dataset is local, you can click on the Preview Button to see the first few entries.

Dataset Zoo

Upload your Own Data​

Go to Training Data -> Local Datasets -> New + in order to upload your own data.

Dataset Zoo

After giving your dataset a unique name, upload your data files using drag and drop.

Uploading training data to Transformer Lab is very simple. Simply create two files in either JSONL or JSON format.

  • The training data must end with "_train.jsonl"
  • The evaluation data must end with "_eval.jsonl"

For example, you could create two files called summarization-task_eval.jsonl and summarization-task_train.jsonl

If using JSONL, each row of your datafiles should contain a single JSON object for example:

{"text": "Lorem ipsum dolor sit samet, consectetur adipiscing elit. Suspendisse ut enim in risus cursus"}
{"text": "Vestibulum id dui feugiat, interdum lacus nec, interdum ex. Maecenas"}

Now take both files and drag them into the Transformer Lab App and click on Upload Files

Dataset Zoo

This will create a new Dataset that you can later use to train or finetine a model.