Dataster Documentation

Bulk Add Prompts

Prompts are central to Dataster, allowing users to test their applications against specific use cases rather than relying on benchmarks with generic data. In this guide, we will walk you through the process of adding prompts to your catalog in bulk and optionally specifying a ground truth for automated evaluation purposes.

Prerequisites

1. A Dataster account

Step 1: Navigate to the Catalog

1. Navigate to the prompt catalog. The prompt catalog is also the landing page after signing in.
2. Dataster bootstraps new accounts with an initial user prompt and a system prompt.

Step 2: Add Prompts

1. Click Bulk Add.
2. Use the file picker to select a JSON Line (JSONL) file from your local computer.
3. Ensure the JSONL file contains the required keys: "name", "useCase", "category", and "content". The "groundTruth" key is optional. Any other keys will be ignored.
4. Click Add.

{"name":"Magic","useCase":"nba","category":"user","content":"Who is Magic Johnson?"}
{"name":"MVP#1","useCase":"nba","category":"user","content":"Who was the first MVP?"}

Step 3: Check the upload status

1. In case of an error, the upload process moves on to the next line in the file.
2. At the end of the upload process, the number of successes and failures are displayed.

Step 4: View/Use the Prompts

1. Return to the Prompt Catalog.
2. The prompts appears in the order they were created.
3. Optionally reorder or filter the prompts.
4. Send the prompts to the playground for testing.

Conclusion

You have successfully bulk created new prompts in Dataster. This is the recommended way to create a full use case that you can eventually test against a broad variety of language models and Retrieval-Augmented Generation (RAG) systems.

If you encounter any issues or need further assistance, please contact our support team at support@dataster.com.