No Training Data? Build a Synthetic Fine-Tuning Dataset with distilabel in 30 Minutes

Chris Harper

3 min read

Aug 6, 2026 · 04:18 UTC

Tutorial

Fine-Tuning

HuggingFace

distilabel pipelines a teacher LLM to generate instruction-response pairs at scale, scores them with an AI judge, and pushes the result to HuggingFace Hub — so you can fine-tune before you've written a single training example by hand.

What you'll be able to do after this:

Generate hundreds of domain-specific instruction-response pairs using a free teacher LLM (HuggingFace Inference Endpoints or Ollama) — no manual labeling required
Score every generated response with an LLM judge (UltraFeedback) before committing to a GPU run
Push the dataset to HuggingFace Hub in ShareGPT format and load it directly into Unsloth or TRL

Every fine-tuning tutorial skips the hard part: where do the training examples come from? For domain-specific use cases — a model that speaks your API's error messages, understands your schema, or follows your coding conventions — you need hundreds of (instruction, response) pairs in the right format, and hand-labeling at scale is expensive. distilabel uses a large teacher model to generate the pairs, and a judge model to filter out the weak ones.

Install

pip install distilabel "distilabel[hf-inference-endpoints]"

Build a pipeline

from distilabel.pipeline import Pipeline
from distilabel.steps import LoadDataFromHub
from distilabel.steps.tasks import TextGeneration, UltraFeedback
from distilabel.llms import InferenceEndpointsLLM

with Pipeline(name="synthetic-domain-dataset") as pipeline:
    # Seed prompts — your domain questions, one per row
    load_data = LoadDataFromHub(
        name="load_instructions",
        repo_id="argilla/distilabel-math-preference-dpo",  # swap for your seed prompts
        num_examples=200,
    )

    # Teacher: generate 2 responses per instruction
    generate = TextGeneration(
        name="generate_responses",
        llm=InferenceEndpointsLLM(
            model_id="meta-llama/Meta-Llama-3.1-70B-Instruct",
        ),
        num_generations=2,
    )

    # Judge: score each response 1-10
    judge = UltraFeedback(
        name="judge_responses",
        llm=InferenceEndpointsLLM(
            model_id="meta-llama/Meta-Llama-3.1-70B-Instruct",
        ),
        aspect="overall-rating",
    )

    load_data >> generate >> judge

distiset = pipeline.run(use_cache=True)
distiset.push_to_hub("your-org/my-domain-dataset")

use_cache=True means interrupted runs resume from the last completed step — no wasted API calls.

Filter on quality score

The UltraFeedback step gives each response a score (1–10). Filter before fine-tuning:

from datasets import load_dataset

dataset = load_dataset("your-org/my-domain-dataset", split="train")
filtered = dataset.filter(lambda x: x["rating"] is not None and x["rating"] >= 6)
print(f"Kept {len(filtered)} of {len(dataset)} examples")

Connect to Unsloth

The output is in ShareGPT/ChatML format. Unsloth's standardize_sharegpt normalizes it:

from unsloth import FastLanguageModel, standardize_sharegpt
from trl import SFTTrainer

dataset = load_dataset("your-org/my-domain-dataset", split="train")
dataset = standardize_sharegpt(dataset)
# ... pass to SFTTrainer as usual

Swap to Ollama for local generation

Running locally with no API costs:

from distilabel.llms import OllamaLLM

generate = TextGeneration(
    llm=OllamaLLM(model="llama3.1:70b"),
    num_generations=2,
)