Merge Two Fine-Tuned LLMs Into One — No GPU Needed: Model Merging with mergekit

Combine two independently fine-tuned LLMs into one model that inherits both skill sets — no GPU, no training data, no gradient descent. Just weight arithmetic in a YAML config.

What you'll be able to do after this:

• Merge two fine-tunes of the same base model into one that's strong at both tasks
• Pick the right method: SLERP (two models), TIES or DARE (three or more)
• Run the entire merge on free Colab CPU and push the result to HuggingFace Hub

Model merging works because fine-tuning from the same base model produces models whose weights differ by small "task vectors." mergekit arithmetically combines those vectors — so a coding fine-tune and a writing fine-tune can coexist in one set of weights. The catch: they need to share the same base model and architecture.

Install

pip install mergekit

SLERP — two models

SLERP (Spherical Linear Interpolation) interpolates weights along the sphere surface rather than averaging. It's the most popular method for two-model merges.

# merge_slerp.yaml
models:
  - model: meta-llama/Llama-3.1-8B-Instruct
  - model: your-org/coding-llama-3.1-8b
merge_method: slerp
base_model: meta-llama/Llama-3.1-8B-Instruct
parameters:
  t: 0.5       # 0.0 = first model only, 1.0 = second model only
dtype: bfloat16

mergekit-yaml merge_slerp.yaml ./merged-model --copy-tokenizer

TIES — three or more models

TIES (Trim, Elect Sign, Merge) reduces interference between task vectors: it prunes low-magnitude parameters, resolves sign conflicts, then merges. Better than SLERP when combining three or more fine-tunes.

# merge_ties.yaml
merge_method: ties
base_model: meta-llama/Llama-3.1-8B-Instruct
models:
  - model: your-org/coding-8b
    parameters:
      weight: 0.5
  - model: your-org/writing-8b
    parameters:
      weight: 0.5
parameters:
  normalize: true
  density: 0.7    # keep top 70% of task-vector parameters
dtype: bfloat16

DARE — aggressive sparsity

DARE (Drop And REscale) randomly drops task-vector parameters and rescales the survivors. It reduces interference more aggressively than TIES. Use when three or more fine-tunes are interfering with each other.

merge_method: dare_ties
# same structure as TIES above; set density: 0.5 for stronger pruning

Push to Hub

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("./merged-model")
tokenizer = AutoTokenizer.from_pretrained("./merged-model")
model.push_to_hub("your-org/merged-model")
tokenizer.push_to_hub("your-org/merged-model")

No local machine? Use LazyMergekit

Maxime Labonne's LazyMergekit Colab notebook wraps this entire workflow — drop in your YAML config, click Run All, and it outputs directly to HuggingFace Hub from a free CPU runtime. No setup, no local disk needed.

When merging beats another training run: if you already have two fine-tunes of the same base model and want to combine their capabilities, merging takes 5 minutes and costs nothing. The tradeoff: always evaluate the merged model on both tasks — merging can degrade one skill. If evals show degradation, adjust the t value (SLERP) or density/weight (TIES) and re-merge.

Sources: Merge Large Language Models with mergekit — Maxime Labonne, HuggingFace Blog · LazyMergekit Colab Notebook · arcee-ai/mergekit — GitHub