---

name: llamafactory description: LlamaFactory training framework for fine-tuning LLMs. Use when training language models with SFT, DPO, KTO, RM, preparing datasets, writing training configs, or debugging training issues like OOM errors. Covers text, multimodal (image/video/audio), and preference learning tasks.

LlamaFactory Training Guide

Quick Start

Installation (AutoDL/Linux)

# Create environment
conda create -n llamaf python=3.9
conda activate llamaf

# Clone and install
git clone --depth 1 https://github.com/hiyouga/LLaMA-Factory.git
cd LLaMA-Factory
pip install -e ".[torch,metrics]"

# If rust error occurs:
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source $HOME/.cargo/env
pip install -e ".[torch,metrics]"

Download Models (ModelScope)

pip install modelscope
modelscope download --model Qwen/Qwen2.5-VL-7B-Instruct --local_dir /autodl-fs/data/qwen25_VL_7B
modelscope download --model Qwen/Qwen3-VL-8B-Instruct --local_dir /autodl-fs/data/qwen3_VL_8B

Run Training

llamafactory-cli train config.yaml
# Or with overrides:
CUDA_VISIBLE_DEVICES=0 llamafactory-cli train config.yaml learning_rate=1e-5

---

Dataset Formats

dataset_info.json Registration

All datasets must be registered in dataset_info.json in your dataset_dir. Example:

{
  "my_sft_dataset": {
    "file_name": "sft_data.json"
  },
  "my_dpo_dataset": {
    "file_name": "dpo_data.json",
    "ranking": true,
    "formatting": "sharegpt",
    "columns": {
      "messages": "conversations",
      "chosen": "chosen",
      "rejected": "rejected"
    }
  }
}

Alpaca Format (Default)

SFT Dataset

[
  {
    "instruction": "User instruction (required)",
    "input": "Additional input (optional)",
    "output": "Model response (required)",
    "system": "System prompt (optional)",
    "history": [
      ["Round 1 instruction", "Round 1 response"],
      ["Round 2 instruction", "Round 2 response"]
    ]
  }
]

Registration:

"dataset_name": {
  "file_name": "data.json"
}

Pretrain Dataset

[
  {"text": "document content"},
  {"text": "document content"}
]

Registration:

"dataset_name": {
  "file_name": "data.json",
  "columns": {"prompt": "text"}
}

DPO/Preference Dataset (Alpaca)

[
  {
    "instruction": "User instruction",
    "input": "Optional input",
    "chosen": "Better response",
    "rejected": "Worse response"
  }
]

Registration:

"dataset_name": {
  "file_name": "data.json",
  "ranking": true,
  "columns": {
    "prompt": "instruction",
    "query": "input",
    "chosen": "chosen",
    "rejected": "rejected"
  }
}

ShareGPT Format

SFT Dataset

[
  {
    "conversations": [
      {"from": "human", "value": "User message"},
      {"from": "gpt", "value": "Assistant response"},
      {"from": "human", "value": "Follow-up"},
      {"from": "gpt", "value": "Response"}
    ],
    "system": "System prompt (optional)",
    "tools": "Tool description (optional)"
  }
]

Registration:

"dataset_name": {
  "file_name": "data.json",
  "formatting": "sharegpt",
  "columns": {"messages": "conversations"}
}

DPO Dataset (ShareGPT)

[
  {
    "conversations": [
      {"from": "human", "value": "User question"},
      {"from": "gpt", "value": "Context response"},
      {"from": "human", "value": "Final question"}
    ],
    "chosen": {"from": "gpt", "value": "Better answer"},
    "rejected": {"from": "gpt", "value": "Worse answer"}
  }
]

Registration:

"dataset_name": {
  "file_name": "data.json",
  "formatting": "sharegpt",
  "ranking": true,
  "columns": {
    "messages": "conversations",
    "chosen": "chosen",
    "rejected": "rejected"
  }
}

KTO Dataset

[
  {
    "messages": [
      {"role": "user", "content": "Question"},
      {"role": "assistant", "content": "Response"}
    ],
    "label": true
  }
]

Registration:

"dataset_name": {
  "file_name": "data.json",
  "formatting": "sharegpt",
  "columns": {
    "messages": "messages",
    "kto_tag": "label"
  },
  "tags": {
    "role_tag": "role",
    "content_tag": "content",
    "user_tag": "user",
    "assistant_tag": "assistant"
  }
}

Multimodal Datasets

Image Dataset

[
  {
    "messages": [
      {"role": "user", "content": "<image>Describe this image"},
      {"role": "assistant", "content": "This image shows..."}
    ],
    "images": ["path/to/image1.jpg"]
  }
]

Registration:

"dataset_name": {
  "file_name": "data.json",
  "formatting": "sharegpt",
  "columns": {
    "messages": "messages",
    "images": "images"
  },
  "tags": {
    "role_tag": "role",
    "content_tag": "content",
    "user_tag": "user",
    "assistant_tag": "assistant"
  }
}

Note: Number of <image> tokens must match number of image paths.

Video Dataset

Use <video> token and videos column. Registration similar to images.

Audio Dataset

Use <audio> token and audios column.

Multimodal DPO Dataset

[
  {
    "conversations": [
      {"from": "human", "value": "<image>What's in this image?"}
    ],
    "chosen": {"from": "gpt", "value": "Detailed accurate description"},
    "rejected": {"from": "gpt", "value": "Vague or wrong description"},
    "images": ["path/to/image.jpg"]
  }
]

---

Training Configurations

Text SFT (LoRA)

### model
model_name_or_path: Qwen/Qwen3-4B-Instruct
trust_remote_code: true

### method
stage: sft
do_train: true
finetuning_type: lora
lora_rank: 8
lora_alpha: 16
lora_target: all

### dataset
dataset_dir: /path/to/data
dataset: my_dataset
template: qwen3_nothink
cutoff_len: 2048
preprocessing_num_workers: 16
dataloader_num_workers: 4

### output
output_dir: saves/qwen3-sft
logging_steps: 10
save_steps: 500
plot_loss: true
overwrite_output_dir: true
report_to: none

### train
per_device_train_batch_size: 1
gradient_accumulation_steps: 8
learning_rate: 1.0e-4
num_train_epochs: 3.0
lr_scheduler_type: cosine
warmup_ratio: 0.1
bf16: true
ddp_timeout: 180000000

### memory optimization
gradient_checkpointing: true
optim: adamw_torch

Multimodal SFT (VLM)

### model
model_name_or_path: Qwen/Qwen3-VL-8B-Instruct
image_max_pixels: 262144
video_max_pixels: 16384
trust_remote_code: true

### method
stage: sft
do_train: true
finetuning_type: lora
lora_rank: 8
lora_target: all

### dataset
dataset_dir: /path/to/data
dataset: my_vlm_dataset
template: qwen3_vl_nothink
cutoff_len: 2048

### (other params same as text SFT)

DPO Training

### model
model_name_or_path: Qwen/Qwen3-VL-8B-Instruct
image_max_pixels: 131072
video_max_pixels: 8192
trust_remote_code: true

### method
stage: dpo
do_train: true
finetuning_type: lora
lora_rank: 4
lora_alpha: 8
lora_target: all
pref_beta: 0.1
pref_loss: sigmoid  # choices: sigmoid (dpo), orpo, simpo

### dataset
dataset_dir: /path/to/data
dataset: my_dpo_dataset
template: qwen3_vl_nothink
cutoff_len: 2048

### train
per_device_train_batch_size: 1
gradient_accumulation_steps: 16
learning_rate: 5.0e-5  # DPO uses lower LR than SFT
num_train_epochs: 2.0

KTO Training

### method
stage: kto
do_train: true
finetuning_type: lora
lora_rank: 8
lora_target: all
pref_beta: 0.1

### dataset
dataset: kto_dataset
template: qwen3_nothink

### train
learning_rate: 5.0e-6

Reward Modeling

### method
stage: rm
do_train: true
finetuning_type: lora

### dataset (uses preference dataset)
dataset: dpo_dataset

Full Fine-Tuning

### method
stage: sft
do_train: true
finetuning_type: full
deepspeed: examples/deepspeed/ds_z3_config.json

### train
per_device_train_batch_size: 1
gradient_accumulation_steps: 2
learning_rate: 1.0e-5  # Lower than LoRA

---

Key Parameters Reference

Model Parameters

| Parameter | Description | |-----------|-------------| | model_name_or_path | HuggingFace model ID or local path | | trust_remote_code | Allow custom model code (required for Qwen) | | image_max_pixels | Max pixels per image (VLM) | | video_max_pixels | Max pixels per video frame (VLM) |

Method Parameters

| Parameter | Description | |-----------|-------------| | stage | Training stage: pt, sft, rm, dpo, kto, ppo | | finetuning_type | full, lora, freeze | | lora_rank | LoRA rank (4-64, higher = more params) | | lora_alpha | LoRA scaling factor (typically 2x rank) | | lora_target | Target modules: all or specific names | | pref_beta | DPO/KTO beta parameter (0.1-0.5) | | pref_loss | sigmoid (DPO), orpo, simpo |

Dataset Parameters

| Parameter | Description | |-----------|-------------| | dataset_dir | Directory containing dataset_info.json | | dataset | Dataset name(s) from dataset_info.json | | template | Chat template matching model | | cutoff_len | Max sequence length | | max_samples | Limit samples (for testing) |

Training Parameters

| Parameter | Description | |-----------|-------------| | per_device_train_batch_size | Batch size per GPU | | gradient_accumulation_steps | Steps before update | | learning_rate | SFT: 1e-4, DPO: 5e-5~5e-6 | | num_train_epochs | Training epochs | | lr_scheduler_type | cosine, linear, constant | | warmup_ratio | Warmup proportion (0.1) | | bf16 | Use bfloat16 precision | | gradient_checkpointing | Reduce memory usage | | optim | Optimizer: adamw_torch (recommended) |

Common Templates

| Model | Template | |-------|----------| | Qwen3 | qwen3 / qwen3_nothink | | Qwen3-VL | qwen3_vl / qwen3_vl_nothink | | Qwen2.5-VL | qwen2_vl | | Llama3 | llama3 | | DeepSeek | deepseek / deepseekr1 |

Note: Use _nothink suffix when you don't want reasoning/thinking tokens.

---

Troubleshooting

OOM (Out of Memory)

Solutions in order of effectiveness:

1. Reduce batch size: per_device_train_batch_size: 1
2. Enable gradient checkpointing: gradient_checkpointing: true
3. Reduce LoRA rank: lora_rank: 4 instead of 8
4. Reduce sequence length: cutoff_len: 1024
5. Reduce image resolution (VLM): image_max_pixels: 131072
6. Use QLoRA: Add quantization_bit: 4
7. Use DeepSpeed ZeRO-2/3: deepspeed: examples/deepspeed/ds_z2_config.json

Dataset Issues

"Dataset not found":

• Verify dataset_info.json is in dataset_dir
• Check dataset name matches exactly

"Column not found":

• Verify column mappings in dataset_info.json
• Check actual JSON field names

"Empty dataset":

• Verify JSON syntax (trailing commas, encoding)
• Check file permissions

Multimodal image not found:

• Use relative paths from dataset_dir
• Or use absolute paths in dataset

Training Issues

Loss is NaN:

• Reduce learning rate
• Check data for invalid values
• Enable bf16: true instead of fp16

Optimizer errors:

• Use optim: adamw_torch instead of adamw_8bit
• Check available GPU memory

Template mismatch:

• Ensure template matches model type
• Use _nothink for non-reasoning models

---

Useful Commands

# Start WebUI
llamafactory-cli webui

# Chat with model
llamafactory-cli chat config.yaml

# API server
llamafactory-cli api config.yaml

# Merge LoRA
llamafactory-cli export merge_config.yaml

# Multi-GPU training
CUDA_VISIBLE_DEVICES=0,1 FORCE_TORCHRUN=1 llamafactory-cli train config.yaml

---

Additional Resources

• For detailed dataset format examples, see reference.md
• For complete training config templates, see templates.md