Unsloth
Dramatically speeds up LoRA and QLoRA fine-tuning by rewriting GPU kernels. Compatible with HuggingFace and works with Llama, Mistral, Gemma, and more. No accuracy loss.
LlamaFactory
Supports full fine-tuning, LoRA, QLoRA, DPO, RLHF, and reward modeling across 100+ models. Web UI (LlamaBoard) for no-code training. The most feature-complete OSS fine-tuning framework.