Axolotl

Axolotl has become one of the go-to open-source frameworks for LLM fine-tuning, and its feature list backs that reputation up: full fine-tuning, LoRA and QLoRA variants, and a full suite of preference-optimization methods (DPO, ORPO, KTO, GRPO) alongside performance features like Flash Attention, sample packing, sequence parallelism, and multi-GPU training via FSDP and DeepSpeed. Reducing configuration to a single YAML file with ready-made recipes lowers the barrier for common fine-tuning jobs considerably compared to writing custom training scripts, and broad compatibility with Hugging Face Transformers models, including multimodal and audio variants, means it covers far more ground than niche fine-tuning tools. The claimed 3-5x speed advantage over alternatives is credible given its kernel-level optimizations, though real-world gains vary by hardware and model. Being fully self-hosted with no managed cloud service is a double-edged sword: it keeps the project free with 12,000+ GitHub stars behind it, but users still need their own GPU infrastructure or a provider like RunPod or Lambda Labs, and troubleshooting distributed training setups still requires real ML engineering know-how. For teams comfortable with that, it's hard to beat on capability per dollar since there's no cost at all.