Soup CLI

Soup CLI dramatically reduces VRAM requirements for training large language models (LLMs). Key features include: • Train LLMs exceeding GPU memory capacity • Layer-by-layer weight streaming from CPU RAM • Support for SFT, DPO, GRPO, KTO training methods • Single YAML configuration and command-line execution • Comprehensive evaluation, gating, and export functionalities Soup keeps the base model frozen and streams it into the GPU one decoder layer at a time, allowing for significantly lower peak VRAM usage. This innovation means that only a single layer of the model resides in VRAM at any given moment, rather than the entire model. For instance, it enables training of Llama-3.1-8B on a 4 GB RTX 3050 Laptop GPU at 119.6 tokens/second with just 3.32 GB peak VRAM. Beyond efficient resource management, Soup CLI offers a full suite of tools for the entire model lifecycle. Users can build, expect, train, x-ray, merge, bisect, and ship models using a unified command-line interface. It includes functionalities for preference losses like DPO and KTO, utilizing the same streamed base for reference models without extra VRAM cost. The platform supports nine architectures, including Llama, Qwen, Mistral, Gemma, and Phi, with features like gradient accumulation, run resume, and VRAM pre-flight checks. This tool is ideal for developers, researchers, and engineers working with large models on resource-constrained hardware, such as laptops or workstations with limited GPU memory. It democratizes access to advanced model training, making it feasible without needing to rent expensive cloud GPUs or acquire high-end hardware. All performance numbers are published and transparent, ensuring reliability.