ds4

DwarfStar 4 (ds4) is a specialized C inference engine designed to run advanced large models like DeepSeek V4/V4.1 Flash, GLM 5.x, and Qwen3.8 Flash Next directly on local hardware. Key features include: • Supports DeepSeek V4/V4.1 Flash, GLM 5.x, and Qwen3.8 Flash Next • Utilizes asymmetric quantization for efficient local execution • Persistent KV cache survives restarts, leveraging SSD storage • Provides a command-line interface (CLI) and local APIs compatible with OpenAI/Anthropic standards • Includes a native coding agent that shares the model state This engine brings the power of frontier open-weight models to high-memory Mac, CUDA, and ROCm machines. By compressing the models without sacrificing critical paths through targeted quantization, ds4 makes these powerful systems practical for local deployment. Users can expect robust performance, exemplified by 34.4 tokens/second generation on an M5 Max 128GB. The tool is ideal for developers, researchers, and technical users who require direct, local control over high-capability language models without relying on remote servers. It enables privacy-focused development, experimentation, and application building with powerful text and vision models. The MIT license ensures flexibility for various projects.