mlx-serve

mlx-serve is a native inference server optimized for Apple Silicon, designed to run any large language model locally on your Mac without the need for Python or additional environments. Key features include compatibility with MLX and GGUF models, a drop-in replacement for popular tools like Ollama and LM Studio, and support for full OpenAI and Anthropic APIs through a single port. It delivers over 35% faster decoding performance compared to competitors on comparable hardware. This all-in-one server empowers users to go beyond chat with functionalities such as local image generation (using FLUX.2 and Krea-2), video creation with audio support (via LTX-Video), voice cloning (featuring Qwen3-TTS and ECAPA-TDNN), and an isolated Linux VM agent sandbox using macOS' Virtualization.framework. The free macOS menu-bar app makes it simple to manage and interact with the server. Ideal for developers, content creators, and researchers working on Apple Silicon Macs, mlx-serve enables secure, private AI-powered applications without relying on cloud services or external dependencies. It supports a wide range of models including Gemma 4, Qwen 3.6, Llama 3, and more, making it a versatile platform for chatbots, media generation, voice applications, and agent-based tools.