Perplexity Open Source Lily Local Inference Server
Perplexity has open-sourced Lily, a lightweight Metal inference server for Apple Silicon. The current implementation only supports MLX 4-bit weights for Qwen3.6-35B-A3B and provides a simplified OpenAI Chat Completions compatible interface.
Perplexity has open-sourced the Lily local inference server, aiming to run specific Qwen models on newer Apple Silicon devices.
The current version only supports MLX affine 4-bit weights for Qwen3.6-35B-A3B, provides a simplified OpenAI Chat Completions interface, and uses greedy decoding. The project requires an Apple GPU of M5 or later, macOS 26, and Rust 1.92.
Lily is not a general-purpose model execution framework: it explicitly does not support GGUF, AWQ, GPTQ, int8, fp8, tool calls, streaming responses, or multimodal inputs. Its strict boundaries make it more suitable as an experimental engineering implementation for specific model and hardware combinations.
refer to:Perplexity official code repository