FreeLLMAPI - An open-source AI model aggregation platform that unifies OpenAI-compatible formats.

What is FreeLLMAPI?

FreeLLMAPI is an open-source, OpenAI-compatible proxy service that aggregates the free credits from approximately 14 AI service providers into a single API endpoint, totaling approximately [amount missing] per month. 1.3 billion tokens Free inference credits. The project is built on TypeScript/Express and supports automatic failover, intelligent routing, key encryption, and usage tracking. It is designed for local experimentation and prototyping by individual developers.

Main functions of FreeLLMAPI

• OpenAI compatible API proxyExternal Exposure Standards /v1/chat/completions and /v1/models The endpoint is compatible with official OpenAI SDK, LangChain, LlamaIndex, and other clients, requiring only modifications. base_url It can be connected immediately.
• Aggregation of free quotas from multiple providersIt simultaneously integrates with approximately 14 platforms, including Google Gemini, Groq, Cerebras, SambaNova, NVIDIA NIM, Mistral, OpenRouter, GitHub Models, HuggingFace, Cohere, Cloudflare, Z.ai, Moonshot Kimi, and MiniMax, totaling approximately [amount missing] per month. 1.3 billion tokens Free reasoning quota.
• Automatic failoverWhen the preferred provider returns a 429/5xx error or the request times out, it is automatically put into a cooling-off state, and the next provider is tried according to the preset priority chain, with a maximum of 20 retries.
• Smart Rate and Quota TrackingFor each (平台, 模型, 密钥) The system combines real-time tracking of RPM, RPD, TPM, and TPD usage to ensure that the health key is always used within limits.
• Sticky Session LockMulti-turn dialogues lock onto the same model within 30 minutes to prevent mid-process model switching from causing a jump in output quality or context breakage.
• AES-256-GCM key encryptionThe API keys of upstream providers are statically encrypted in the SQLite database and only temporarily decrypted in memory before a request is made, reducing the risk of key leakage.
• Unified Authentication Token ManagementThe client uses a single freellmapi-… Bearer Token access proxy eliminates the need to expose the real keys of multiple upstream entities in the code.
• SSE Streaming OutputFull support stream: true Server-Sent Events are streamed and compatible with scenarios where long text is generated word by word.
• Function CallingSupports OpenAI style tools / tool_choice Multi-round tool call process to meet Agent development needs.
• Visual dashboards and PlaygroundIt features a built-in React/Vite management backend, supporting drag-and-drop adjustment of backlink priority, quick model testing in the Playground, and viewing 24h/7d/30d statistics such as latency, success rate, and token usage.
• Lightweight local deploymentBuilt on TypeScript/Express, with only about 40MB of RSS feeds when not in use, it can run resident on ARM devices such as Raspberry Pi via PM2/systemd.

How to use FreeLLMAPI

• Cloning installation:git clone https://github.com/tashfeenahmed/freellmapi.git && cd freellmapi && npm install.
• Configuration key:cp .env.example .env And generate 32 bytes ENCRYPTION_KEY Used for AES-256-GCM encryption.
• Start service:npm run dev Simultaneously start the backend API (:3001) and dashboard (:5173).
• Add upstream key:exist http://localhost:5173 of Keys Paste the free API Keys for each platform onto the page.
• Adjust the backlink:exist Fallback Chain Page drag-and-drop settings allow you to prioritize providers.
• Get a unified tokenGenerate in the panel freellmapi-… A single Bearer Token.
• SDK call:Revise base_url="http://localhost:3001/v1",model Set as "auto" Or the specific model name.
• cURL call:Towards http://localhost:3001/v1/chat/completions Send a request in the standard OpenAI format.
• Playground VerificationTest connectivity and routing by directly entering prompts in the Playground dashboard.
• Permanently located in the backgroundIt can run long-term on ARM devices such as Raspberry Pi via PM2 or systemd hosting.

The core advantages of FreeLLMAPI

• OpenAI Standard CompatibleExposure to the outside world /v1/chat/completions and /v1/models The endpoint supports any compatible client, including the official OpenAI SDK, LangChain, and LlamaIndex, requiring only modifications. base_url It can be connected immediately.
• Aggregate free quotas from 14 platformsSimultaneously integrated with Google Gemini, Groq, Cerebras, SambaNova, NVIDIA NIM, Mistral, OpenRouter, GitHub Models, HuggingFace, Cohere, Cloudflare, Z.ai, Moonshot Kimi, MiniMax, etc., totaling approximately [amount missing] per month. 1.3 billion tokens Free reasoning quota.
• Automatic failoverWhen the preferred provider returns a 429/5xx error or times out, it is automatically put on cooldown and the next provider is tried in the priority chain, with a maximum of 20 retries to ensure service continuity.
• Smart Rate and Quota TrackingFor each (平台, 模型, 密钥) The system combines real-time tracking of RPM, RPD, TPM, and TPD usage, always selecting a health key that is within limits to avoid manual switching.
• sticky conversation mechanismMulti-turn dialogues lock onto the same model within 30 minutes to prevent mid-process model switching from causing a jump in output quality or context breakage.
• AES-256-GCM key encryptionThe upstream provider's API key is statically encrypted in SQLite and only temporarily decrypted in memory before a request is made. It is then destroyed after use, reducing the risk of key leakage.
• Unified Authentication TokenThe client only needs to use a single freellmapi-… Bearer Token simplifies management by eliminating the need to expose the real keys of multiple upstream suppliers in the code.
• Complete streaming and tool callsSupports SSE streaming output (stream: true) and OpenAI style tools / tool_choice Multi-round tool call process.

FreeLLMAPI project address

• GitHub repository:https://github.com/tashfeenahmed/freellmapi

Comparison of FreeLLMAPI with similar products

Application scenarios of FreeLLMAPI

• Local experimentation and prototype verification for individual developersThe project is specifically designed for personal learning and prototyping, combining multiple free quotas into a single local OpenAI endpoint, allowing users to quickly validate AI application ideas without paying.
• Multi-platform free quota aggregation and callingIt also integrates with the free tiers of about 14 platforms, including Gemini, Groq, Cerebras, GitHub Models, and Zhipu, with a total of about 1.3 billion tokens per month, making it suitable for personal projects that require large usage but have limited budgets.
• Low-power device permanent agentWith an idle RSS of only about 40MB, it can run long-term on ARM devices such as Raspberry Pi via PM2/systemd, serving as a lightweight AI gateway for home networks or edge nodes.
• Learning OpenAI Compatible InterfacesExternal Exposure Standards /v1/chat/completions Endpoints are suitable for learning or debugging ecosystem tools such as OpenAI SDK, LangChain, and LlamaIndex, requiring only modifications. base_url You can switch immediately.
• Rapid comparison test of multiple modelsThrough a unified interface and Playground dashboard, you can quickly compare the response quality, latency, and output style of different free providers in the same environment.