Skip to content
llms.py logo llms.py

llms.py

Support for Text, Image and Audio generation. Seamlessly mix and match local models with premium cloud LLMs
Get Started View on GitHub

Quick Start

Get started with llms.py in seconds

Terminal
$ pip install llms-py
$ llms --init
$ llms --enable openrouter_free google_free groq
$ llms "Explain quantum computing in simple terms"

OpenRouter but Local 🎯

Section titled “OpenRouter but Local 🎯”

llms.py is designed as a unified gateway that seamlessly connects you to multiple LLM providers through a single, consistent interface. Whether using cloud APIs or local models, llms provides intelligent routing and automatic failover to ensure your AI workflows connect to your chosen providers in your preferred priority - whether optimizing for cost, performance or availability.

Key Features

Section titled “Key Features”

Ultra-Lightweight

Single file with just one dependency (aiohttp). No setup required - just download and use.

Multi-Provider Gateway

Route requests across 160+ models from OpenAI, Anthropic, Google, Grok, Groq, Ollama and more.

Intelligent Routing

Automatic failover and cost optimization. Define free/cheap/local providers first to minimize costs.

Multi-Modal Support

Text, vision, audio processing and file attachments through vision and audio-capable models.

ChatGPT-like UI

Simple, fast, offline UI with dark mode, analytics, and all data stored locally in your browser.

OpenAI Compatible

Works with any OpenAI-compatible client or framework. Drop-in replacement for OpenAI API.

Why llms.py?

Section titled “Why llms.py?”
  1. Simplicity: One file, one dependency, infinite possibilities
  2. Flexibility: Works with any OpenAI-compatible client or framework
  3. Reliability: Automatic failover ensures your workflows never break
  4. Economy: Intelligent routing minimizes API costs
  5. Privacy: Mix local and cloud models based on your data sensitivity
  6. Future-Proof: Easily add new providers as they emerge

llms.py transforms the complexity of managing multiple LLM providers into a simple, unified experience. Whether you’re researching capabilities of new models, building the next breakthrough AI application or just want reliable access to the best models available, llms.py has you covered.

Section titled “Links”