Tag
#llm
26 open-source projects filed under this tag.
aisuite
Andrew Ng's aisuite is a unified Python library for building with multiple LLM providers — one API for OpenAI, Anthropic, Google, Mistral, and more, plus an Agents API with toolkits and MCP support.
bifrost
Bifrost is a high-performance Go AI gateway unifying 23+ LLM providers behind one OpenAI-compatible API with semantic caching, MCP support, and sub-15µs overhead.
context7
Context7 is an MCP-powered documentation platform that feeds version-specific library docs directly into your AI coding assistant — no more hallucinated APIs.
crabtrap
Open-source HTTP proxy by Brex that uses LLM-as-a-judge to secure AI agents in production. Static rules + AI policy evaluation with full audit trail.
ds4
DwarfStar is antirez's standalone DeepSeek V4 inference engine — run a 284B MoE model locally on your Mac with Metal or CUDA, with an integrated coding agent.
forge
Python framework that makes self-hosted LLM tool-calling reliable — guardrails boost 8B models from 5% to 84% accuracy on agentic tasks.
freellmapi
FreeLLMAPI stacks free tiers from 16 LLM providers behind one OpenAI-compatible endpoint — ~1.7B tokens/month with smart routing and automatic failover.
goose
Open source AI agent by Block with 48k+ stars — Rust-based desktop, CLI, and API for code, workflows, and automation with MCP support and 70+ extensions.
headroom
Headroom is a context compression layer for AI agents that reduces token usage by 60-95% while preserving answer quality. Library, proxy, MCP server.
improve
Shadcn's agent skill audits your codebase with an expensive model and writes execution plans for cheaper AI models — 5K stars in its first week.
kvarn
KVarN is a vLLM KV-cache quantization backend from Huawei that delivers 3-5x more context capacity with FP16-level accuracy — one flag, no calibration.
lathe
Lathe is a Go CLI that generates hands-on technical tutorials using LLMs with a local UI, designed for developers who learn by doing rather than watching AI code for them.
llm-wiki
LLM Wiki is a cross-platform desktop app that turns your documents into a persistent, interlinked knowledge base — built on Karpathy's llm-wiki pattern with Tauri, React, and sigma.js.
llmtrim
A Rust-based local proxy that compresses LLM API requests in real-time, cutting costs by 66% without changing your AI tool's answers. Works with Claude Code, Codex, Cursor, and more.
lmcache
LMCache is a KV cache management layer that slashes LLM inference latency by reusing computed attention states across requests, sessions, and GPU clusters.
mastra
Mastra is a TypeScript framework for building AI agents and workflows — from the team behind Gatsby, backed by Y Combinator W25.
mnemo
Mnemo is a local-first AI memory sidecar built in Rust — persistent knowledge graph, entity extraction, and semantic retrieval for any LLM, zero cloud dependency.
opensquilla
Token-efficient microkernel AI agent with local model router, persistent memory, layered sandbox, and support for 20+ LLM providers — same budget, higher intelligence density.
openui
OpenUI is an open standard for generative UI — a streaming-first language and React runtime that lets LLMs render structured interfaces with 67% fewer tokens than JSON.
skillopt
SkillOpt by Microsoft trains reusable natural-language skills for frozen LLM agents using trajectory-driven edits and validation-gated updates — no fine-tuning required.
system-prompts-leaks
Extracted system prompts from 50+ AI models and tools including Claude, ChatGPT, Gemini, Codex, Cursor, and Copilot. 43K+ stars on GitHub.
tencentdb-agent-memory
TencentDB Agent Memory gives AI agents layered long-term memory with Mermaid symbolic compression, cutting token usage by 61% and boosting task success by 51%.
tensorzero
TensorZero is an open-source Rust LLMOps platform unifying LLM gateway, observability, evaluation, and optimization with sub-millisecond latency overhead.
tokenspeed
TokenSpeed is an open-source LLM inference engine built for agentic workloads — 580 tokens/sec on Qwen3.5-397B with TensorRT-LLM performance and vLLM-level ease of use.
toon
TOON is a compact, human-readable JSON encoding that cuts LLM token usage by ~40% while improving accuracy. A must-know data format for AI-powered apps.
whichllm
Find the best local LLM for your hardware with real benchmark rankings. Auto-detect GPU, CPU, and RAM to get evidence-based model recommendations from HuggingFace.