Compare commits

..

1 Commits

Author SHA1 Message Date
dab31fff82 Add Relay to Frameworks/Servers for Serving 2026-10-25 02:49:55 +03:00

View File

@ -159,6 +159,7 @@ An awesome & curated list of the best LLMOps tools for developers.
| -------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------- |
| [BentoML](https://github.com/bentoml/BentoML) | The Unified Model Serving Framework | ![GitHub Badge](https://img.shields.io/github/stars/bentoml/BentoML.svg?style=flat-square) |
| [Bifrost](https://github.com/maximhq/bifrost) | High-performance AI gateway written in Go. Unifies 23+ providers behind one OpenAI-compatible endpoint with automatic failover, adaptive load balancing, semantic caching, virtual keys and budgets, MCP tool calling, and native Prometheus observability. | ![GitHub Badge](https://img.shields.io/github/stars/maximhq/bifrost.svg?style=flat-square) |
| [Relay](https://github.com/caelum0x/railyard/tree/main/apps/relay) | Apache-2.0 maintained fork of TensorZero (archived June 2026). Same `gateway` binary, `tensorzero.toml` config and `/inference`/`/openai/v1/*` endpoints, plus per-key budgets, a price catalog, fallback routing, an operator UI, and prepaid success-only billing with AITP-402 pay-per-call challenges. | ![GitHub Badge](https://img.shields.io/github/stars/caelum0x/railyard.svg?style=flat-square) |
| [Jina](https://github.com/jina-ai/jina) | Build multimodal AI services via cloud native technologies · Model Serving · Generative AI · Neural Search · Cloud Native | ![GitHub Badge](https://img.shields.io/github/stars/jina-ai/jina.svg?style=flat-square) |
| [Mosec](https://github.com/mosecorg/mosec) | A machine learning model serving framework with dynamic batching and pipelined stages, provides an easy-to-use Python interface. | ![GitHub Badge](https://img.shields.io/github/stars/mosecorg/mosec?style=flat-square) |
| [mcpproxy-go](https://github.com/smart-mcp-proxy/mcpproxy-go) | Open-source MCP proxy with BM25 tool filtering, quarantine security, activity logging, and web UI. Routes multiple MCP servers through single endpoint, reducing context bloat by ~97%. | ![GitHub Badge](https://img.shields.io/github/stars/smart-mcp-proxy/mcpproxy-go.svg?style=flat-square) |