Compare commits

..

146 Commits

Author SHA1 Message Date
Aleh Kruhlikau
ac203d898a
Add subagent-tuner to Optimizations (#941)
Signed-off-by: Aleh Kruhlikau <79532594+Kruhlikau@users.noreply.github.com>
2026-10-16 14:34:42 +08:00
Arman
93110fd46a
Add TheYgent to LLMOps (#944)
Signed-off-by: al2m4n <al2m4n@gmail.com>
2026-10-16 14:34:33 +08:00
Symbioza
8a1ffb9369
Add Symbioza (#888)
Signed-off-by: moshe-edri <326504156+moshe-edri@users.noreply.github.com>
Co-authored-by: moshe-edri <326504156+moshe-edri@users.noreply.github.com>
2026-10-16 14:34:24 +08:00
samovermind
6743d8b148
Add Overmind (#951)
Signed-off-by: Sam Morris <sammorris@overmindlab.ai>
2026-10-16 14:34:15 +08:00
shaun
c64e0b9b88
Add OpenConnector (#939)
Signed-off-by: shaun <beingswu@gmail.com>
2026-10-16 14:34:05 +08:00
art-mandarelabs
39453ee24f
Add Mandare to LLMOps (#962)
Signed-off-by: Mandare Labs <arthur@mandarelabs.com>
2026-10-16 14:33:56 +08:00
Emrick Donadei
daceaf6819
Add Caliper (#950)
Signed-off-by: Emrick Donadei <emrick.donadei@gmail.com>
2026-10-16 14:33:47 +08:00
Maksym Mykhailenko
890f00a13c
Add App AI Gateway to LLMOps (#961)
Signed-off-by: Maksym Mykhailenko <maxceem@gmail.com>
2026-10-16 14:33:38 +08:00
Roli Bosch
ca189b0ae7
Add agent-convergence-scorer to LLMOps tools (#848)
* Add agent-convergence-scorer

Signed-off-by: roli-lpci <rbosch@lpci.ai>

* DCO Remediation Commit for Rolando Bosch <rbosch@lpci.ai>

I, Rolando Bosch <rbosch@lpci.ai>, hereby add my Signed-off-by to this commit: 14d40cf490acdb5906a4aa328f341280643d913d

Signed-off-by: Rolando Bosch <rbosch@lpci.ai>

* DCO Remediation Commit for roli-lpci <rbosch@lpci.ai>

I, roli-lpci <rbosch@lpci.ai>, hereby add my Signed-off-by to this commit: 14d40cf490acdb5906a4aa328f341280643d913d

Signed-off-by: roli-lpci <rbosch@lpci.ai>

* Place agent-convergence-scorer alphabetically

Signed-off-by: Roli Bosch <roli@hermes-labs.ai>

* docs: use canonical agent-convergence-scorer URLs

Signed-off-by: Roli Bosch <roli@hermes-labs.ai>

---------

Signed-off-by: roli-lpci <rbosch@lpci.ai>
Signed-off-by: Rolando Bosch <rbosch@lpci.ai>
Signed-off-by: Roli Bosch <roli@hermes-labs.ai>
2026-10-16 14:33:28 +08:00
Larry & Leo
48aba93869
Add Orbi to Code AI (#895)
Signed-off-by: Lawrence Liu <smartlitchi@gmail.com>
2026-10-16 14:33:20 +08:00
Will G
50437ab54f
Add aigent-os to Code AI (#945)
Signed-off-by: Will Gwyn <wrg32786@users.noreply.github.com>
Co-authored-by: Will Gwyn <wrg32786@users.noreply.github.com>
2026-10-16 14:33:11 +08:00
Haseeb Khalid
0f56545062
Add Synaps to Code AI (#955)
Signed-off-by: Haseeb Khalid <HaseebKhalid1507@users.noreply.github.com>
Co-authored-by: Haseeb Khalid <HaseebKhalid1507@users.noreply.github.com>
2026-10-16 14:33:01 +08:00
Vijay
7ffc2422aa
Add otelcol-genai-sketches to Observability (#935)
Signed-off-by: Vijay <evijay@gmail.com>
2026-10-16 14:32:52 +08:00
Mert Kaya
5af6063cc4
Add JevOss to Observability (#923)
Signed-off-by: Mert Kaya <47145541+mertkayacs@users.noreply.github.com>
2026-10-16 14:32:43 +08:00
Seybou Siddo Yacine
09567d2486
Add RAGeval to Observability (#675)
Signed-off-by: Yacine-ai-tech <siddoyacinetech227@gmail.com>
2026-10-16 14:32:33 +08:00
Yashvardhan Thanvi
0972ad8e06
Add LLMSlim to Optimizations (#938)
Signed-off-by: Yashvardhan Thanvi <yashvt9404@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 14:32:25 +08:00
Archestack
6fe8edfdde
Add Best of Open-Source AI (#942)
Signed-off-by: Archestack <337802620+archestack@users.noreply.github.com>
Co-authored-by: Archestack <337802620+archestack@users.noreply.github.com>
2026-10-16 14:32:16 +08:00
ShakeLv
1e9f8ab2b9
Add Remio to hybrid search tools (#570)
Signed-off-by: 吕扬晃 <lvyanghuang@lvyanghuangdeMacBook-Air.local>
Co-authored-by: 吕扬晃 <lvyanghuang@lvyanghuangdeMacBook-Air.local>
2026-10-16 14:32:07 +08:00
don
98def2e0f1
Add UnoRouter to LLMOps (#562)
Signed-off-by: 0-don <diabolic1996@gmail.com>
Co-authored-by: 0-don <diabolic1996@gmail.com>
2026-10-05 16:40:56 +08:00
fylorn
b76f97669b
Add ThinkWatch Lite to LLMOps (#894)
Signed-off-by: fylorn <249551762+fylorn@users.noreply.github.com>
2026-10-05 16:40:46 +08:00
daiana-cybrient
1ee33c90dd
Add boldrouter (#870)
Signed-off-by: Daiana <276971743+daiana-cybrient@users.noreply.github.com>
Co-authored-by: Daiana <276971743+daiana-cybrient@users.noreply.github.com>
2026-10-05 16:40:37 +08:00
dk
c1908d86ff
Add Kunavo to LLMOps (#744)
Signed-off-by: cooldk <cool.zdk@gmail.com>
2026-10-05 16:40:28 +08:00
Shakes
46b929a6f0
Add Distil to LLMOps (#673)
Signed-off-by: Shekhar Mudarapu <chandu1221@gmail.com>
2026-10-05 16:40:19 +08:00
Roli Bosch
7e9ed279e4
Add lintlang to LLMOps table (#827)
* Add lintlang to LLMOps table

Signed-off-by: Rolando Bosch <rbosch@lpci.ai>

* Place lintlang in alphabetical order

Signed-off-by: Roli Bosch <roli@hermes-labs.ai>

* Move lintlang to the LLMOps category

Implemented with OpenAI Codex (gpt-6-sol).
Independently reviewed with OpenAI Codex (gpt-6.1-sol).

Signed-off-by: Rolando Bosch <roli@hermes-labs.ai>

---------

Signed-off-by: Rolando Bosch <rbosch@lpci.ai>
Signed-off-by: Roli Bosch <roli@hermes-labs.ai>
Signed-off-by: Rolando Bosch <roli@hermes-labs.ai>
2026-10-05 16:40:10 +08:00
kaushal trivedi
bd7adb1811
Add CogniCore to LLMOps (#782)
Signed-off-by: Kaushalt2004 <Kaushaltrivedi268@gmail.com>
2026-10-05 16:40:01 +08:00
Ming Wen
108ee03f12
Add AISIX to LLMOps section (#711)
Signed-off-by: Ming Wen <moonbingbing@gmail.com>
2026-10-05 16:39:52 +08:00
Pung Worathiti Manosroi
00ebeb27d7
Add Superagent to Code AI (#792)
Signed-off-by: pungme <w.manosroi@gmail.com>
2026-10-05 16:39:42 +08:00
Evan Dempsey
64d8b883ac
Add Kolega Code to Code AI (#758)
Signed-off-by: evandempsey <evandempsey@gmail.com>
2026-10-05 16:39:28 +08:00
David Farah
a5ca21ad14
Add Cotal to Code AI (#891)
Signed-off-by: David Farah <37240868+davidfarah2003@users.noreply.github.com>
2026-10-05 16:39:19 +08:00
Oğuzhan TOPCU
107b491d19
Add wardcat to LLM security tools (#847)
Added 'wardcat' project to the list of tools in the README.

Signed-off-by: Oğuzhan TOPCU <73748939+oguzhantopcu0@users.noreply.github.com>
2026-10-05 16:39:10 +08:00
crp4222
ba377ad797
Add PrivAiTe to Frameworks for LLM security (#653)
Signed-off-by: crp4222 <121295348+crp4222@users.noreply.github.com>
2026-10-05 16:39:00 +08:00
Tarun Agarwal
7358ea2cb8
Add wobbly — label-free metamorphic testing for LLM extraction (#720)
Placed after the witness row to keep the section alphabetical.
Rebased on current main; diff limited to the single wobbly entry.

Signed-off-by: tarunagarwal1981 <tarunagarwal1981@users.noreply.github.com>
Co-authored-by: tarunagarwal1981 <tarunagarwal1981@users.noreply.github.com>
2026-10-05 16:38:51 +08:00
ParallelIQ
dcc4fec542
Add piqc to ML Platforms (#564)
* Add Paralleliq and piqc to MLOps tools list

Added Paralleliq and piqc to the MLOps tools list in README.

Signed-off-by: ParallelIQ <sam@paralleliq.ai>

* Move Paralleliq and piqc to alphabetical position in ML Platforms

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Sam <sam@paralleliq.ai>

* Keep a single entry (piqc) in this PR

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Sam <sam@paralleliq.ai>

---------

Signed-off-by: ParallelIQ <sam@paralleliq.ai>
Signed-off-by: Sam <sam@paralleliq.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 16:38:40 +08:00
domondi1
f5da4a4bb7
Add Inferrail to Observability (#882)
Signed-off-by: domondi1 <91575411+domondi1@users.noreply.github.com>
2026-10-05 16:38:30 +08:00
r0lfi
51a74547a2
Add LayerSmith to training workspaces (#886)
Add one alphabetized entry for OCI image builds, including optional LLM training environments.

Signed-off-by: r0lfi <rolf@linux.com>
2026-10-05 16:38:21 +08:00
Jason Brashear
49ad6546f2
Add Jebadiah decision models (#887)
Signed-off-by: webdevtodayjason <jason@webdevtoday.com>
2026-10-05 16:38:12 +08:00
nikolovv861
15b2ce95aa
Add NobodyWho (#869)
Signed-off-by: nikolovv861 <nikolovv861@users.noreply.github.com>
Co-authored-by: nikolovv861 <nikolovv861@users.noreply.github.com>
2026-10-05 16:38:02 +08:00
Lin Song
af43c1160c
Add Self-Hosted AI Stack (#876)
Signed-off-by: Lin Song <5104323+hwdsl2@users.noreply.github.com>
2026-10-02 15:11:56 +08:00
Qi-Xuan Lu
71ad13f74a
Add Wenlan (#741)
* Add Wenlan

Signed-off-by: 7xuanlu <h164654156465@gmail.com>

* Move Wenlan entry to alphabetical position

Signed-off-by: Qi-Xuan Lu <h164654156465@gmail.com>

---------

Signed-off-by: 7xuanlu <h164654156465@gmail.com>
Signed-off-by: Qi-Xuan Lu <h164654156465@gmail.com>
2026-10-02 15:11:47 +08:00
Antonio Mello
4b4032fb7d
Add CrewDock to LLMOps (#548)
Signed-off-by: Antonio <ajgcvm@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-02 15:11:38 +08:00
Roli Bosch
71211c1501
Add claude-router to LLMOps (#833)
Local prompt router that picks the right Claude model tier and
prepends the right scaffold using local embeddings, before the API
call. Deterministic routing to stop overspending on Sonnet/Opus when
Haiku plus structure is enough.

Signed-off-by: Rolando Bosch <rbosch@lpci.ai>
Signed-off-by: Roli Bosch <roli@hermes-labs.ai>
2026-10-02 15:11:28 +08:00
Linda Oraegbunam
f6e0e075c6
Add reliopt (#865)
Signed-off-by: Linda Oraegbunam <OBIELINDA@GMAIL.COM>
2026-10-02 15:11:19 +08:00
Nick Lamb
82bcf5a901
Add OpenGATE to Observability (#709)
Signed-off-by: Nick Lamb <nickjlamb@gmail.com>
2026-10-02 15:11:10 +08:00
Mertcan Sağlam
714dfb7c47
Add Shim to Frameworks for LLM security (#701)
* Add Shim to Frameworks for LLM security

Signed-off-by: Mertcan Sağlam <92371591+DurthVadr@users.noreply.github.com>

* Link shim entry to its public repository and add star badge

Signed-off-by: Mertcan Sağlam <92371591+DurthVadr@users.noreply.github.com>

---------

Signed-off-by: Mertcan Sağlam <92371591+DurthVadr@users.noreply.github.com>
2026-10-02 15:11:01 +08:00
bmw8080
c6bbb699f5
Add laya-decision-api to Frameworks/Servers for Serving (#860)
* Add laya-decision-api to Frameworks/Servers for Serving

A self-hosted HTTP server around the Laya System-1 decision model: offline structured
decisions (choice / score / yes-no) with probabilities. https://github.com/bmw8080/laya-decision-api

Signed-off-by: bmw8080 <gjh_bbs@qq.com>

* Enhance laya-decision-api entry with badge

Added GitHub stars badge to laya-decision-api entry.

Signed-off-by: bmw8080 <gjh_bbs@qq.com>

---------

Signed-off-by: bmw8080 <gjh_bbs@qq.com>
2026-10-02 15:10:52 +08:00
PRAKHAR PANDEY
125ae1a747
Add webcmd to Code AI (#816)
Signed-off-by: Prakhar Pandey <prakhar9999pandey@gmail.com>
2026-10-02 15:10:42 +08:00
zhifu gao
df828ba818
Add FunASR to Audio Foundation Model section (#533)
* Add FunASR to Audio Foundation Model section

Signed-off-by: zhifu gao <zhifu.gzf@alibaba-inc.com>

* docs: alphabetize the FunASR audio entry

Keep unrelated rows unchanged and describe capabilities without unqualified speed or language-count claims.

Signed-off-by: LauraGPT <18321252+LauraGPT@users.noreply.github.com>

---------

Signed-off-by: zhifu gao <zhifu.gzf@alibaba-inc.com>
Signed-off-by: LauraGPT <18321252+LauraGPT@users.noreply.github.com>
Co-authored-by: LauraGPT <18321252+LauraGPT@users.noreply.github.com>
2026-10-02 15:10:32 +08:00
Nero Un
69c1ea35ef
Fix Trinity table row whitespace (#878)
Signed-off-by: Nero Un <17636182+NeroHin@users.noreply.github.com>
Co-authored-by: Nero Un <17636182+NeroHin@users.noreply.github.com>
2026-10-01 22:06:33 +08:00
Mudui
b04f752bd3
docs: add XiuRouter to LLMOps tools (#779)
Signed-off-by: MuduiClaw <28682216+MuduiClaw@users.noreply.github.com>

Pre-commit-gate: fc873e0a862691fd69661fbd34c32125a60d5843

Co-authored-by: MuduiClaw <28682216+MuduiClaw@users.noreply.github.com>
2026-10-01 22:02:10 +08:00
TokenMix
8688fbd460
feat: add TokenMix to LLMOps gateway section (#550)
Signed-off-by: TokenMix <diaoyulao9657@gmail.com>
2026-10-01 22:01:47 +08:00
tokristinamay
60040d0741
Add Trinity to LLMOps (#861)
Signed-off-by: tokristinamay <161319455+tokristinamay@users.noreply.github.com>
2026-10-01 22:01:36 +08:00
Samarth Nagpal
184e2fc63c
Add Spendline (#774)
* Add Spendline

Signed-off-by: Samarth Nagpal <145949224+Samarth-254@users.noreply.github.com>

* Update README.md

Restore Statewave link text

Signed-off-by: Samarth Nagpal <145949224+Samarth-254@users.noreply.github.com>

---------

Signed-off-by: Samarth Nagpal <145949224+Samarth-254@users.noreply.github.com>
2026-10-01 22:01:24 +08:00
data-gras
9d970ef7f1
Add SAVI SDK (#820)
Signed-off-by: data-gras <kekalsolutions@gmail.com>
2026-10-01 22:01:13 +08:00
Trevor Hudson
d1183dc906
Add rote (#712)
Signed-off-by: Trevor Hudson <trevhud@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-10-01 22:01:02 +08:00
Alex Li
a2dcb064fe
Add Quotaflow to LLMOps tools (#496)
Signed-off-by: Alex Li <alex@crowd-core.com>
Co-authored-by: Alex Li <alex@crowd-core.com>
2026-10-01 22:00:51 +08:00
philip638
726652b995
Add Pilot Protocol to LLMOps section (#633)
Signed-off-by: philip638 <philip638@users.noreply.github.com>
Co-authored-by: philip638 <philip638@users.noreply.github.com>
2026-10-01 22:00:40 +08:00
xizhuomengcontin
3570b5ef4f
Add OrcaRouter Lite to LLMOps section (#772)
Self-hosted, OpenAI-compatible LLM router (MIT). Placed alphabetically
between Opik and Parea AI, with a star badge in the existing format.

Signed-off-by: xizhuomengcontin <meng.xizhuo@continuum01.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-01 22:00:25 +08:00
underwater research
c4d9c81676
Add Nora to LLMOps (#654)
Signed-off-by: underwater research <solomon2773@yahoo.com>
2026-10-01 22:00:11 +08:00
auxiliar-ag
1240a90c3b
feat: add NativePort to LLMOps section (#721)
Add NativePort (https://nativeport.ai/), infrastructure for unified access to web-search, scraping, browser-automation, voice, and model-inference providers via one account/API key/balance, with dated capability-specific leaderboards and a public methodology.

Signed-off-by: auxiliar-ag <301726194+auxiliar-ag@users.noreply.github.com>
Co-authored-by: auxiliar-ag <301726194+auxiliar-ag@users.noreply.github.com>
2026-10-01 21:59:57 +08:00
Scott Schinkel
7a5236995b
Add Modelglass to LLMOps (#806)
Signed-off-by: Scott Schinkel <schinkel.scott@gmail.com>
2026-10-01 21:59:46 +08:00
Abaho Katabarwa
5d37632ab6
Add LLMGraph (#646)
Signed-off-by: Abaho Katabarwa <abahomyster@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-10-01 21:59:32 +08:00
Mohsen Seyedkazemi Ardebili
c8f51231dc
Add KubeIntellect to LLMOps (#817)
KubeIntellect is an open-source, LLM-orchestrated multi-agent framework for
Kubernetes operations. A coordinator fans out to subagents that query kubectl,
Prometheus (PromQL) and Loki (LogQL), correlates the evidence into a root cause,
and gates every mutating operation behind a human approval check with
role-based access control.

AGPL-3.0. The v1 architecture is peer-reviewed in the Journal of Grid Computing
(https://doi.org/10.1007/s10723-026-09837-6).

Signed-off-by: Mohsen Seyedkazemi Ardebili <mohsen.seyedkazemi@gmail.com>
2026-10-01 21:59:17 +08:00
Jakub A. Wąsek
316980a90a
Add GoModel to LLMOps section (#663)
Signed-off-by: Jakub A. W <jakubwasek@gmail.com>
2026-10-01 21:59:03 +08:00
Tashfeen
8f045aceb2
feat: add FreeLLMAPI to LLMOps (#697)
Signed-off-by: Tashfeen <tashfene@gmail.com>
2026-10-01 21:58:49 +08:00
江欢
67caa7ef93
Add FerryAPI to LLMOps tools (#543)
Signed-off-by: Sundry007 <jacksoul58@gmail.com>
2026-10-01 21:58:38 +08:00
ejentum.com
e7d00248fe
feat: add Ejentum cognitive-harness MCP server (#504)
Signed-off-by: Ejentum <info@ejentum.com>
2026-10-01 21:58:27 +08:00
Jaedon Munton
36da9046ff
Add Doubleword Control Layer to LLMOps (#647)
Signed-off-by: Jaedon Munton <jaedon.munton@doubleword.ai>
2026-10-01 21:58:17 +08:00
Wendel Ortiz
f31b7e470c
Add depdesk to LLMOps (#857)
Signed-off-by: Ort0x36 <wendel.ortiz.dev@gmail.com>
2026-10-01 21:58:07 +08:00
Nick C.
3364597ba5
Add Cortex to LLMOps (#854)
Signed-off-by: Nick Chisiu <8492343+nickchisiu@users.noreply.github.com>
Co-authored-by: Nick Chisiu <8492343+nickchisiu@users.noreply.github.com>
2026-10-01 21:57:53 +08:00
Sahil Kadadekar
b3cc4441b2
Add Chimeraforge to LLMOps (#759)
Signed-off-by: Sahil170595 <147995121+Sahil170595@users.noreply.github.com>
2026-10-01 21:57:30 +08:00
Jyolsna
6fd076a297
Add Caura to LLMOps (#804)
Governed shared memory for AI agent fleets (MCP-native, Apache 2.0).


Claude-Session: https://claude.ai/code/session_01GpTzfTEGgqdBtqNCMa63Pe

Signed-off-by: jy7lsna <jyolsnajoemon@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-10-01 21:57:16 +08:00
Spikel
b5309d11c9
Add BitRouter to LLMOps (#560)
Signed-off-by: Spikel <shangweiliu66@gmail.com>
Co-authored-by: Spikel <shangweiliu66@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-10-01 21:57:02 +08:00
Joseph Magly
1b0377c295
Add AIWG (#793)
Signed-off-by: Joseph Magly <1159087+jmagly@users.noreply.github.com>
2026-10-01 21:56:51 +08:00
Siva Prakash K
e9dbb32752
Add Agnos to LLMOps (#784)
Agnos is a gateway-agnostic, self-hosted, MIT LLM control plane: an
OpenAI-compatible governance proxy that runs LiteLLM/Bifrost/Portkey as a
swappable, stateless engine, keeping provider keys in your own control plane.
Placed alphabetically in the LLMOps table.

Signed-off-by: Agnos Proxy Contributors <siva010928@users.noreply.github.com>
Co-authored-by: Agnos Proxy Contributors <siva010928@users.noreply.github.com>
2026-10-01 21:56:33 +08:00
Anzal Husain Abidi
d290f09bd6
Add witness to Observability (#822)
Signed-off-by: Anzal Husain Abidi <anzalabidi@gmail.com>
2026-10-01 21:56:23 +08:00
TOKENPOLICE
a59d08388a
Add Token Police to Observability (#849)
Signed-off-by: Thasli <thasli@tokenpolice.ai>
2026-10-01 21:56:07 +08:00
Rehan Mohammed
9a6a6ffccc
Add runtape to Observability (#875)
Signed-off-by: Rehan Mohammed <112909229+RehanMohammed985@users.noreply.github.com>
2026-10-01 21:55:44 +08:00
Mikhail Golikov
443800c097
Add phoenix2pytest to Observability (#514)
Signed-off-by: Mikhail Golikov <golikovichev@gmail.com>
2026-10-01 21:55:30 +08:00
shashank agarwal
6c11c9ad73
Add Noveum to Observability (#863)
Signed-off-by: shashank agarwal <shashank@shankee.com>
2026-10-01 21:55:19 +08:00
Jonathan Martín
a064c018d7
Add Lookspan to Observability (#546)
* Add Lookspan to Observability

Signed-off-by: Jonathan Martín <jonathanmartinpaez@gmail.com>

* Point Lookspan entry to its project homepage

Signed-off-by: Jonathan Martín <jonathanmartinpaez@gmail.com>

---------

Signed-off-by: Jonathan Martín <jonathanmartinpaez@gmail.com>
2026-10-01 21:55:05 +08:00
Roli Bosch
edffb92f79
Add hermes-rubric to Observability section (#726)
* Add hermes-rubric to Observability section

Signed-off-by: Roli Bosch <rolandjbosch@gmail.com>

* docs: correct hermes-rubric metadata

Signed-off-by: Roli Bosch <rolandjbosch@gmail.com>

---------

Signed-off-by: Roli Bosch <rolandjbosch@gmail.com>
2026-10-01 21:54:55 +08:00
mcouderc-flowlines
a1dc23b641
Add Flowlines to observability tools (#745)
Signed-off-by: Marin Couderc <marin@flowlines.ai>
2026-10-01 21:54:41 +08:00
Alessandro Prandini
19f40f44e0
Add digline (#790)
Added 'digline' entry for regression testing of LLM apps.

Signed-off-by: Alessandro Prandini <alexpran@gmail.com>
2026-10-01 21:54:15 +08:00
Resham Joshi
79026abac5
Add CodeBurn to Observability (#862)
Signed-off-by: AgentSeal <hello@agentseal.org>
2026-10-01 21:54:04 +08:00
kimbeomgyu
6366b98543
feat: add budget-guard to Observability (#717)
Signed-off-by: kimbeomgyu <qjarb1994@gmail.com>
2026-10-01 21:53:44 +08:00
Kushagra
60f0a3b8e4
Add AgentCost to Observability (#619)
Signed-off-by: Kushagra  <kushagraagrawal128@gmail.com>
Signed-off-by: Kushagra Agrawal <kushagraagrawal128@gmail.com>
2026-10-01 21:53:19 +08:00
Ilya Sibiryakov
eae34a6f91
feat(security): add PrivacyScrubber client-side PII sanitizer (#780)
Signed-off-by: Ilya Sibiryakov <ilsimox@gmail.com>
2026-10-01 21:52:53 +08:00
Neeraj Kumar Singh B
be2fde445a
Add EmbedGuard to Frameworks for LLM security (#532)
Signed-off-by: Neeraj Kumar Singh <b.nirajkumarsingh@hotmail.com>
Co-authored-by: Neeraj Kumar Singh <b.nirajkumarsingh@hotmail.com>
2026-10-01 21:52:30 +08:00
Anthony Chaudhary
24c12bec40
Add DOS Kernel to Frameworks for LLM security (#571)
Signed-off-by: Anthony Chaudhary <18080164+anthony-chaudhary@users.noreply.github.com>
2026-10-01 21:52:19 +08:00
李韭二
eb2c8280c2
Add API Relay Audit (#557)
Signed-off-by: toby-bridges <toby-bridges@users.noreply.github.com>
Co-authored-by: toby-bridges <toby-bridges@users.noreply.github.com>
2026-10-01 21:52:09 +08:00
Tobias
fa90f68442
Add lilbee to Serving (#627)
Signed-off-by: Tobias Perelstein <5562156+tobocop2@users.noreply.github.com>
2026-10-01 21:51:58 +08:00
Jonathan Reyes
0e431295d6
Add Higgs to Large Model Serving (#791)
Signed-off-by: Jonathan Reyes <me@jonathanreyes.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-01 21:51:47 +08:00
Shai Shmuel
563752c144
Add SIE (#823)
Signed-off-by: Shai Shmuel <shai@superlinked.com>
2026-10-01 21:51:36 +08:00
nehadangwal
5b26cde2ac
Add SentryNode Gateway to README (#580)
Added SentryNode Gateway to the list of AI inference governance frameworks.

Signed-off-by: nehadangwal <dangwalneha2013@gmail.com>
2026-10-01 21:51:27 +08:00
Swapnoneel Saha
4f068dc69d
Add Bifrost to serving tools (#728)
Signed-off-by: Swapnoneel Saha <swapnoneelsaha111@gmail.com>
2026-10-01 21:51:13 +08:00
Imamuzzaki Abu Salam
cc23cd03db
feat: add Relay to Code AI section (#536)
Signed-off-by: Imamuzzaki Abu Salam <hi@imbios.dev>
2026-10-01 21:50:54 +08:00
Leeroo Coding Agent
cfbe39e35d
Add Kapso to Code AI (#798)
Kapso by Leeroo: a self-improving AI software factory for measurable objectives. https://github.com/Leeroo-AI/kapso

Signed-off-by: Alireza Mohammadshahi <alireza@leeroo.com>
Co-authored-by: Alireza Mohammadshahi <alireza@leeroo.com>
2026-10-01 21:50:40 +08:00
Ryan Pettry
ca826c89ad
Add fractal to Code AI (#692)
Signed-off-by: Ryan Pettry <rpettry@plasma.ai>
Co-authored-by: Ryan Pettry <rpettry@plasma.ai>
2026-10-01 21:50:30 +08:00
Moresyl
22fae633e4
Add DSH Studio to Code AI (#752)
* Add DSH Studio to Code AI

Add the DeepSeek Harness desktop host in alphabetical order with the repository's star badge format.

Signed-off-by: Moresyl <85490079+Moresyl@users.noreply.github.com>

* Restore full README content

Restore the complete upstream README and retain only the intended DSH Studio entry.

Signed-off-by: Moresyl <85490079+Moresyl@users.noreply.github.com>

---------

Signed-off-by: Moresyl <85490079+Moresyl@users.noreply.github.com>
2026-10-01 21:50:19 +08:00
uipreliga
4446c8a40e
Add Coder Eval to Code AI (#704)
Framework for evaluating, benchmarking, and A/B-testing AI coding agents
and their skills with sandboxed, reproducible YAML task suites.

Signed-off-by: uipreliga <tomasz.religa@uipath.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-01 21:50:09 +08:00
Dudka
2fd92c2165
Add Atomic Agent to Code AI (#742)
Signed-off-by: Надя Родионова <sosidudku@MacBook-Air-7.local>
Co-authored-by: sosidudku1 <273119990+sosidudku1@users.noreply.github.com>
2026-10-01 21:49:57 +08:00
Himanshu Kurrey
fcc893d8c8
Add TrainJudge (#872)
Signed-off-by: Himanshu Kurrey <133849470+Himanshukurrey@users.noreply.github.com>
2026-10-01 21:49:47 +08:00
Luis M
44a37b093e
Add SynapCores to Vector search (#540)
Signed-off-by: Luis Mata <mataluis2k@gmail.com>
2026-10-01 21:49:37 +08:00
Murali Krishna P
72cb4b0fc9
Add Infino to Vector search (#808)
Signed-off-by: Murali Krishna <9718411+muralikpbhat@users.noreply.github.com>
Co-authored-by: Murali Krishna <9718411+muralikpbhat@users.noreply.github.com>
2026-10-01 21:49:26 +08:00
mdhaseeb343q-pixel
e64bd25b11
Add MemorySync to Hybrid search (#843)
Add MemorySync persistent memory layer and MCP server for AI coding assistants with sub-50ms hybrid recall.

Signed-off-by: mdhaseeb343q-pixel <mdhaseeb343q@gmail.com>
2026-10-01 21:49:09 +08:00
Eran Raichstein
84b2781ab9
Add cap-evolve (agent prompt/tool/skill optimizer) (#696)
* Add cap-evolve (agent prompt/tool/skill optimizer) to Optimizations

Signed-off-by: Eran Raichstein <eran.raichstein@gmail.com>

* Add cap-evolve (agent prompt/tool/skill optimizer) to Optimizations

Signed-off-by: Eran Raichstein <eran.raichstein@gmail.com>

* Add cap-evolve (agent prompt/tool/skill optimizer) to Optimizations

Signed-off-by: Eran Raichstein <eran.raichstein@gmail.com>

---------

Signed-off-by: Eran Raichstein <eran.raichstein@gmail.com>
2026-10-01 21:48:48 +08:00
TinyOps Studio LLC
5cb993e543
Add AI agent token cost calculator (#765)
Signed-off-by: TinyOps Studio LLC <support@tinyopsstudio.com>
2026-10-01 21:48:37 +08:00
Burak Akgün
ede5d01c28
Add Heym (#642)
Signed-off-by: mbakgun <burak@mbakgun.com>
2026-10-01 21:48:26 +08:00
Adewole Babatunde
0334a9090c
Add awaithumans to LLMOps > Workflow (#501)
awaithumans is the human-in-the-loop primitive for AI agent workflows.
One function call (await_human / awaitHuman) lets an agent block like
any Promise; a human reviews via Slack, email, or a web dashboard;
the agent resumes with the typed Pydantic / Zod response. Durable
across agent restarts via Stripe-style idempotency keys. Adapters
for Temporal and LangGraph included. Apache 2.0.

Repo: https://github.com/awaithumans/awaithumans

Signed-off-by: TA <adewole63@gmail.com>
2026-10-01 21:48:16 +08:00
Pranjal Parmar
ca10438893
Add fastdocparse to Data/Feature enrichment (#781)
Signed-off-by: Pranjal Parmar <76609992+pranjalparmar@users.noreply.github.com>
2026-10-01 21:48:06 +08:00
LJ 🥥🌴
a4e69ee17b
Add CocoIndex to Data/Feature enrichment (#652)
Signed-off-by: LJ 🥥🌴 <linghua@cocoindex.io>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-01 21:47:56 +08:00
nac7
45739975b3
Add ragfresh to Data Tracking (#770)
Signed-off-by: nac7 <29607512+nac7@users.noreply.github.com>
Co-authored-by: nac7 <29607512+nac7@users.noreply.github.com>
2026-10-01 21:47:46 +08:00
Pierre Brunelle
21f6f4cb69
Add Pixeltable under Data Storage (#835)
Signed-off-by: Pierre Brunelle <pierre@noteable.io>
2026-10-01 21:47:35 +08:00
Franz Hiltscher
7711c65da1
Add Awesome AI Coding Sandboxes to Awesome Lists (#676)
Signed-off-by: Franz Hiltscher <franz.hiltscher@realtech.com>
Co-authored-by: Franz Hiltscher <franz.hiltscher@realtech.com>
2026-10-01 21:46:17 +08:00
chernistry
4fbf8d456e
Add Bernstein (#436)
Signed-off-by: Alex Chernysh <sanderchernitsky@gmail.com>
Signed-off-by: chernistry <sanderchernitsky@gmail.com>
Co-authored-by: sipyourdrink-ltd <hey@sipyourdrink.ltd>
2026-05-21 17:12:50 +08:00
SaiGanesh Menon
689f171cd7
Add Off Grid to Serving section (#441) 2026-05-21 13:23:15 +08:00
Timothy @aden
448ecd17ab
Add Hive to LLMOps section (#289)
Signed-off-by: Timothy <timothy@adenhq.com>
2026-05-21 13:23:11 +08:00
Yaron
51c997e2f9
Add Cordum to Frameworks for LLM security (#252)
Co-authored-by: Ce Gao <gaocegege@hotmail.com>
2026-05-21 13:21:47 +08:00
Wayde
cc821de695
Add SwarmClaw (#379)
Co-authored-by: Ce Gao <gaocegege@hotmail.com>
2026-05-21 13:20:58 +08:00
Christopher Maher
6d26e026df
Add LLMKube to inference section (#277)
* Add LLMKube to inference section

Signed-off-by: Christopher Maher <chris@mahercode.io>

* fix: correct alphabetical placement and update LLMKube description

- Move LLMKube entry to correct alphabetical position (after llama.cpp)
- Update description to reflect pluggable runtime backends and supported model formats

Signed-off-by: Christopher Maher <chris@mahercode.io>

---------

Signed-off-by: Christopher Maher <chris@mahercode.io>
2026-05-21 13:20:02 +08:00
Hidai Bar-Mor
2a5af0f50a
feat: add EvalView to evaluation/observability tools (#316) 2026-05-21 13:19:59 +08:00
Ali Baizhanov
629b053a6c
Add Mengram to LLMOps (#378)
Signed-off-by: a.baizhanov <alibaizhanovbcc@gmail.com>
2026-05-21 13:19:55 +08:00
Statewave
55bb6b8686
Add Statewave to LLMOps section (#480)
Signed-off-by: smaramwbc <145447586+smaramwbc@users.noreply.github.com>
2026-05-21 13:19:52 +08:00
Dumbris
6b2d59229e
Add mcpproxy-go (#235) 2026-05-21 13:19:49 +08:00
Aaron Goh
d765825e89
Add Omnigraph (#411) 2026-05-21 13:19:45 +08:00
Yuma Ichikawa
0dbc933758
Add OneComp row to Large Model Serving table (#404) 2026-05-21 13:19:42 +08:00
Jupiter-Guy
30491d70fa
Add Entroly Context Optimization Proxy to Optimizations (#485) 2026-05-21 13:19:38 +08:00
om_jambhale
698dcbfa4b
Add Contexto - self-hosted context engine for AI agents (#280)
Signed-off-by: om_jambhale <omjambhale0014@gmail.com>
2026-05-21 13:19:30 +08:00
Prakersh Maheshwari
dd4d07b12d
Add onWatch to Observability section (#296)
onWatch is an open-source Go CLI that tracks AI API quota usage
across 7 providers. Lightweight background daemon with <50MB RAM,
zero telemetry, and SQLite storage.

https://github.com/onllm-dev/onwatch
2026-05-21 13:19:27 +08:00
Santosh kumar
819181b52c
Add AgentField to LLMOps (#368) 2026-05-21 13:19:23 +08:00
Yves Gugger
e16319122e
docs: add lean-ctx to Optimizations section (#435)
Signed-off-by: Yves Gugger <yvgude@users.noreply.github.com>
Co-authored-by: Yves Gugger <yvgude@users.noreply.github.com>
2026-05-21 13:19:19 +08:00
yishuiliunian
d7ce2378ab
Add AgentsMesh to Code AI (#390) 2026-05-21 13:19:16 +08:00
Naveen RS
bd4b77c772
Add Robotics Foundation Model section with VLA models (#481)
* docs: add Robotics Foundation Model section with VLA models

Signed-off-by: naveentnj <naveentnj@users.noreply.github.com>

* docs: add notes on recurrent and continuous diffusion architectures for VLA

Signed-off-by: naveentnj <naveentnj@users.noreply.github.com>

---------

Signed-off-by: naveentnj <naveentnj@users.noreply.github.com>
Co-authored-by: naveentnj <naveentnj@users.noreply.github.com>
2026-05-21 12:31:30 +08:00
Suhani Nagpal
92d852dbbe
docs:Add future-agi by Future AGI (#467)
Co-authored-by: Suhani Nagpal <suhani.nagpal725@gmail.com>
2026-05-21 12:31:27 +08:00
Andy Anderson
5e080e4023
feat: add KubeStellar Console to Large Scale Deployment (#419)
* feat: add KubeStellar Console to Large Scale Deployment section

KubeStellar Console provides multi-cluster Kubernetes management for
LLM workloads with GPU monitoring, benchmark streaming, and AI-guided
cluster operations. CNCF Sandbox project.

https://console.kubestellar.io
https://github.com/kubestellar/console
Signed-off-by: Andy Anderson <andy@clubanderson.com>

* feat: add KubeStellar Console to Frameworks/Servers for Serving section

KubeStellar Console is an AI-powered multi-cluster Kubernetes dashboard
for hybrid edge and cloud with GPU monitoring, LLM inference cluster
management, benchmark streaming, and 20+ CNCF integrations. CNCF Sandbox
project (Apache 2.0).

https://github.com/kubestellar/console
https://console.kubestellar.io
Signed-off-by: Andy Anderson <andy@clubanderson.com>

---------

Signed-off-by: Andy Anderson <andy@clubanderson.com>
2026-05-21 12:31:24 +08:00
Zhengyao Jiang
c796da7ee8
Add Weco Observe and AIDE to the list (#329)
- Add Weco Observe to the Observability section: AI agent observability tool for tracing multi-step LLM agent runs and visualizing decision trees
- Add AIDE to the Code AI section: open-source ML engineering agent using tree search for automated ML experimentation (arxiv:2502.13138)

Co-authored-by: Zhengyao Jiang <zhengyao.jiang.13@ucl.ac.uk>
2026-05-21 12:31:21 +08:00
Juan Antonio Osorio
b0bc66e54c
Add brood-box to Frameworks for LLM security (#305)
Signed-off-by: Juan Antonio Osorio <ozz@stacklok.com>
Co-authored-by: Ce Gao <gaocegege@hotmail.com>
2026-05-21 12:31:12 +08:00
Jordi Mas
9404e0cda3
Add whispser-ctranslate2 (#245)
Signed-off-by: Jordi Mas <jmas@softcatala.org>
2026-05-21 12:29:58 +08:00
Nicolai Bohn (Rhesis AI)
df4b2106c1
Update README.md (#238)
Add Rhesis to LLMOps tools list

Added Rhesis AI as an open-source testing infrastructure platform for LLM and agentic applications, positioned alphabetically after Registry Broker.

Signed-off-by: Nicolai Bohn (Rhesis AI) <168477896+nicolai-rhesis@users.noreply.github.com>
2026-05-21 12:29:54 +08:00
Hang Yin
9278cfb82d
Add dstack to LLM security frameworks (#206) 2026-05-21 12:29:51 +08:00
Rauf Akdemir
cd5daf9d48
Add Airweave to Hybrid search (#195)
Signed-off-by: Rauf Akdemir <rauf@neena.io>
2026-05-21 12:29:48 +08:00
Pei-Wen Tang (Vincent)
710991845b
Add Semantic Cache Router to LLMOps section (#357)
* Add Semantic Cache Router to LLMOps section

Signed-off-by: Peiwen Tang <redjackfred@gmail.com>

* fix: move Semantic Cache Router to after Portkey (alphabetical order)

Signed-off-by: Peiwen Tang <redjackfred@gmail.com>

---------

Signed-off-by: Peiwen Tang <redjackfred@gmail.com>
2026-04-06 07:07:49 +02:00
Raullen Chai
2572a07c20
Add Rapid-MLX to Large Model Serving (#310) 2026-04-06 06:42:43 +02:00
sophiaashi
c7cc1c7cee
Add TeamoRouter to LLMOps section (#328)
TeamoRouter is an LLM routing gateway that provides a single API key
to access multiple top LLMs (Claude, GPT-4o, Gemini, DeepSeek, Kimi,
MiniMax) with smart routing modes and up to 50% off official prices.

Website: https://router.teamolab.com

Co-authored-by: sophiaashi <sophiaashi@users.noreply.github.com>
2026-04-06 06:42:01 +02:00
seankwon816
46d9a9c126
Add ClevAgent to Observability section (#339)
Co-authored-by: Sean Joongho Kwon <seankwon@Seanui-Macmini.local>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 06:40:20 +02:00
Alex Strick van Linschoten
e2576d3ef5
Add Kitaru to the list of workflow tools (#360)
Signed-off-by: Alex Strick van Linschoten <strickvl@users.noreply.github.com>
2026-04-06 06:38:17 +02:00

178
README.md
View File

@ -16,6 +16,7 @@ An awesome & curated list of the best LLMOps tools for developers.
- [Large Language Model](#large-language-model)
- [CV Foundation Model](#cv-foundation-model)
- [Audio Foundation Model](#audio-foundation-model)
- [Robotics Foundation Model](#robotics-foundation-model)
- [Serving](#serving)
- [Large Model Serving](#large-model-serving)
- [Frameworks/Servers for Serving](#frameworksservers-for-serving)
@ -71,6 +72,7 @@ An awesome & curated list of the best LLMOps tools for developers.
| [ChatGLM2-6B](https://github.com/THUDM/ChatGLM2-6B) | ChatGLM2-6B is the second-generation version of the open-source bilingual (Chinese-English) chat model [ChatGLM-6B](https://github.com/THUDM/ChatGLM-6B). | ![GitHub Badge](https://img.shields.io/github/stars/THUDM/ChatGLM2-6B.svg?style=flat-square) |
| [GLM-130B (ChatGLM)](https://github.com/THUDM/GLM-130B) | An Open Bilingual Pre-Trained Model (ICLR 2023) | ![GitHub Badge](https://img.shields.io/github/stars/THUDM/GLM-130B.svg?style=flat-square) |
| [GPT-NeoX](https://github.com/EleutherAI/gpt-neox) | An implementation of model parallel autoregressive transformers on GPUs, based on the DeepSpeed library. | ![GitHub Badge](https://img.shields.io/github/stars/EleutherAI/gpt-neox.svg?style=flat-square) |
| [Jebadiah](https://github.com/getainode/jebadiah) | Open decision models that return a probability for each option instead of generating text; available in 4B, 9B and 27B sizes. | ![GitHub Badge](https://img.shields.io/github/stars/getainode/jebadiah.svg?style=flat-square) |
| [Luotuo](https://github.com/LC1332/Luotuo-Chinese-LLM) | A Chinese LLM, Based on LLaMA and fine tune by Stanford Alpaca, Alpaca LoRA, Japanese-Alpaca-LoRA. | ![GitHub Badge](https://img.shields.io/github/stars/LC1332/Luotuo-Chinese-LLM.svg?style=flat-square) |
| [Mixtral-8x7B-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-v0.1) | The Mixtral-8x7B Large Language Model (LLM) is a pretrained generative Sparse Mixture of Experts. | |
| [StableLM](https://github.com/Stability-AI/StableLM) | StableLM: Stability AI Language Models | ![GitHub Badge](https://img.shields.io/github/stars/Stability-AI/StableLM.svg?style=flat-square) |
@ -93,8 +95,28 @@ An awesome & curated list of the best LLMOps tools for developers.
| Project | Details | Repository |
| -------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- |
| [bark](https://github.com/suno-ai/bark) | Bark is a transformer-based text-to-audio model created by Suno. Bark can generate highly realistic, multilingual speech as well as other audio - including music, background noise and simple sound effects. | ![GitHub Badge](https://img.shields.io/github/stars/suno-ai/bark.svg?style=flat-square) |
| [FunASR](https://github.com/modelscope/FunASR) | Speech recognition toolkit with pretrained models and tools for voice activity detection, punctuation restoration, and speaker diarization. | ![GitHub Badge](https://img.shields.io/github/stars/modelscope/FunASR.svg?style=flat-square) |
| [whisper](https://github.com/openai/whisper) | Robust Speech Recognition via Large-Scale Weak Supervision | ![GitHub Badge](https://img.shields.io/github/stars/openai/whisper.svg?style=flat-square) |
**[⬆ back to ToC](#table-of-contents)**
### Robotics Foundation Model
> [!NOTE]
> **Emerging Architectures in VLA:**
> - **Continuous Diffusion Language Models:** Integrate diffusion heads or flow-matching to VLMs (e.g., DiVLA, OpenPI), enabling smooth, precise continuous action generation rather than discretized tokens.
> - **Recurrent Language Models:** Utilize State Space Models (SSMs) like Mamba or recurrent transformers (e.g., RoboMamba, RD-VLA) to reduce inference memory and handle temporal dependencies, allowing iterative reasoning for complex robotic decision-making.
| Project | Details | Repository |
| --------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------ |
| [DiVLA](https://github.com/hustvl/DiVLA) | A continuous diffusion-based Vision-Language-Action model that integrates diffusion policies into autoregressive VLMs for robust and precise continuous robotic control. | ![GitHub Badge](https://img.shields.io/github/stars/hustvl/DiVLA.svg?style=flat-square) |
| [LeRobot](https://github.com/huggingface/lerobot) | A central community library by Hugging Face for AI in robotics — end-to-end learning tools, data pipelines, and support for training/deploying VLA models. | ![GitHub Badge](https://img.shields.io/github/stars/huggingface/lerobot.svg?style=flat-square) |
| [Octo](https://github.com/octo-models/octo) | A transformer-based generalist robot policy pretrained on 800K+ robot trajectories from the Open X-Embodiment dataset. Supports language instructions, goal images, and fine-tuning to new embodiments. | ![GitHub Badge](https://img.shields.io/github/stars/octo-models/octo.svg?style=flat-square) |
| [OpenPI](https://github.com/Physical-Intelligence/openpi) | Open-source VLA models from Physical Intelligence, including π₀ and π₀.5 — flow-based vision-language-action models pretrained on large-scale robot data with fine-tuning support. | ![GitHub Badge](https://img.shields.io/github/stars/Physical-Intelligence/openpi.svg?style=flat-square) |
| [OpenVLA](https://github.com/openvla/openvla) | A 7B-parameter open-source Vision-Language-Action model trained on 970K+ robot demonstrations from the Open X-Embodiment dataset for generalist robotic manipulation. | ![GitHub Badge](https://img.shields.io/github/stars/openvla/openvla.svg?style=flat-square) |
| [RoboMamba](https://github.com/hustvl/RoboMamba) | An efficient VLA model leveraging State Space Models (Mamba) instead of standard self-attention, offering linear inference complexity for efficient, recurrent robotic reasoning. | ![GitHub Badge](https://img.shields.io/github/stars/hustvl/RoboMamba.svg?style=flat-square) |
| [SmolVLA](https://huggingface.co/blog/smolvla) | A compact ~450M parameter VLA by Hugging Face, designed to be computationally efficient and accessible, running on consumer GPUs or CPUs. Part of the LeRobot ecosystem. | |
## Serving
### Large Model Serving
@ -102,22 +124,30 @@ An awesome & curated list of the best LLMOps tools for developers.
| Project | Details | Repository |
| ------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- |
| [Alpaca-LoRA-Serve](https://github.com/deep-diver/Alpaca-LoRA-Serve) | Alpaca-LoRA as Chatbot service | ![GitHub Badge](https://img.shields.io/github/stars/deep-diver/Alpaca-LoRA-Serve.svg?style=flat-square) |
| [Higgs](https://github.com/panbanda/higgs) | Rust inference server for Apple Silicon: MLX models behind OpenAI and Anthropic APIs, routing to remote providers, desktop dashboard. | ![GitHub Badge](https://img.shields.io/github/stars/panbanda/higgs.svg?style=flat-square) |
| [OneComp](https://github.com/FujitsuResearch/OneCompression) | Fujitsu Research's post-training quantization pipeline for LLMs (QEP, AutoBit, JointQ, rotation) with vLLM plugin (arXiv:2603.28845). | ![GitHub Badge](https://img.shields.io/github/stars/FujitsuResearch/OneCompression.svg?style=flat-square) |
| [CTranslate2](https://github.com/OpenNMT/CTranslate2) | fast inference engine for Transformer models in C++ | ![GitHub Badge](https://img.shields.io/github/stars/OpenNMT/CTranslate2.svg?style=flat-square) |
| [Clip-as-a-service](https://github.com/jina-ai/clip-as-service) | serving the OpenAI CLIP model | ![GitHub Badge](https://img.shields.io/github/stars/jina-ai/clip-as-service.svg?style=flat-square) |
| [DeepSpeed-MII](https://github.com/microsoft/DeepSpeed-MII) | MII makes low-latency and high-throughput inference possible, powered by DeepSpeed. | ![GitHub Badge](https://img.shields.io/github/stars/microsoft/DeepSpeed-MII.svg?style=flat-square) |
| [Faster Whisper](https://github.com/guillaumekln/faster-whisper) | fast inference engine for whisper in C++ using CTranslate2. | ![GitHub Badge](https://img.shields.io/github/stars/guillaumekln/faster-whisper.svg?style=flat-square) |
| [FlexGen](https://github.com/FMInference/FlexGen) | Running large language models on a single GPU for throughput-oriented scenarios. *(Archived)* | ![GitHub Badge](https://img.shields.io/github/stars/FMInference/FlexGen.svg?style=flat-square) |
| [Flowise](https://github.com/FlowiseAI/Flowise) | Drag & drop UI to build your customized LLM flow using LangchainJS. | ![GitHub Badge](https://img.shields.io/github/stars/FlowiseAI/Flowise.svg?style=flat-square) |
| [lilbee](https://github.com/tobocop2/lilbee) | Single-binary local manager and search engine that stands up a llama-server fleet sized to your GPUs by gguf-parser for VRAM-aware multi-GPU placement | ![GitHub Badge](https://img.shields.io/github/stars/tobocop2/lilbee.svg?style=flat-square) |
| [llama.cpp](https://github.com/ggerganov/llama.cpp) | Port of Facebook's LLaMA model in C/C++ | ![GitHub Badge](https://img.shields.io/github/stars/ggerganov/llama.cpp.svg?style=flat-square) |
| [LLMKube](https://github.com/defilantech/LLMKube) | Kubernetes operator for LLM inference with pluggable runtimes (llama.cpp, PersonaPlex/Moshi, generic), multi-GPU sharding, NVIDIA CUDA and Apple Silicon Metal support, and GGUF/MLX/SafeTensors model formats. | ![GitHub Badge](https://img.shields.io/github/stars/defilantech/LLMKube.svg?style=flat-square) |
| [Shimmy](https://github.com/Michael-A-Kuykendall/shimmy) | Python-free Rust inference server with OpenAI API compatibility and hot model swapping | ![GitHub Badge](https://img.shields.io/github/stars/Michael-A-Kuykendall/shimmy.svg?style=flat-square) |
| [Infinity](https://github.com/michaelfeil/infinity) | Rest API server for serving text-embeddings | ![GitHub Badge](https://img.shields.io/github/stars/michaelfeil/infinity.svg?style=flat-square) |
| [Modelz-LLM](https://github.com/tensorchord/modelz-llm) | OpenAI compatible API for LLMs and embeddings (LLaMA, Vicuna, ChatGLM and many others) | ![GitHub Badge](https://img.shields.io/github/stars/tensorchord/modelz-llm.svg?style=flat-square) |
| [NobodyWho](https://github.com/nobodywho-ooo/nobodywho) | On-device LLM inference engine (Rust / llama.cpp) to embed local models directly in games and apps, with Godot / Flutter / React Native / Swift bindings; streaming, embeddings, tool calling, GBNF-structured output, and STT/TTS. | ![GitHub Badge](https://img.shields.io/github/stars/nobodywho-ooo/nobodywho.svg?style=flat-square) |
| [Off Grid](https://github.com/alichherawalla/off-grid-mobile-ai) | Open-source iOS/Android app running LLMs on-device via llama.cpp. Voice (Whisper), vision, image gen, tool calling — fully offline. | ![GitHub Badge](https://img.shields.io/github/stars/alichherawalla/off-grid-mobile-ai.svg?style=flat-square) |
| [Ollama](https://github.com/jmorganca/ollama) | Serve Llama 2 and other large language models locally from command line or through a browser interface. | ![GitHub Badge](https://img.shields.io/github/stars/jmorganca/ollama.svg?style=flat-square) |
| [Rapid-MLX](https://github.com/raullenchai/Rapid-MLX) | OpenAI-compatible LLM inference server for Apple Silicon using MLX. 2-4x faster than Ollama with tool calling and prompt caching. | ![GitHub Badge](https://img.shields.io/github/stars/raullenchai/Rapid-MLX.svg?style=flat-square) |
| [TensorRT-LLM](https://github.com/NVIDIA/TensorRT-LLM) | Inference engine for TensorRT on Nvidia GPUs | ![GitHub Badge](https://img.shields.io/github/stars/NVIDIA/TensorRT-LLM.svg?style=flat-square) |
| [text-generation-inference](https://github.com/huggingface/text-generation-inference) | Large Language Model Text Generation Inference | ![GitHub Badge](https://img.shields.io/github/stars/huggingface/text-generation-inference.svg?style=flat-square) |
| [text-embeddings-inference](https://github.com/huggingface/text-embeddings-inference) | Inference for text-embedding models | ![GitHub Badge](https://img.shields.io/github/stars/huggingface/text-embeddings-inference.svg?style=flat-square) |
| [tokenizers](https://github.com/huggingface/tokenizers) | 💥 Fast State-of-the-Art Tokenizers optimized for Research and Production | ![GitHub Badge](https://img.shields.io/github/stars/huggingface/tokenizers.svg?style=flat-square) |
| [vllm](https://github.com/vllm-project/vllm) | A high-throughput and memory-efficient inference and serving engine for LLMs. | ![GitHub stars](https://img.shields.io/github/stars/vllm-project/vllm.svg?style=flat-square) |
| [whisper-ctranslate2](https://github.com/Softcatala/whisper-ctranslate2) | is a 4x faster and low-memory usage drop-in cli replacement that supports word-level timestamps and VAD filter | ![GitHub Badge](https://img.shields.io/github/stars/Softcatala/whisper-cTranslate2?style=flat-square) |
| [whisper.cpp](https://github.com/ggerganov/whisper.cpp) | Port of OpenAI's Whisper model in C/C++ | ![GitHub Badge](https://img.shields.io/github/stars/ggerganov/whisper.cpp.svg?style=flat-square) |
| [x-stable-diffusion](https://github.com/stochasticai/x-stable-diffusion) | Real-time inference for Stable Diffusion - 0.88s latency. Covers AITemplate, nvFuser, TensorRT, FlashAttention. *(Archived)* | ![GitHub Badge](https://img.shields.io/github/stars/stochasticai/x-stable-diffusion.svg?style=flat-square) |
@ -128,18 +158,25 @@ An awesome & curated list of the best LLMOps tools for developers.
| Project | Details | Repository |
| -------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------- |
| [BentoML](https://github.com/bentoml/BentoML) | The Unified Model Serving Framework | ![GitHub Badge](https://img.shields.io/github/stars/bentoml/BentoML.svg?style=flat-square) |
| [Bifrost](https://github.com/maximhq/bifrost) | High-performance AI gateway written in Go. Unifies 23+ providers behind one OpenAI-compatible endpoint with automatic failover, adaptive load balancing, semantic caching, virtual keys and budgets, MCP tool calling, and native Prometheus observability. | ![GitHub Badge](https://img.shields.io/github/stars/maximhq/bifrost.svg?style=flat-square) |
| [Jina](https://github.com/jina-ai/jina) | Build multimodal AI services via cloud native technologies · Model Serving · Generative AI · Neural Search · Cloud Native | ![GitHub Badge](https://img.shields.io/github/stars/jina-ai/jina.svg?style=flat-square) |
| [Mosec](https://github.com/mosecorg/mosec) | A machine learning model serving framework with dynamic batching and pipelined stages, provides an easy-to-use Python interface. | ![GitHub Badge](https://img.shields.io/github/stars/mosecorg/mosec?style=flat-square) |
| [mcpproxy-go](https://github.com/smart-mcp-proxy/mcpproxy-go) | Open-source MCP proxy with BM25 tool filtering, quarantine security, activity logging, and web UI. Routes multiple MCP servers through single endpoint, reducing context bloat by ~97%. | ![GitHub Badge](https://img.shields.io/github/stars/smart-mcp-proxy/mcpproxy-go.svg?style=flat-square) |
| [SentryNode Gateway](https://github.com/nehadangwal/sentrynode-gateway) | Open-core AI inference governance and FinOps platform providing semantic routing, <40ms agent loop intervention, strict budget caps, and cryptographic audit trails. Includes voice pipeline governance (STT→LLM→TTS). | ![GitHub Badge](https://img.shields.io/github/stars/nehadangwal/sentrynode-gateway.svg?style=flat-square) |
| [TFServing](https://github.com/tensorflow/serving) | A flexible, high-performance serving system for machine learning models. | ![GitHub Badge](https://img.shields.io/github/stars/tensorflow/serving.svg?style=flat-square) |
| [Torchserve](https://github.com/pytorch/serve) | Serve, optimize and scale PyTorch models in production *(Archived)* | ![GitHub Badge](https://img.shields.io/github/stars/pytorch/serve.svg?style=flat-square) |
| [Triton Server (TRTIS)](https://github.com/triton-inference-server/server) | The Triton Inference Server provides an optimized cloud and edge inferencing solution. | ![GitHub Badge](https://img.shields.io/github/stars/triton-inference-server/server.svg?style=flat-square) |
| [langchain-serve](https://github.com/jina-ai/langchain-serve) | Serverless LLM apps on Production with Jina AI Cloud *(Archived)* | ![GitHub Badge](https://img.shields.io/github/stars/jina-ai/langchain-serve.svg?style=flat-square) |
| [lanarky](https://github.com/ajndkr/lanarky) | FastAPI framework to build production-grade LLM applications | ![GitHub Badge](https://img.shields.io/github/stars/ajndkr/lanarky.svg?style=flat-square) |
| [ray-llm](https://github.com/ray-project/ray-llm) | LLMs on Ray - RayLLM *(Archived)* | ![GitHub Badge](https://img.shields.io/github/stars/ray-project/ray-llm.svg?style=flat-square) |
| [SIE](https://github.com/superlinked/sie) | Open-source inference server and production cluster for serving embeddings, reranking, OCR, structured output, content safety, and generative models through one API. | ![GitHub Badge](https://img.shields.io/github/stars/superlinked/sie.svg?style=flat-square) |
| [Xinference](https://github.com/xorbitsai/inference) | Replace OpenAI GPT with another LLM in your app by changing a single line of code. Xinference gives you the freedom to use any LLM you need. With Xinference, you're empowered to run inference with any open-source language models, speech recognition models, and multimodal models, whether in the cloud, on-premises, or even on your laptop. | ![GitHub Badge](https://img.shields.io/github/stars/xorbitsai/inference.svg?style=flat-square) |
| [KubeAI](https://github.com/substratusai/kubeai) | Deploy and scale machine learning models on Kubernetes. Built for LLMs, embeddings, and speech-to-text. | ![GitHub Badge](https://img.shields.io/github/stars/substratusai/kubeai.svg?style=flat-square) |
| [Kaito](https://github.com/kaito-project/kaito) | A Kubernetes operator that simplifies serving and tuning large AI models (e.g. Falcon or phi-3) using container images and GPU auto-provisioning. Includes an OpenAI-compatible server for inference and preset configurations for popular runtimes such as vLLM and transformers. | ![GitHub Badge](https://img.shields.io/github/stars/kaito-project/kaito.svg?style=flat-square) |
| [Open Responses](https://docs.julep.ai/open-responses) | Serverless open-source platform for building long-running LLM agents with tool use. | ![GitHub Badge](https://img.shields.io/github/stars/julep-ai/julep.svg?style=flat-square) |
| [KubeStellar Console](https://github.com/kubestellar/console) | AI-powered multi-cluster Kubernetes dashboard for hybrid edge and cloud. GPU monitoring, LLM inference cluster management, benchmark streaming, and 20+ CNCF integrations. CNCF Sandbox (Apache 2.0). | ![GitHub Badge](https://img.shields.io/github/stars/kubestellar/console.svg?style=flat-square) |
| [laya-decision-api](https://github.com/bmw8080/laya-decision-api) | FastAPI HTTP service for the local Laya System-1 decision model: offline structured decisions (choice / score / yes-no) with probabilities and confidence; MLX and torch backends |[![GitHub stars](https://img.shields.io/github/stars/bmw8080/laya-decision-api?style=flat)](https://github.com/bmw8080/laya-decision-api) |
**[⬆ back to ToC](#table-of-contents)**
@ -150,7 +187,17 @@ An awesome & curated list of the best LLMOps tools for developers.
| Project | Details | Repository |
| ------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------- |
| [API Relay Audit](https://github.com/toby-bridges/api-relay-audit) | Local security audit for AI API relays and LLM proxies; checks prompt injection, model identity drift, tool-call rewriting, error leakage, SSE anomalies, and Web3 wallet probes. | ![GitHub Badge](https://img.shields.io/github/stars/toby-bridges/api-relay-audit.svg?style=flat-square) |
| [Cordum](https://github.com/cordum-io/cordum) | Safety-first agent orchestration platform with pre-dispatch policy evaluation, output scanning (PII, secrets, injection), job scheduling, workflow engine, and full audit trail. | ![GitHub Badge](https://img.shields.io/github/stars/cordum-io/cordum.svg?style=flat-square) |
| [brood-box](https://github.com/stacklok/brood-box) | CLI tool for running coding agents inside hardware-isolated microVMs with snapshot isolation, egress control, and MCP authorization. | ![GitHub Badge](https://img.shields.io/github/stars/stacklok/brood-box?style=flat-square) |
| [DOS Kernel](https://github.com/anthony-chaudhary/dos-kernel) | Deterministic trust kernel for AI-agent fleets: verifies agent "done" claims from git evidence rather than self-report, arbitrates concurrent file access between agents, and audits commit messages against their own diffs. | ![GitHub Badge](https://img.shields.io/github/stars/anthony-chaudhary/dos-kernel.svg?style=flat-square) |
| [dstack](https://github.com/Dstack-TEE/dstack) | Open-source confidential AI framework for secure LLM deployment with data privacy, providing hardware-enforced isolation using Intel TDX and NVIDIA Confidential Computing. | ![GitHub Badge](https://img.shields.io/github/stars/Dstack-TEE/dstack?style=flat-square) |
| [EmbedGuard](https://github.com/neerazz/embedguard) | Cross-layer detection and provenance attestation for adversarial embedding attacks in RAG systems. Flags poisoned or manipulated embeddings before they reach the retrieval layer. | ![GitHub Badge](https://img.shields.io/github/stars/neerazz/embedguard?style=flat-square) |
| [Plexiglass](https://github.com/kortex-labs/plexiglass) | A Python Machine Learning Pentesting Toolbox for Adversarial Attacks. Works with LLMs, DNNs, and other machine learning algorithms. | ![GitHub Badge](https://img.shields.io/github/stars/kortex-labs/plexiglass?style=flat-square) |
| [PrivacyScrubber](https://github.com/moxno/privacyscrubber-mcp) | Zero-Trust client-side PII and developer secrets sanitizer and MCP server for Cursor, Windsurf, and Claude Desktop with sub-2ms in-memory tokenization. | ![GitHub Badge](https://img.shields.io/github/stars/moxno/privacyscrubber-mcp.svg?style=flat-square) |
| [PrivAiTe](https://github.com/crp4222/PrivAiTe) | Self-hosted OpenAI-compatible proxy that reversibly pseudonymizes PII before requests reach the provider, tool-call arguments and multimodal included. | ![GitHub Badge](https://img.shields.io/github/stars/crp4222/PrivAiTe.svg?style=flat-square) |
| [shim](https://github.com/GetSHIM/shim) | Open-core AI gateway that redacts PII before prompts reach model providers (Presidio plus Turkish TCKN/VKN recognizers). The source-available enterprise edition adds a tamper-evident hash-chain audit log with daily Merkle anchors as evidence for EU AI Act and KVKK record-keeping. | ![GitHub Badge](https://img.shields.io/github/stars/GetSHIM/shim.svg?style=flat-square) |
| [wardcat](https://github.com/oguzhantopcu0/wardcat) | On-premises PII and sensitive-data detection and masking for LLM and RAG workflows, combining regex, NER, and optional local LLMs with reversible masking. | ![GitHub Badge](https://img.shields.io/github/stars/oguzhantopcu0/wardcat?style=flat-square) |
**[⬆ back to ToC](#table-of-contents)**
@ -158,22 +205,44 @@ An awesome & curated list of the best LLMOps tools for developers.
| Project | Details | Repository |
| ------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------- |
| [AgentCost](https://agentcost.tech) | Track and optimize LLM API costs across OpenAI, Anthropic, and LangChain. | ![GitHub Badge](https://img.shields.io/github/stars/agentcost-ai/agentcost-backend.svg?style=flat-square) |
| [Azure OpenAI Logger](https://github.com/aavetis/azure-openai-logger) | "Batteries included" logging solution for your Azure OpenAI instance. | ![GitHub Badge](https://img.shields.io/github/stars/aavetis/azure-openai-logger?style=flat-square) |
| [budget-guard](https://github.com/kimbeomgyu/budget-guard) | Zero-infra hard spending caps for LLM APIs — blocks before overspending (atomic reservation), per-feature cost attribution, streaming usage, retry-storm detection, and adapters for Vercel AI SDK / LangChain.js / LlamaIndex.TS / Mastra. | ![GitHub Badge](https://img.shields.io/github/stars/kimbeomgyu/budget-guard.svg?style=flat-square) |
| [ClevAgent](https://clevagent.io) | Runtime monitoring for AI agents — heartbeat watchdog, loop detection, cost tracking, auto-restart. Python SDK or HTTP API. | |
| [CodeBurn](https://github.com/getagentseal/codeburn) | Local-first cost and token tracking for AI coding agents (Claude Code, Codex, Cursor and 38 more) by project, model and task, with waste detection and plan quota tracking. | ![GitHub Badge](https://img.shields.io/github/stars/getagentseal/codeburn.svg?style=flat-square) |
| [Deepchecks](https://github.com/deepchecks/deepchecks) | Tests for Continuous Validation of ML Models & Data. Deepchecks is a Python package for comprehensively validating your machine learning models and data with minimal effort. | ![GitHub Badge](https://img.shields.io/github/stars/deepchecks/deepchecks.svg?style=flat-square) |
| [digline](https://github.com/digline/digline) | Regression testing for LLM apps: approved scores, prompt and commit pinned as a versioned baseline in your repo; `compare` says which case got worse and by how much. | ![GitHub Badge](https://img.shields.io/github/stars/digline/digline.svg?style=flat-square) |
| [Evidently](https://github.com/evidentlyai/evidently) | An open-source framework to evaluate, test and monitor ML and LLM-powered systems. | ![GitHub Badge](https://img.shields.io/github/stars/evidentlyai/evidently.svg?style=flat-square) |
| [EvalView](https://github.com/hidai25/eval-view) | Regression testing for AI agents. Snapshot behavior, detect tool-call and output regressions, with golden-baseline diffing and LLM-as-judge scoring. Supports LangGraph, CrewAI, OpenAI, Claude, and any HTTP API. | ![GitHub Badge](https://img.shields.io/github/stars/hidai25/eval-view.svg?style=flat-square) |
| [Fiddler AI](https://github.com/fiddler-labs/fiddler-auditor) | Evaluate, monitor, analyze, and improve machine learning and generative models from pre-production to production. Ship more ML and LLMs into production, and monitor ML and LLM metrics like hallucination, PII, and toxicity. | ![GitHub Badge](https://img.shields.io/github/stars/fiddler-labs/fiddler-auditor.svg?style=flat-square) |
| [Giskard](https://github.com/Giskard-AI/giskard) | Testing framework dedicated to ML models, from tabular to LLMs. Detect risks of biases, performance issues and errors in 4 lines of code. | ![GitHub Badge](https://img.shields.io/github/stars/Giskard-AI/giskard.svg?style=flat-square)
| [Flowlines](https://flowlines.ai/) | Behavioral observability for production AI agents that detects drift, context loss, constraint violations, and user frustration across sessions. | |
| [Giskard](https://github.com/Giskard-AI/giskard) | Testing framework dedicated to ML models, from tabular to LLMs. Detect risks of biases, performance issues and errors in 4 lines of code. | ![GitHub Badge](https://img.shields.io/github/stars/Giskard-AI/giskard.svg?style=flat-square)
| [QWED](https://github.com/QWED-AI/qwed-verification) | Deterministic verification protocol for LLM outputs using 8 formal verification engines (SymPy, Z3, AST, SQLGlot). Prevents hallucinations through mathematical proofs rather than statistical methods. | ![GitHub Badge](https://img.shields.io/github/stars/QWED-AI/qwed-verification.svg?style=flat-square) |
| [Great Expectations](https://github.com/great-expectations/great_expectations) | Always know what to expect from your data. | ![GitHub Badge](https://img.shields.io/github/stars/great-expectations/great_expectations.svg?style=flat-square) |
| [Helicone](https://github.com/Helicone/helicone) | Open source LLM observability platform. One line of code to monitor, evaluate, and experiment with features like prompt management, agent tracing, and evaluations. | ![GitHub Badge](https://img.shields.io/github/stars/Helicone/helicone.svg?style=flat-square) |
| [hermes-rubric](https://github.com/hermes-labs-ai/hermes-rubric) | Evidence-first assessment for agent outputs and applications. Synthesizes, loads, or accepts a rubric; collects citations for each dimension; scores against accepted evidence; reports coverage; and emits reproducibility receipts. Apache-2.0. | ![GitHub Badge](https://img.shields.io/github/stars/hermes-labs-ai/hermes-rubric.svg?style=flat-square) |
| [Inferrail](https://github.com/domondi1/inferrail) | Per-run dollar budgets for AI agents and jobs: an OpenAI- and Anthropic-compatible gateway (in-process or self-hosted) that reserves each call's worst-case cost before it is sent, refuses calls that don't fit before the provider, and keeps a payload-free cost record per run. | ![GitHub Badge](https://img.shields.io/github/stars/domondi1/inferrail.svg?style=flat-square) |
| [JevOss](https://github.com/mertkayacs/jevoss) | Test suite for decision-model servers that implement the Jev API: accuracy, probability calibration, and probes for option order, injected instructions, irrelevant padding, yes/no rewording and repeated calls. Apache-2.0. | ![GitHub Badge](https://img.shields.io/github/stars/mertkayacs/jevoss.svg?style=flat-square) |
| [Traceloop OpenLLMetry](https://github.com/traceloop/openllmetry) | OpenTelemetry-based observability and monitoring for LLM and agents workflows. | ![GitHub Badge](https://img.shields.io/github/stars/traceloop/openllmetry.svg?style=flat-square)
| [Langfuse 🪢](https://langfuse.com) | Open-source LLM observability platform that helps teams collaboratively debug, analyze, and iterate on their LLM applications. | ![GitHub Badge](https://img.shields.io/github/stars/langfuse/langfuse.svg?style=flat-square) |
| [Lookspan](https://fervon.dev/lookspan/) | Local-first observability for AI agents. One command (`npx lookspan`) runs a dashboard with traces, a timeline/waterfall view, cost tracking, replay & LLM-as-judge, and datasets. MCP-native, with OpenAI/Anthropic drop-ins and an OpenTelemetry receiver — data stays on your machine. | ![GitHub Badge](https://img.shields.io/github/stars/JoniMartin27/lookspan.svg?style=flat-square) |
| [whylogs](https://github.com/whylabs/whylogs) | The open standard for data logging | ![GitHub Badge](https://img.shields.io/github/stars/whylabs/whylogs.svg?style=flat-square) |
| [Maxim AI](https://getmaxim.ai) | Platform for AI Agent Simulation, Evaluation & Observability |
| [Noveum](https://noveum.ai) | AI agent tracing, evaluation datasets from production interactions, and model comparisons on quality, latency, and cost. | |
| [onWatch](https://github.com/onllm-dev/onwatch) | Lightweight Go CLI that tracks AI API quota usage across 7 providers (Anthropic, OpenAI, GitHub Copilot, MiniMax, and more). Background daemon, <50MB RAM, zero telemetry, SQLite storage. | ![GitHub Badge](https://img.shields.io/github/stars/onllm-dev/onwatch.svg?style=flat-square) |
| [OpenGATE](https://github.com/nickjlamb/opengate) | Deterministic verification for evidence-grounded AI — gold-anchored grounding checks (required facts, ungrounded numbers, abstention) with no LLM judge, plus CI regression gates, pytest/DeepEval integrations, and an MCP server. | ![GitHub Badge](https://img.shields.io/github/stars/nickjlamb/opengate?style=flat-square) |
| [otelcol-genai-sketches](https://github.com/llm-measurement/otelcol-genai-sketches) | OpenTelemetry Collector connector that tracks LLM token usage by user and session without a Prometheus cardinality explosion: bounded per-model metrics plus keyed top users and sessions. Includes a tested LiteLLM recipe and a Helm chart. | ![GitHub Badge](https://img.shields.io/github/stars/llm-measurement/otelcol-genai-sketches.svg?style=flat-square) |
| [phoenix2pytest](https://github.com/golikovichev/phoenix2pytest) | Reads Arize Phoenix traces flagged as failures and synthesises pytest regression tests, so production-discovered LLM failures feed back into the test suite without manual translation. | ![GitHub Badge](https://img.shields.io/github/stars/golikovichev/phoenix2pytest.svg?style=flat-square) |
| [RAGeval](https://github.com/Yacine-ai-tech/RAGeval?ref=awesome-llmops) | Drop-in LLMOps observability platform for RAG with built-in UI and tracing. | ![GitHub Badge](https://img.shields.io/github/stars/Yacine-ai-tech/RAGeval.svg?style=flat-square) |
| [RagTune](https://github.com/metawake/ragtune) | CLI tool for debugging and benchmarking RAG retrieval. EXPLAIN ANALYZE for your retrieval layer. | ![GitHub Badge](https://img.shields.io/github/stars/metawake/ragtune.svg?style=flat-square) |
| [runtape](https://github.com/RehanMohammed985/runtape) | Finds which part of an agent's context caused a bad decision by rerunning it with pieces removed, checks candidate fixes the same way, and writes a pytest regression test. Runs locally with OpenAI, Anthropic, LangGraph and Ollama. | ![GitHub Badge](https://img.shields.io/github/stars/RehanMohammed985/runtape.svg?style=flat-square) |
| [traceAI](https://github.com/future-agi/traceAI) | Open-source AI tracing framework built on OpenTelemetry for deep observability across agentic and LLM workflows. | ![GitHub Badge](https://img.shields.io/github/stars/future-agi/traceAI?style=flat-square) |
| [Future AGI](https://github.com/future-agi/futureagi-sdk) | Production-grade SDK for observability, automated evaluations and prompt management with sub-100ms guardrails for LLM/agent workflows. | ![GitHub Badge](https://img.shields.io/github/stars/future-agi/futureagi-sdk?style=flat-square) |
| [semantic-coverage](https://github.com/aashirpersonal/semantic-coverage) | Visualizes RAG knowledge gaps and "blind spots" using 2D UMAP clustering and density detection. | ![GitHub Badge](https://img.shields.io/github/stars/future-agi/futureagi-sdk?style=flat-square) |
| [Token Police](https://github.com/tokenpolice/token-police-python) | SDK (Python, Node) that prices each LLM call before it is made and blocks or reroutes it on per-user, per-session or per-workflow budget rules; prompt text never leaves the app. | ![GitHub Badge](https://img.shields.io/github/stars/tokenpolice/token-police-python?style=flat-square) |
| [Weco Observe](https://weco.ai) | Observability and debugging tool for AI research agents. Trace multi-step LLM agent runs, visualize decision trees, and identify failure modes in autonomous research workflows. Cloud hosted with open-source agent integration. | |
| [witness](https://github.com/anzal1/witness) | A recording cache for model APIs: one Rust binary that proxies any LLM endpoint, dedupes repeat calls, and writes every model call, MCP tool call, and command side effect into a tamper-evident hash-chained journal with Ed25519 agent identity and Merkle proofs anchored to Sigstore Rekor. | ![GitHub Badge](https://img.shields.io/github/stars/anzal1/witness?style=flat-square) |
| [wobbly](https://github.com/tarunagarwal1981/wobbly) | Metamorphic testing for LLM extraction — catch wrong outputs with no ground-truth labels: perturb inputs in ways that shouldn't change the answer, flag when they do. | ![](https://img.shields.io/github/stars/tarunagarwal1981/wobbly.svg?style=flat-square) |
**[⬆ back to ToC](#table-of-contents)**
@ -182,29 +251,55 @@ An awesome & curated list of the best LLMOps tools for developers.
| Project | Details | Repository |
| ------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------- |
| [agenta](https://github.com/Agenta-AI/agenta) | The LLMOps platform to build robust LLM apps. Easily experiment and evaluate different prompts, models, and workflows to build robust apps. | ![GitHub Badge](https://img.shields.io/github/stars/Agenta-AI/agenta.svg?style=flat-square) |
| [agent-convergence-scorer](https://github.com/roli-lpci/agent-convergence-scorer) | Zero-dependency Python CLI and library for measuring lexical convergence, exact-match rate, overlap, and divergence across parallel agent runs. | ![GitHub Badge](https://img.shields.io/github/stars/roli-lpci/agent-convergence-scorer.svg?style=flat-square) |
| [AgentMark](https://github.com/puzzlet-ai/agentmark) | Type-Safe Markdown-based Agents | ![GitHub Badge](https://img.shields.io/github/stars/Puzzlet-ai/agentmark.svg?style=flat-square) |
| [AgentField](https://github.com/Agent-Field/agentfield) | Open-source control plane for building and operating AI agents like APIs at scale, with routing, memory, observability, identity, auth, and policy controls. | ![GitHub Badge](https://img.shields.io/github/stars/Agent-Field/agentfield.svg?style=flat-square) |
| [Agnos](https://github.com/siva010928/agnos-proxy-oss) | Gateway-agnostic, self-hosted LLM control plane (MIT): an OpenAI-compatible governance proxy (auth, budgets, guardrails, cost, observability) that runs LiteLLM, Bifrost or Portkey as a swappable, stateless translation engine, so provider keys stay in your own control plane instead of the gateway. | ![GitHub Badge](https://img.shields.io/github/stars/siva010928/agnos-proxy-oss.svg?style=flat-square) |
| [AI studio](https://github.com/missingstudio/ai) | A Reliable Open Source AI studio to build core infrastructure stack for your LLM Applications. It allows you to gain visibility, make your application reliable, and prepare it for production with features such as caching, rate limiting, exponential retry, model fallback, and more. | ![GitHub Badge](https://img.shields.io/github/stars/missingstudio/ai.svg?style=flat-square) |
| [AISIX](https://github.com/api7/aisix) | Open-source AI gateway written in Rust: one OpenAI-compatible API (plus a native Anthropic Messages API) in front of LLM providers, and a gateway for MCP servers and A2A agents, with shared API keys, rate limits, guardrails, caching, and Prometheus/OTLP observability. | ![GitHub Badge](https://img.shields.io/github/stars/api7/aisix.svg?style=flat-square) |
| [AIWG](https://github.com/jmagly/aiwg) | Deploys project-owned agents, skills, rules, and governed workflows across AI coding platforms. | ![GitHub Badge](https://img.shields.io/github/stars/jmagly/aiwg.svg?style=flat-square) |
| [App AI Gateway](https://github.com/maxceem/app-ai-gateway) | Open-source (Apache-2.0) AI gateway for iOS apps: provider keys stay on the server instead of shipping in the app, every request is checked with Apple App Attest, and per-install and per-user rate limits and budgets cap spending. Self-host on Cloudflare or use the hosted service. | ![GitHub Badge](https://img.shields.io/github/stars/maxceem/app-ai-gateway.svg?style=flat-square) |
| [Arize-Phoenix](https://github.com/Arize-ai/phoenix) | ML observability for LLMs, vision, language, and tabular models. | ![GitHub Badge](https://img.shields.io/github/stars/Arize-ai/phoenix.svg?style=flat-square) |
| [BitRouter](https://github.com/bitrouter/bitrouter) | Agent-native LLM router that optimizes your agent with every run — zero harness changes, with every model call reliable, traceable, secure, and cost-effective. Routes across OpenAI, Anthropic, Google, OpenRouter, Bedrock, GitHub Copilot, and more through one local endpoint, with cross-protocol translation, an MCP gateway, guardrails, observability, and multi-account failover. Written in Rust. | ![GitHub Badge](https://img.shields.io/github/stars/bitrouter/bitrouter.svg?style=flat-square) |
| [boldrouter](https://boldrouter.com) | OpenAI-compatible AI gateway developed and hosted in Switzerland, with automatic routing, failover, and unified prepaid billing. | |
| [BudgetML](https://github.com/ebhy/budgetml) | Deploy a ML inference service on a budget in less than 10 lines of code. | ![GitHub Badge](https://img.shields.io/github/stars/ebhy/budgetml.svg?style=flat-square) |
| [Caliper](https://github.com/edonadei/caliper) | Measures whether an agent skill helps: runs real Claude Code, Codex, Pi, or Hermes k times per task with and without the skill, scores task success and skill activation separately, and reports token and wall-time cost. | ![GitHub Badge](https://img.shields.io/github/stars/edonadei/caliper.svg?style=flat-square) |
| [Caura](https://github.com/caura-ai/caura) | Governed shared memory for AI agent fleets. Multi-agent, multi-tenant and MCP-native, with trust tiers, audit trails, a knowledge graph and self-improving retrieval. | ![GitHub Badge](https://img.shields.io/github/stars/caura-ai/caura.svg?style=flat-square) |
| [Cheshire Cat AI](https://github.com/cheshire-cat-ai/core) | Web framework to create vertical AI agents. FastAPI based, plugin system inspired to WordPress, admin panel, vector DB included | ![GitHub Badge](https://img.shields.io/github/stars/cheshire-cat-ai/core.svg?style=flat-square) |
| [Chimeraforge](https://github.com/Sahil170595/Chimeraforge) | LLM deployment planner and benchmarking CLI: given a model and a GPU it answers will it fit, will it hit your SLO, and what will it cost. Searches model x quantization x backend x GPU count against VRAM, quality, latency, cost, energy and an opt-in safety gate; models tensor/pipeline parallelism, MoE active parameters, FP8, KV-quant and self-host-vs-API break-even. Every number is labeled measured/estimated/unknown. Also ships an MCP server. | ![GitHub Badge](https://img.shields.io/github/stars/Sahil170595/Chimeraforge.svg?style=flat-square) |
| [claude-router](https://github.com/hermes-labs-ai/claude-router) | Local prompt router that picks the right Claude model tier and prepends the right scaffold using local embeddings, before the API call. | ![GitHub Badge](https://img.shields.io/github/stars/hermes-labs-ai/claude-router.svg?style=flat-square) |
| [CogniCore](https://github.com/cognicore-dev/cognicore-env) | Structured experience memory and learning infrastructure for AI agents, providing verified experience reuse, failure learning, environment-aware validity, and cross-session/cross-agent memory. | ![GitHub Badge](https://img.shields.io/github/stars/cognicore-dev/cognicore-env.svg?style=flat-square) |
| [Contexto](https://github.com/ekailabs/contexto) | Self-hosted context engine for AI agents with persistent conversation memory and recall. Works as a drop-in OpenAI-compatible proxy, OpenClaw plugin, or memory SDK — no code changes required. | ![GitHub Badge](https://img.shields.io/github/stars/ekailabs/contexto.svg?style=flat-square) |
| [Cortex](https://github.com/cortex-docs/cortex) | Generates typed MCP servers, interactive API documentation, and SDKs from OpenAPI, AsyncAPI, GraphQL, gRPC, and OpenRPC sources. | ![GitHub Badge](https://img.shields.io/github/stars/cortex-docs/cortex.svg?style=flat-square) |
| [CrewDock](https://github.com/antonio-mello-ai/crewdock) | Self-hosted control plane for managing AI agents, with task approvals, activity logs, and cost tracking. | ![GitHub Badge](https://img.shields.io/github/stars/antonio-mello-ai/crewdock.svg?style=flat-square) |
| [Dataoorts](https://dataoorts.com/ai) | Enjoy unlimited API calls with Serverless AI Workers/LLMs for just $25 per month. No rate or concurrency limits. | |
| [deeplake](https://github.com/activeloopai/deeplake) | Stream large multimodal datasets to achieve near 100% GPU utilization. Query, visualize, & version control data. Access data w/o the need to recompute the embeddings for the model finetuning. | ![GitHub Badge](https://img.shields.io/github/stars/activeloopai/Hub.svg?style=flat-square) |
| [depdesk](https://github.com/Ort0x36/deprecation-desk) | Finds deprecated and retired LLM model identifiers in a codebase and fails CI before a provider retirement reaches production. Parses Python with the stdlib ast module, weights findings by real call volume from a usage export, and ships a hand-transcribed Anthropic and OpenAI catalog carrying the date each provider page was verified. Zero dependencies. | ![GitHub Badge](https://img.shields.io/github/stars/Ort0x36/deprecation-desk.svg?style=flat-square) |
| [Dify](https://github.com/langgenius/dify) | Open-source framework aims to enable developers (and even non-developers) to quickly build useful applications based on large language models, ensuring they are visual, operable, and improvable. | ![GitHub Badge](https://img.shields.io/github/stars/langgenius/dify.svg?style=flat-square) |
| [Distil](https://github.com/dshakes/distil) | Reversible, certified context compression for LLM agents: gates each compression through a statistical non-inferiority test so the agent's decisions don't change. Stdlib-only, zero-dependency proxy for Anthropic/OpenAI/Gemini. | ![GitHub Badge](https://img.shields.io/github/stars/dshakes/distil.svg?style=flat-square) |
| [Doubleword Control Layer](https://github.com/doublewordai/control-layer) | The world's fastest open-source AI model gateway — ~450× less overhead than LiteLLM. Turns any model into a production-ready, OpenAI-compatible API with built-in auth, rate limits, and user controls. | ![GitHub Badge](https://img.shields.io/github/stars/doublewordai/control-layer.svg?style=flat-square) |
| [Dstack](https://github.com/dstackai/dstack) | Cost-effective LLM development in any cloud (AWS, GCP, Azure, Lambda, etc). | ![GitHub Badge](https://img.shields.io/github/stars/dstackai/dstack.svg?style=flat-square) |
| [Ejentum](https://github.com/ejentum/ejentum-mcp) | Cognitive-harness MCP server with four tools (reasoning, code, anti-deception, memory) returning a structured scaffold (failure pattern, procedure, suppression vectors, falsification test) the agent absorbs before generating. Hosted at api.ejentum.com/mcp and on npm as ejentum-mcp. | ![GitHub Badge](https://img.shields.io/github/stars/ejentum/ejentum-mcp.svg?style=flat-square) |
| [Embedchain](https://github.com/embedchain/embedchain) | Framework to create ChatGPT like bots over your dataset. | ![GitHub Badge](https://img.shields.io/github/stars/embedchain/embedchain.svg?style=flat-square) |
| [Epsilla](https://epsilla.com) | An all-in-one platform to create vertical AI agents powered by your private data and knowledge. | |
| [Evidently](https://github.com/evidentlyai/evidently) | An open-source framework to evaluate, test and monitor ML and LLM-powered systems. | ![GitHub Badge](https://img.shields.io/github/stars/evidentlyai/evidently.svg?style=flat-square) |
| [FerryAPI](https://www.ferryapi.io/) | Low-cost OpenAI-compatible AI API gateway for production workloads, with prepaid balance, usage billing, customer API keys, and provider account pools. | |
| [Fiddler AI](https://www.fiddler.ai/llmops) | Evaluate, monitor, analyze, and improve MLOps and LLMOps from pre-production to production. | |
| [FreeLLMAPI](https://github.com/tashfeenahmed/freellmapi) | OpenAI-compatible proxy that stacks the free tiers of 28 LLM providers (~4B free tokens/month) behind one /v1 endpoint — smart routing, automatic failover, encrypted keys. | ![GitHub Badge](https://img.shields.io/github/stars/tashfeenahmed/freellmapi.svg?style=flat-square) |
| [Glide](https://github.com/EinStack/glide) | Cloud-Native LLM Routing Engine. Improve LLM app resilience and speed. | ![GitHub Badge](https://img.shields.io/github/stars/einstack/glide.svg?style=flat-square) |
| [GoModel](https://github.com/ENTERPILOT/GoModel) | AI gateway exposing a unified OpenAI-compatible API across OpenAI, Anthropic, Gemini, Groq, xAI, Ollama and other providers, with routing, usage tracking, rate limits, and guardrails. | ![GitHub Badge](https://img.shields.io/github/stars/ENTERPILOT/GoModel.svg?style=flat-square) |
| [gotoHuman](https://www.gotohuman.com) | Bring a **human into the loop** in your LLM-based and agentic workflows. Prompt users to approve actions, select next steps, or review and validate generated results. |
| [GPTCache](https://github.com/zilliztech/GPTCache) | Creating semantic cache to store responses from LLM queries. | ![GitHub Badge](https://img.shields.io/github/stars/zilliztech/GPTCache.svg?style=flat-square) |
| [GPUStack](https://github.com/gpustack/gpustack) | An open-source GPU cluster manager for running and managing LLMs | ![GitHub Badge](https://img.shields.io/github/stars/gpustack/gpustack.svg?style=flat-square) |
| [Haystack](https://github.com/deepset-ai/haystack) | Quickly compose applications with LLM Agents, semantic search, question-answering and more. | ![GitHub Badge](https://img.shields.io/github/stars/deepset-ai/haystack.svg?style=flat-square) |
| [Hive](https://github.com/aden-hive/hive) | Open-source AI agent framework for building goal-driven, self-improving autonomous agents with auto-generated graphs, evolution loops, and MCP integration. | ![GitHub Badge](https://img.shields.io/github/stars/aden-hive/hive.svg?style=flat-square) |
| [Helicone](https://github.com/Helicone/helicone) | Open-source LLM observability platform for logging, monitoring, and debugging AI applications. Simple 1-line integration to get started. | ![GitHub Badge](https://img.shields.io/github/stars/helicone/helicone.svg?style=flat-square) |
| [Humanloop](https://humanloop.com) | The LLM evals platform for enterprises, providing tools to develop, evaluate, and observe AI systems. | |
| [Hypersigil](https://github.com/hypersigilhq/hypersigil) | Open-source prompt lifecycle management and gateway with a Web UI. | ![GitHub Badge](https://img.shields.io/github/stars/hypersigilhq/hypersigil.svg?style=flat-square) |
| [Izlo](https://getizlo.com/) | Prompt management tools for teams. Store, improve, test, and deploy your prompts in one unified workspace. | |
| [Keywords AI](https://keywordsai.co/) | A unified DevOps platform for AI software. Keywords AI makes it easy for developers to build LLM applications. | |
| [KubeIntellect](https://github.com/MSKazemi/kubeintellect) | LLM-orchestrated multi-agent framework for Kubernetes operations. Investigates a live cluster via kubectl, Prometheus (PromQL) and Loki (LogQL), correlates the evidence into a root cause, and gates every mutating action behind human approval with role-based access control. | ![GitHub Badge](https://img.shields.io/github/stars/MSKazemi/kubeintellect.svg?style=flat-square) |
| [Kunavo](https://kunavo.com) | Hosted AI API gateway with OpenAI-compatible and native Anthropic endpoints, one prepaid balance, and per-request cost accounting. | |
| [MLflow](https://github.com/mlflow/mlflow/tree/master) | An open-source framework for the end-to-end machine learning lifecycle, helping developers track experiments, evaluate models/prompts, deploy models, and add observability with tracing. | ![GitHub Badge](https://img.shields.io/github/stars/mlflow/mlflow.svg?style=flat-square) |
| [Laminar](https://github.com/lmnr-ai/lmnr) | Open-source all-in-one platform for engineering AI products. Traces, Evals, Datasets, Labels. | ![GitHub Badge](https://img.shields.io/github/stars/lmnr-ai/lmnr.svg?style=flat-square) |
| [langchain](https://github.com/hwchase17/langchain) | Building applications with LLMs through composability | ![GitHub Badge](https://img.shields.io/github/stars/hwchase17/langchain.svg?style=flat-square) |
@ -212,21 +307,31 @@ An awesome & curated list of the best LLMOps tools for developers.
| [Langfuse](https://github.com/langfuse/langfuse) | Open Source LLM Engineering Platform: Traces, evals, prompt management and metrics to debug and improve your LLM application. | ![GitHub Badge](https://img.shields.io/github/stars/langfuse/langfuse.svg?style=flat-square) |
| [LangKit](https://github.com/whylabs/langkit) | Out-of-the-box LLM telemetry collection library that extracts features and profiles prompts, responses and metadata about how your LLM is performing over time to find problems at scale. | ![GitHub Badge](https://img.shields.io/github/stars/whylabs/langkit.svg?style=flat-square) |
| [LangWatch](https://github.com/langwatch/langwatch) | LLM Ops platform with Analytics, Monitoring, Evaluations and an LLM Optimization Studio powered by DSPy | ![GitHub Badge](https://img.shields.io/github/stars/langwatch/langwatch.svg?style=flat-square) |
| [lintlang](https://github.com/hermes-labs-ai/lintlang) | Static analysis for AI agent configs, tool descriptions, and system prompts. Catches vague tool descriptions, missing stop conditions, and schema gaps before they reach runtime. Zero-LLM, deterministic checks, built for CI. | ![GitHub Badge](https://img.shields.io/github/stars/hermes-labs-ai/lintlang.svg?style=flat-square) |
| [LiteLLM 🚅](https://github.com/BerriAI/litellm/) | A simple & light 100 line package to **standardize LLM API calls** across OpenAI, Azure, Cohere, Anthropic, Replicate API Endpoints | ![GitHub Badge](https://img.shields.io/github/stars/BerriAI/litellm.svg?style=flat-square) |
| [Literal AI](https://literalai.com/) | Multi-modal LLM observability and evaluation platform. Create prompt templates, deploy prompts versions, debug LLM runs, create datasets, run evaluations, monitor LLM metrics and collect human feedback. | |
| [LlamaIndex](https://github.com/jerryjliu/llama_index) | Provides a central interface to connect your LLMs with external data. | ![GitHub Badge](https://img.shields.io/github/stars/jerryjliu/llama_index.svg?style=flat-square) |
| [LLMApp](https://github.com/pathwaycom/llm-app) | LLM App is a Python library that helps you build real-time LLM-enabled data pipelines with few lines of code. | ![GitHub Badge](https://img.shields.io/github/stars/pathwaycom/llm-app.svg?style=flat-square) |
| [LLMFlows](https://github.com/stoyan-stoyanov/llmflows) | LLMFlows is a framework for building simple, explicit, and transparent LLM applications such as chatbots, question-answering systems, and agents. | ![GitHub Badge](https://img.shields.io/github/stars/stoyan-stoyanov/llmflows.svg?style=flat-square) |
| [LLMGraph](https://llmgraph.ai) | No-code visual builder for LLM/AI workflows: turn your docs and models into RAG chatbots and AI agents, then deploy each to a REST API and an embeddable chat widget in one click. | |
| [LRM](https://github.com/nickprotop/LocalizationManager) | CLI/TUI tool for managing localization files (.resx, JSON, Android, iOS) with LLM-powered translation via Ollama, validation, and code scanning for unused/missing keys. | ![GitHub Badge](https://img.shields.io/github/stars/nickprotop/LocalizationManager.svg?style=flat-square) |
| [Lunary](https://github.com/lunary-ai/lunary) | Observability and prompt management for LLM chabots and agents. Debug agents with powerful tracing and logging. Usage analytics and dive deep into the history of your requests. Developer friendly modules with plug-and-play integration into LangChain. | ![GitHub Badge](https://img.shields.io/github/stars/lunary-ai/lunary.svg?style=flat-square) |
| [Mengram](https://github.com/alibaizhanov/mengram) | Open-source memory infrastructure for AI agents. Provides semantic (entities/facts), episodic (conversations), and procedural (learned behaviors) memory with auto-reflection. Python SDK, JS SDK, MCP server, and REST API. | ![GitHub Badge](https://img.shields.io/github/stars/alibaizhanov/mengram.svg?style=flat-square) |
| [magentic](https://github.com/jackmpcollins/magentic) | Seamlessly integrate LLMs as Python functions. Use type annotations to specify structured output. Mix LLM queries and function calling with regular Python code to create complex LLM-powered functionality. | ![GitHub Badge](https://img.shields.io/github/stars/jackmpcollins/magentic.svg?style=flat-square) |
| [Manag.ai](https://www.manag.ai) | Your all-in-one prompt management and observability platform. Craft, track, and perfect your LLM prompts with ease. | |
| [Mandare](https://github.com/mandarelabs/mandare) | Self-hosted LLM gateway that enforces human-signed per-agent spend mandates (cost reserved before the call), records every allow and refusal in a tamper-evident ledger, and has a kill switch. | ![GitHub Badge](https://img.shields.io/github/stars/mandarelabs/mandare.svg?style=flat-square) |
| [Mirascope](https://github.com/Mirascope/mirascope) | Intuitive convenience tooling for lightning-fast, efficient development and ensuring quality in LLM-based applications | ![GitHub Badge](https://img.shields.io/github/stars/Mirascope/mirascope.svg?style=flat-square) |
| [NativePort](https://nativeport.ai/) | Infrastructure for accessing web-search, scraping, browser-automation, voice, and model-inference providers through a single account, API key, and balance. Publishes dated, capability-specific [leaderboards](https://nativeport.ai/leaderboards/) and a public [methodology](https://nativeport.ai/methodology/). | |
| [Neurolink](https://github.com/juspay/neurolink) | Multi-provider AI agent framework that unifies 12+ LLM providers (OpenAI, Google, Anthropic, AWS, Azure, Groq, etc.) with workflow orchestration. Production-grade platform for building LLM applications with streaming, tool calling, caching, and enterprise features. Battle-tested at 15M+ requests/month. | ![GitHub Badge](https://img.shields.io/github/stars/juspay/neurolink.svg?style=flat-square) |
| [Nora](https://github.com/solomon2773/nora) | Self-hosted control plane for deploying and operating OpenClaw and Hermes agent fleets on Docker and Kubernetes, with lifecycle management, observability, cost tracking, secrets, schedules, and REST, CLI, and MCP interfaces. | ![GitHub Badge](https://img.shields.io/github/stars/solomon2773/nora.svg?style=flat-square) |
| [OpenConnector](https://github.com/oomol-lab/open-connector) | Self-hostable tool gateway for LLM agents with SaaS action discovery, OAuth/API-key credential management, scoped runtime tokens, and MCP, HTTP, and OpenAPI interfaces. | ![GitHub Badge](https://img.shields.io/github/stars/oomol-lab/open-connector.svg?style=flat-square) |
| [OpenLIT](https://github.com/openlit/openlit) | OpenLIT is an OpenTelemetry-native GenAI and LLM Application Observability tool and provides OpenTelmetry Auto-instrumentation for monitoring LLMs, VectorDBs and Frameworks. It provides valuable insights into token & cost usage, user interaction, and performance related metrics. | ![GitHub Badge](https://img.shields.io/github/stars/dokulabs/doku.svg?style=flat-square) |
| [Opik](https://github.com/comet-ml/opik) | Confidently evaluate, test, and ship LLM applications with a suite of observability tools to calibrate language model outputs across your dev and production lifecycle. | ![GitHub Badge](https://img.shields.io/github/stars/comet-ml/opik.svg?style=flat-square) |
| [OrcaRouter Lite](https://github.com/Continuum-AI-Corp/OrcaRouter-Lite) | Self-hosted, OpenAI-compatible LLM router. Bring your own provider keys and route across 100+ models from multiple providers with automatic failover and streaming; `model="auto"` selects a model by cost, latency, or quality. Includes a local analytics dashboard and does not send telemetry. | ![GitHub Badge](https://img.shields.io/github/stars/Continuum-AI-Corp/OrcaRouter-Lite.svg?style=flat-square) |
| [Overmind](https://github.com/overmind-core/overmind) | Turns production traces into specialised models you own: OpenTelemetry trace ingest, datasets, evals, LoRA fine-tuning of open-weight models and OpenAI-compatible serving. Self-hostable. | ![GitHub Badge](https://img.shields.io/github/stars/overmind-core/overmind.svg?style=flat-square) |
| [Parea AI](https://www.parea.ai/) | Platform and SDK for AI Engineers providing tools for LLM evaluation, observability, and a version-controlled enhanced prompt playground. | ![GitHub Badge](https://img.shields.io/github/stars/parea-ai/parea-sdk-py?style=flat-square) |
| [Pezzo 🕹️](https://github.com/pezzolabs/pezzo) | Pezzo is the open-source LLMOps platform built for developers and teams. In just two lines of code, you can seamlessly troubleshoot your AI operations, collaborate and manage your prompts in one place, and instantly deploy changes to any environment. | ![GitHub Badge](https://img.shields.io/github/stars/pezzolabs/pezzo.svg?style=flat-square) |
| [Pilot Protocol](https://pilotprotocol.network) | Open-source overlay network giving AI agents a permanent virtual address, encrypted UDP tunnels with NAT traversal, and an explicit per-peer trust model, plus an app store of installable agent-native capabilities (discover → install → call). | ![GitHub Badge](https://img.shields.io/github/stars/TeoSlayer/pilotprotocol.svg?style=flat-square) |
| [PraisonAI](https://github.com/MervinPraison/PraisonAI) | Production-ready Multi-AI Agents framework with self-reflection. Fastest agent instantiation (3.77μs), 100+ LLM support via LiteLLM, MCP integration, agentic workflows (route/parallel/loop/repeat), built-in memory, Python & JS SDKs. | ![GitHub Badge](https://img.shields.io/github/stars/MervinPraison/PraisonAI.svg?style=flat-square) |
| [PromptDX](https://github.com/puzzlet-ai/promptdx) | A declarative, extensible, and composable approach for developing LLM prompts using Markdown and JSX. | ![GitHub Badge](https://img.shields.io/github/stars/puzzlet-ai/promptdx.svg?style=flat-square) |
| [PromptHub](https://www.prompthub.us) | Full stack prompt management tool designed to be usable by technical and non-technical team members. Test, version, collaborate, deploy, and monitor, all from one place. | |
@ -238,26 +343,54 @@ An awesome & curated list of the best LLMOps tools for developers.
| [Prompteams](https://www.prompteams.com) | Prompt management system. Version, test, collaborate, and retrieve prompts through real-time APIs. Have GitHub style with repos, branches, and commits (and commit history). | |
| [prompttools](https://github.com/hegelai/prompttools) | Open-source tools for testing and experimenting with prompts. The core idea is to enable developers to evaluate prompts using familiar interfaces like code and notebooks. In just a few lines of codes, you can test your prompts and parameters across different models (whether you are using OpenAI, Anthropic, or LLaMA models). You can even evaluate the retrieval accuracy of vector databases. | ![GitHub Badge](https://img.shields.io/github/stars/hegelai/prompttools.svg?style=flat-square) |
| [Puzzlet AI](https://www.puzzlet.ai) | The Git-Based LLM Engineering Platform. Achieve more from GenAI: Manage, evaluate, and improve your full-stack LLM application - with version control, type-safety, and local development built-in. | |
| [Quotaflow](https://quotaflow.ai) | AI token and API resource utilization platform that helps teams reduce wasted subscribed quota and improve turnover across controlled internal pools. | |
| [systemprompt.io](https://systemprompt.io) | Systemprompt.io is a Rest API with quality tooling to enable the creation, use and observability of prompts in any AI system. Control every detail of your prompt for a SOTA prompt management experience. | |
| [TeamoRouter](https://router.teamolab.com) | LLM routing gateway for OpenClaw. One API key to access Claude, GPT-4o, Gemini, DeepSeek, Kimi, MiniMax. Smart routing modes (teamo-best, teamo-balanced, teamo-eco) auto-pick the optimal model. Up to 50% off official prices. 2-second install via skill.md. | |
| [TokenMix](https://tokenmix.ai) | AI gateway routing 171 LLMs from 14 providers (Claude, GPT, Gemini, DeepSeek, Qwen, and more) through one OpenAI-compatible endpoint. Pass-through pricing, automatic failover, no monthly subscription. | |
| [TreeScale](https://treescale.com) | All In One Dev Platform For LLM Apps. Deploy LLM-enhanced APIs seamlessly using tools for prompt optimization, semantic querying, version management, statistical evaluation, and performance tracking. As a part of the developer friendly API implementation TreeScale offers Elastic LLM product, which makes a unified API Endpoint for all major LLM providers and open source models. | |
| [TrueFoundry](https://www.truefoundry.com/) | Deploy LLMOps tools like Vector DBs, Embedding server etc on your own Kubernetes (EKS,AKS,GKE,On-prem) Infra including deploying, Fine-tuning, tracking Prompts and serving Open Source LLM Models with full Data Security and Optimal GPU Management. Train and Launch your LLM Application at Production scale with best Software Engineering practices. | |
| [ReliableGPT 💪](https://github.com/BerriAI/reliableGPT/) | Handle OpenAI Errors (overloaded OpenAI servers, rotated keys, or context window errors) for your production LLM Applications. | ![GitHub Badge](https://img.shields.io/github/stars/BerriAI/reliableGPT.svg?style=flat-square) |
| [Registry Broker](https://github.com/hashgraph-online/registry-broker) | Universal index and routing layer for AI agents. Aggregates agent metadata from multiple registries (NANDA, MCP, Virtuals, OpenRouter, A2A, X402 Bazaar) across web2 and web3, normalizes profiles, and provides protocol translation between agent ecosystems. | ![GitHub Badge](https://img.shields.io/github/stars/hashgraph-online/registry-broker.svg?style=flat-square) |
| [Rhesis](https://github.com/rhesis-ai/rhesis) | Open-source testing infrastructure for LLM and agentic applications. Collaborative platform enabling teams to define quality metrics, run evaluations, and ship confidently with version control and peer review workflows built for AI engineering. | ![GitHub Badge](https://img.shields.io/github/stars/rhesis-ai/rhesis.svg?style=flat-square) |
| [rote](https://github.com/trevhud/rote) | Open-source CLI that compiles a proven agent skill (a SKILL.md plus references) into a typed, deterministic pipeline. Fixed logic becomes reviewable Python or TypeScript with per-step tests, while judgment steps stay as typed LLM-judge signatures. Emits DBOS, Temporal, Cloudflare Workflows, Inngest, or plain Python/TS, and can serve compiled pipelines as MCP tools. | ![GitHub Badge](https://img.shields.io/github/stars/trevhud/rote.svg?style=flat-square) |
| [Roundtable](https://github.com/askbudi/roundtable) | Zero-configuration unified AI assistant management built on the FastMCP framework. Provides seamless integration with Claude, ChatGPT, and other AI assistants through a single MCP interface with session management, logging, and production-ready operations. | ![GitHub Badge](https://img.shields.io/github/stars/askbudi/roundtable.svg?style=flat-square) |
| [Portkey](https://portkey.ai/) | Control Panel with an observability suite & an AI gateway — to ship fast, reliable, and cost-efficient apps. | |
| [TensorZero](https://www.tensorzero.com/) | TensorZero is an open-source framework for building production-grade LLM applications. It unifies an LLM gateway, observability, optimization, evaluations, and experimentation. | ![GitHub Badge](https://img.shields.io/github/stars/tensorzero/tensorzero.svg?style=flat-square) |
| [SAVI SDK](https://github.com/data-gras/savi-sdk) | Open-source observability SDK for LLM cost, PII masking, carbon, and compliance — drop-in wrapper for OpenAI/Anthropic/Bedrock/Cohere/Mistral/Vertex; works standalone with zero account via local_mode. | ![GitHub Badge](https://img.shields.io/github/stars/data-gras/savi-sdk.svg?style=flat-square) |
| [Self-Hosted AI Stack](https://github.com/hwdsl2/self-hosted-ai-stack) | Docker Compose stack for local AI with Ollama, a LiteLLM gateway, RAG, voice services, MCP tools, persistent data, and health checks. | ![GitHub Badge](https://img.shields.io/github/stars/hwdsl2/self-hosted-ai-stack.svg?style=flat-square) |
| [Semantic Cache Router](https://github.com/redjackfred/distributed-semantic-cache-and-stateful-routing-system) | Distributed semantic cache and stateful routing system that cuts LLM API costs by returning cached responses for semantically similar queries. Uses ANN vector search (cosine ≥ 0.8) and consistent hashing to pin requests to the same worker, achieving ~7× latency reduction on cache hits while scaling horizontally without cache thrash. | ![GitHub Badge](https://img.shields.io/github/stars/redjackfred/distributed-semantic-cache-and-stateful-routing-system.svg?style=flat-square) |
| [Spendline](https://www.spendline.ai/) | Financial control layer for AI spend — per-customer cost attribution and hierarchical budgets enforced before the provider call. | |
| [Statewave](https://github.com/smaramwbc/statewave) | Open-source memory runtime for AI agents. Compiles events into deterministic, provenance-tagged context bundles instead of query-time retrieval. Apache-2.0, self-hostable on Postgres + pgvector. | ![GitHub Badge](https://img.shields.io/github/stars/smaramwbc/statewave.svg?style=flat-square) |
| [Symbioza](https://symbioza.dev) | Run a GPU job under a hard dollar cap and collect the files it writes. | |
| [TensorZero](https://www.tensorzero.com/) | TensorZero is an open-source framework for building production-grade LLM applications. It unifies an LLM gateway, observability, optimization, evaluations, and experimentation. | ![GitHub Badge](https://img.shields.io/github/stars/tensorzero/tensorzero.svg?style=flat-square) |
| [TheYgent](https://github.com/al2m4n/TheYgent) | Self-hosted, no-code platform for building and running AI agents on a visual canvas, with built-in local inference (llama.cpp, MLX, vLLM) or any OpenAI-compatible API. Durable crash-resumable runs, per-run trace waterfalls with OTLP export, MCP tools, RAG on pgvector, and human-in-the-loop approvals. | ![GitHub Badge](https://img.shields.io/github/stars/al2m4n/TheYgent.svg?style=flat-square) |
| [ThinkWatch Lite](https://github.com/ThinkWatchProject/ThinkWatch-Lite) | Desktop app for macOS, Windows and Linux that runs a local LLM gateway for Claude Code, Codex and other AI coding clients: routing rules with failover, conversion between the Anthropic, OpenAI and Gemini APIs, a record of each request's route and cost, and redaction of API keys before requests leave. The gateway engine, ThinkWatch Core, also runs on its own on a Linux server. | ![GitHub Badge](https://img.shields.io/github/stars/ThinkWatchProject/ThinkWatch-Lite.svg?style=flat-square) |
| [Trinity](https://github.com/Abilityai/trinity) | Self-hosted platform that runs AI coding agents as persistent scheduled services. Each agent runs in a Docker container with its own workspace and git-backed memory, plus scheduling, an approvals queue, an MCP server and a web console. | ![GitHub Badge](https://img.shields.io/github/stars/Abilityai/trinity.svg?style=flat-square) |
| [UnoRouter](https://unorouter.com) | OpenAI-compatible LLM gateway with one API key for every major provider and smart routing across models. Drop-in for code, Claude Code, and chat clients like SillyTavern, Janitor.AI, RisuAI, and Chub. | |
| [Vellum](https://www.vellum.ai/) | An AI product development platform to experiment with, evaluate, and deploy advanced LLM apps. | |
| [Weights & Biases (Prompts)](https://docs.wandb.ai/guides/prompts) | A suite of LLMOps tools within the developer-first W&B MLOps platform. Utilize W&B Prompts for visualizing and inspecting LLM execution flow, tracking inputs and outputs, viewing intermediate results, securely managing prompts and LLM chain configurations. | |
| [Wenlan](https://github.com/7xuanlu/wenlan) | Local-first AI knowledge base and LLM wiki that distills agent work into source-cited pages with graph context and hybrid retrieval across MCP clients. | ![GitHub Badge](https://img.shields.io/github/stars/7xuanlu/wenlan.svg?style=flat-square) |
| [Wordware](https://www.wordware.ai) | A web-hosted IDE where non-technical domain experts work with AI Engineers to build task-specific AI agents. It approaches prompting as a new programming language rather than low/no-code blocks. | |
| [XiuRouter](https://router.xiu.ai/) | Hosted multi-model API service with OpenAI Chat Completions and Responses, Anthropic Messages, and Gemini GenerateContent routes, scoped API keys, usage-based pricing, and request-level usage and cost records. | |
| [xTuring](https://github.com/stochasticai/xturing) | Build and control your personal LLMs with fast and efficient fine-tuning. | ![GitHub Badge](https://img.shields.io/github/stars/stochasticai/xturing.svg?style=flat-square) |
| [ZenML](https://github.com/zenml-io/zenml) | Open-source framework for orchestrating, experimenting and deploying production-grade ML solutions, with built-in `langchain` & `llama_index` integrations. | ![GitHub Badge](https://img.shields.io/github/stars/zenml-io/zenml.svg?style=flat-square) |
| [Future AGI](https://github.com/future-agi/ai-evaluation) | Evaluation framework for automated, reproducible scoring of LLM, agent, and workflow performance. | ![GitHub Badge](https://img.shields.io/github/stars/future-agi/ai-evaluation?style=flat-square) |
| [SwarmClaw](https://github.com/swarmclawai/swarmclaw) | Self-hosted multi-agent AI runtime with 23+ LLM providers, persistent memory, skills, schedules, sub-agent spawning, and MCP client + server support. Ships as desktop app, CLI, or Docker. | ![GitHub Badge](https://img.shields.io/github/stars/swarmclawai/swarmclaw.svg?style=flat-square) |
| [ai-evaluation](https://github.com/future-agi/ai-evaluation) | Evaluation framework for automated, reproducible scoring of LLM, agent, and workflow performance. | ![GitHub Badge](https://img.shields.io/github/stars/future-agi/ai-evaluation?style=flat-square) |
| [future-agi](https://github.com/future-agi/future-agi) | Open-source self-hostable end-to-end agent engineering and optimization platform unifying tracing, evals, simulations, datasets, gateway, and guardrails for LLM and AI agent applications. | ![GitHub Badge](https://img.shields.io/github/stars/future-agi/future-agi?style=flat-square) |
| [Modelglass](https://modelglass.com.au) | Sourced, versioned pricing and capability data for AI models (image, language, video, audio, plus coding/science/agentic benchmark verticals) to find the cheapest model that clears a capability bar. | |
**[⬆ back to ToC](#table-of-contents)**
## Search
### Hybrid search
| Project | Details | Repository |
| --------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- |
| [Airweave](https://github.com/airweave-ai/airweave) | An easy way to turn any app into searchable data for LLMs. | ![GitHub Badge](https://img.shields.io/github/stars/airweave-ai/airweave.svg?style=flat-square) |
| [MemorySync](https://github.com/memorysyncio/memorysync-cursor-starter) | Persistent multi-tenant memory layer and MCP server for AI coding assistants with sub-50ms hybrid recall. | [![GitHub Badge](https://img.shields.io/github/stars/memorysyncio/memorysync-cursor-starter.svg?style=flat-square)](https://github.com/memorysyncio/memorysync-cursor-starter) |
| [Remio](https://remio.ai/) | Local-first personal knowledge base that pre-parses files, webpages, recordings, notes, emails, messages, and images into indexed and vector-retrievable context for faster AI search and agent workflows, reducing repeated raw-file scans and context-token cost. | |
### Vector search
| Project | Details | Repository |
@ -267,13 +400,16 @@ An awesome & curated list of the best LLMOps tools for developers.
| [Chroma](https://github.com/chroma-core/chroma) | the open source embedding database | ![GitHub Badge](https://img.shields.io/github/stars/chroma-core/chroma.svg?style=flat-square) |
| [Epsilla](https://github.com/epsilla-cloud/vectordb) | A 10x faster, cheaper, and better vector database | ![GitHub Badge](https://img.shields.io/github/stars/epsilla-cloud/vectordb.svg?style=flat-square) |
| [Infinity](https://github.com/infiniflow/infinity) | The AI-native database built for LLM applications, providing incredibly fast vector and full-text search | ![GitHub Badge](https://img.shields.io/github/stars/infiniflow/infinity.svg?style=flat-square) |
| [Infino](https://github.com/infino-ai/infino) | Embedded retrieval engine on Apache Parquet: BM25 full-text, vector, hybrid (RRF), and SQL from one engine over object storage. | ![GitHub Badge](https://img.shields.io/github/stars/infino-ai/infino.svg?style=flat-square) |
| [Lancedb](https://github.com/lancedb/lancedb) | Developer-friendly, serverless vector database for AI applications. Easily add long-term memory to your LLM apps! | ![GitHub Badge](https://img.shields.io/github/stars/lancedb/lancedb.svg?style=flat-square) |
| [Marqo](https://github.com/marqo-ai/marqo) | Tensor search for humans. | ![GitHub Badge](https://img.shields.io/github/stars/marqo-ai/marqo.svg?style=flat-square) |
| [Milvus](https://github.com/milvus-io/milvus) | Vector database for scalable similarity search and AI applications. | ![GitHub Badge](https://img.shields.io/github/stars/milvus-io/milvus.svg?style=flat-square) |
| [Omnigraph](https://github.com/ModernRelay/omnigraph) | Typed graph database where agents branch and merge like Git. S3-native, Rust, traversal + vector + BM25 in one runtime. | ![GitHub Badge](https://img.shields.io/github/stars/ModernRelay/omnigraph.svg?style=flat-square) |
| [ParadeDB](https://github.com/paradedb/paradedb) | The transactional alternative to Elasticsearch, built on Postgres. | ![GitHub Badge](https://img.shields.io/github/stars/paradedb/paradedb.svg?style=flat-square) |
| [Pinecone](https://www.pinecone.io/) | The Pinecone vector database makes it easy to build high-performance vector search applications. Developer-friendly, fully managed, and easily scalable without infrastructure hassles. | |
| [pgvector](https://github.com/pgvector/pgvector) | Open-source vector similarity search for Postgres. | ![GitHub Badge](https://img.shields.io/github/stars/pgvector/pgvector.svg?style=flat-square) |
| [Rivestack](https://rivestack.io) | Managed PostgreSQL with pgvector for AI workloads. Built-in SQL editor lets you query your database with natural language (auto-converted to vector embeddings). Free tier includes 2GB storage. | |
| [SynapCores](https://synapcores.com) | Self-hosted AI-native database: vector search + Cypher graph + embedded GGUF inference and SQL in one engine. Free Community Edition binary; source proprietary. | ![GitHub Badge](https://img.shields.io/github/stars/SynapCores/synapcores-releases.svg?style=flat-square) |
| [VectorChord](https://github.com/tensorchord/VectorChord) | Scalable, fast, and disk-friendly vector search in Postgres, the successor of `pgvecto.rs`. | ![GitHub Badge](https://img.shields.io/github/stars/tensorchord/VectorChord.svg?style=flat-square) |
| [pgvecto.rs](https://github.com/tensorchord/pgvecto.rs) | Vector database plugin for Postgres, written in Rust, specifically designed for LLM. | ![GitHub Badge](https://img.shields.io/github/stars/tensorchord/pgvecto.rs.svg?style=flat-square) |
| [Qdrant](https://github.com/qdrant/qdrant) | Vector Search Engine and Database for the next generation of AI applications. Also available in the cloud | ![GitHub Badge](https://img.shields.io/github/stars/qdrant/qdrant.svg?style=flat-square) |
@ -290,13 +426,29 @@ An awesome & curated list of the best LLMOps tools for developers.
| Project | Details | Repository |
| --------------------------------------------------- | -------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------- |
| [AgentsMesh](https://github.com/AgentsMesh/AgentsMesh) | Self-hostable AI Agent Workforce Platform. Multi-agent orchestration with remote AI workstations (AgentPods), PTY sandbox + git worktree isolation, built-in Kanban, and per-pod MCP server. Supports Claude Code, Codex CLI, Gemini CLI, Aider, OpenCode. | ![GitHub Badge](https://img.shields.io/github/stars/AgentsMesh/AgentsMesh.svg?style=flat-square) |
| [aigent-os](https://github.com/wrg32786/aigent-os) | MIT harness for long-running Claude Code sessions that saves a capsule, runs /clear and resumes instead of compacting. | ![GitHub Badge](https://img.shields.io/github/stars/wrg32786/aigent-os.svg?style=flat-square) |
| [Atomic Agent](https://github.com/AtomicBot-ai/atomic-agent) | Local-first CLI and TUI coding agent that runs open-weight models entirely on your machine through a llama.cpp fork. No account or API key required. 56 built-in tools (browser, filesystem, git, memory, vision), MCP support, five-layer local memory, macOS/Linux/Windows. | ![GitHub Badge](https://img.shields.io/github/stars/AtomicBot-ai/atomic-agent.svg?style=flat-square) |
| [Bernstein](https://github.com/sipyourdrink-ltd/bernstein) | Deterministic Python orchestrator for 37 CLI coding agents (Claude Code, Codex CLI, Gemini CLI, GitHub Copilot CLI, Cursor, Aider, OpenHands, OpenCode, Goose, Qwen, Ollama, ...) running in parallel git worktrees. First-class MCP server, quality gates, cost tracking with budgets. | ![GitHub Badge](https://img.shields.io/github/stars/sipyourdrink-ltd/bernstein.svg?style=flat-square) |
| [CodeGeeX](https://github.com/THUDM/CodeGeeX) | CodeGeeX: An Open Multilingual Code Generation Model (KDD 2023) | ![GitHub Badge](https://img.shields.io/github/stars/THUDM/CodeGeeX.svg?style=flat-square) |
| [CodeGen](https://github.com/salesforce/CodeGen) | CodeGen is an open-source model for program synthesis. Trained on TPU-v4. Competitive with OpenAI Codex. | ![GitHub Badge](https://img.shields.io/github/stars/salesforce/CodeGen.svg?style=flat-square) |
| [Coder Eval](https://github.com/UiPath/coder_eval) | Framework for evaluating, benchmarking, and A/B-testing AI coding agents (Claude Code, Codex CLI, Gemini/Antigravity) and their skills. Runs a real agent in a sandbox against declarative YAML tasks, then scores the resulting files and commands with weighted 0.0-1.0 criteria. Includes 14 criterion types, per-tool token/cost telemetry, dataset fan-out with skill-activation precision/recall gates, and a GitHub Action with JUnit XML output for CI. | ![GitHub Badge](https://img.shields.io/github/stars/UiPath/coder_eval.svg?style=flat-square) |
| [CodeT5](https://github.com/salesforce/CodeT5) | Open Code LLMs for Code Understanding and Generation. | ![GitHub Badge](https://img.shields.io/github/stars/salesforce/CodeT5.svg?style=flat-square) |
| [Continue](https://github.com/continuedev/continue) | ⏩ the open-source autopilot for software development—bring the power of ChatGPT to VS Code | ![GitHub Badge](https://img.shields.io/github/stars/continuedev/continue.svg?style=flat-square) |
| [Cotal](https://github.com/Cotal-AI/Cotal) | Open pub/sub standard over NATS JetStream for coordinating coding agents. Claude Code, Codex, OpenCode, Hermes, Jcode and pi agents share a space with presence, channels, durable direct messages and role-addressed delivery. | ![GitHub Badge](https://img.shields.io/github/stars/Cotal-AI/Cotal.svg?style=flat-square) |
| [DSH Studio](https://github.com/Moresyl/dsh-studio) | Cross-platform desktop host for installing, configuring, and managing DeepSeek Harness. | ![GitHub Badge](https://img.shields.io/github/stars/Moresyl/dsh-studio.svg?style=flat-square) |
| [fauxpilot](https://github.com/fauxpilot/fauxpilot) | An open-source alternative to GitHub Copilot server | ![GitHub Badge](https://img.shields.io/github/stars/fauxpilot/fauxpilot.svg?style=flat-square) |
| [fractal](https://github.com/plasma-ai/fractal) | Hierarchical coding-agent orchestrator with recursive delegation, per-node Git worktrees, configurable limits, persistent SQLite state, and live terminal monitoring and steering. | ![GitHub Badge](https://img.shields.io/github/stars/plasma-ai/fractal.svg?style=flat-square) |
| [Kolega Code](https://github.com/kolega-ai/kolega-code) | Python terminal coding agent where the model writes its own multi-agent workflows (Gigacode). Provider-agnostic, local-first, 15+ model providers, MCP client, browser agent. | ![GitHub Badge](https://img.shields.io/github/stars/kolega-ai/kolega-code.svg?style=flat-square) |
| [Orbi](https://github.com/orbi-build/orbi) | Self-hosted runner that takes a labelled GitHub Issue to a pull request, has a separate review session check the frozen diff against the acceptance criteria, merges only the reviewed head, and publishes the tagged release. Works with any OpenAI-compatible model. | ![GitHub Badge](https://img.shields.io/github/stars/orbi-build/orbi.svg?style=flat-square) |
| [promptext](https://github.com/1broseidon/promptext) | Smart code context extractor for AI assistants with accurate token counting and budget management | ![GitHub Badge](https://img.shields.io/github/stars/1broseidon/promptext.svg?style=flat-square) |
| [Relay](https://github.com/ImBIOS/relay) | Universal AI API proxy — hot-switch between 14+ LLM providers (GitHub Copilot, OpenAI, Anthropic, DeepSeek, Groq, Ollama, and more) from any AI coding agent without restarting sessions. Local proxy with automatic Anthropic <-> OpenAI protocol translation, account rotation, and real-time usage tracking. | ![GitHub Badge](https://img.shields.io/github/stars/ImBIOS/relay.svg?style=flat-square) |
| [Superagent](https://github.com/pungme/superagent-desktop) | Open-source macOS desktop app that gives Claude Code and Codex a real browser to drive, an iOS Simulator to install and screenshot apps in, and a phone companion app for remote monitoring. | ![GitHub Badge](https://img.shields.io/github/stars/pungme/superagent-desktop.svg?style=flat-square) |
| [Synaps](https://github.com/HaseebKhalid1507/SynapsCLI) | Terminal AI agent runtime in Rust with a ratatui TUI, parallel subagents, MCP, and plugins; works with Claude, OpenAI, Gemini, or local models. | ![GitHub Badge](https://img.shields.io/github/stars/HaseebKhalid1507/SynapsCLI.svg?style=flat-square) |
| [tabby](https://github.com/TabbyML/tabby) | Self-hosted AI coding assistant. An opensource / on-prem alternative to GitHub Copilot. | ![GitHub Badge](https://img.shields.io/github/stars/TabbyML/tabby.svg?style=flat-square) |
| [AIDE](https://github.com/WecoAI/aideml) | Open-source ML engineering agent that uses tree search to explore solution spaces. Automates machine learning experimentation from data analysis to model training. [Paper](https://arxiv.org/abs/2502.13138). | ![GitHub Badge](https://img.shields.io/github/stars/WecoAI/aideml.svg?style=flat-square) |
| [Kapso](https://github.com/Leeroo-AI/kapso) | Long-running agents that optimize AI and Data systems, and learn from every experience. \#1 open-source on MLE-Bench; ALE-Bench; RelBench. | ![GitHub Badge](https://img.shields.io/github/stars/Leeroo-AI/kapso.svg?style=flat-square) |
| [webcmd](https://github.com/agentrhq/webcmd) | Self-learning browser infrastructure for AI coding agents that compiles site navigation into deterministic per-site CLI commands. | ![GitHub Badge](https://img.shields.io/github/stars/agentrhq/webcmd.svg?style=flat-square) |
## Training
@ -310,6 +462,7 @@ An awesome & curated list of the best LLMOps tools for developers.
| [envd](https://github.com/tensorchord/envd) | 🏕️ Reproducible development environment for AI/ML. | ![GitHub Badge](https://img.shields.io/github/stars/tensorchord/envd.svg?style=flat-square) |
| [Jupyter Notebooks](https://github.com/jupyter/notebook) | The Jupyter notebook is a web-based notebook environment for interactive computing. | ![GitHub Badge](https://img.shields.io/github/stars/jupyter/notebook.svg?style=flat-square) |
| [Kurtosis](https://github.com/kurtosis-tech/kurtosis) | A build, packaging, and run system for ephemeral multi-container environments. | ![GitHub Badge](https://img.shields.io/github/stars/kurtosis-tech/kurtosis.svg?style=flat-square) |
| [LayerSmith](https://github.com/r0lfi/layersmith) | Self-hosted web UI, TUI, and CLI for building OCI images with Docker or Podman, including LLM training environments and air-gap bundles. | ![GitHub Badge](https://img.shields.io/github/stars/r0lfi/layersmith.svg?style=flat-square) |
| [Wordware](https://www.wordware.ai) | A web-hosted IDE where non-technical domain experts work with AI Engineers to build task-specific AI agents. It approaches prompting as a new programming language rather than low/no-code blocks. | |
**[⬆ back to ToC](#table-of-contents)**
@ -326,6 +479,7 @@ An awesome & curated list of the best LLMOps tools for developers.
| [peft](https://github.com/huggingface/peft) | State-of-the-art Parameter-Efficient Fine-Tuning. | ![GitHub Badge](https://img.shields.io/github/stars/huggingface/peft.svg?style=flat-square) |
| [p-tuning-v2](https://github.com/THUDM/P-tuning-v2) | An optimized prompt tuning strategy achieving comparable performance to fine-tuning on small/medium-sized models and sequence tagging challenges. [(ACL 2022)](https://arxiv.org/abs/2110.07602) | ![GitHub Badge](https://img.shields.io/github/stars/THUDM/P-tuning-v2.svg?style=flat-square) |
| [QLoRA](https://github.com/artidoro/qlora) | Efficient finetuning approach that reduces memory usage enough to finetune a 65B parameter model on a single 48GB GPU while preserving full 16-bit finetuning task performance. | ![GitHub Badge](https://img.shields.io/github/stars/artidoro/qlora.svg?style=flat-square) |
| [TrainJudge](https://github.com/Himanshukurrey/trainjudge) | Diagnoses whether fine-tuning fits, then verifies fine-tunes on held-out task metrics and a regression suite, not training loss. | ![GitHub Badge](https://img.shields.io/github/stars/Himanshukurrey/trainjudge.svg?style=flat-square) |
| [TRL](https://github.com/huggingface/trl) | Train transformer language models with reinforcement learning. | ![GitHub Badge](https://img.shields.io/github/stars/huggingface/trl.svg?style=flat-square) |
**[⬆ back to ToC](#table-of-contents)**
@ -421,6 +575,7 @@ An awesome & curated list of the best LLMOps tools for developers.
| [JuiceFS](https://github.com/juicedata/juicefs) | A distributed POSIX file system built on top of Redis and S3. | ![GitHub Badge](https://img.shields.io/github/stars/juicedata/juicefs.svg?style=flat-square) |
| [LakeFS](https://github.com/treeverse/lakeFS) | Git-like capabilities for your object storage. | ![GitHub Badge](https://img.shields.io/github/stars/treeverse/lakeFS.svg?style=flat-square) |
| [Lance](https://github.com/eto-ai/lance) | Modern columnar data format for ML implemented in Rust. | ![GitHub Badge](https://img.shields.io/github/stars/eto-ai/lance.svg?style=flat-square) |
| [Pixeltable](https://github.com/pixeltable/pixeltable) | Declarative multimodal AI data engine for versioned tables, computed columns, and vector search. | ![GitHub Badge](https://img.shields.io/github/stars/pixeltable/pixeltable.svg?style=flat-square) |
**[⬆ back to ToC](#table-of-contents)**
@ -430,6 +585,7 @@ An awesome & curated list of the best LLMOps tools for developers.
| -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------- |
| [Piperider](https://github.com/InfuseAI/piperider) | A CLI tool that allows you to build data profiles and write assertion tests for easily evaluating and tracking your data's reliability over time. | ![GitHub Badge](https://img.shields.io/github/stars/InfuseAI/piperider.svg?style=flat-square) |
| [LUX](https://github.com/lux-org/lux) | A Python library that facilitates fast and easy data exploration by automating the visualization and data analysis process. | ![GitHub Badge](https://img.shields.io/github/stars/lux-org/lux.svg?style=flat-square) |
| [ragfresh](https://github.com/nac7/ragfresh) | A CLI that detects stale, drifted, and ghost documents in RAG vector indexes by diffing content hashes and embedding drift against the source of truth. | ![GitHub Badge](https://img.shields.io/github/stars/nac7/ragfresh.svg?style=flat-square) |
**[⬆ back to ToC](#table-of-contents)**
@ -446,10 +602,12 @@ An awesome & curated list of the best LLMOps tools for developers.
| Project | Details | Repository |
| ------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------ |
| [CocoIndex](https://github.com/cocoindex-io/cocoindex) | An ETL framework for AI that transforms data into embeddings and knowledge graphs, with incremental processing to recompute only what changed and keep indexes fresh. | ![GitHub Badge](https://img.shields.io/github/stars/cocoindex-io/cocoindex.svg?style=flat-square) |
| [Upgini](https://github.com/upgini/upgini) | Free automated data & feature enrichment library for machine learning: automatically searches through thousands of ready-to-use features from public and community shared data sources and enriches your training dataset with only the accuracy improving features | ![GitHub Badge](https://img.shields.io/github/stars/upgini/upgini.svg?style=flat-square) |
| [Feast](https://github.com/feast-dev/feast) | An open source feature store for machine learning. | ![GitHub Badge](https://img.shields.io/github/stars/feast-dev/feast.svg?style=flat-square) |
| [distilabel](https://github.com/argilla-io/distilabel) | ⚗️ distilabel is a framework for synthetic data and AI feedback for AI engineers that require high-quality outputs, full data ownership, and overall efficiency. | ![GitHub Badge](https://img.shields.io/github/stars/argilla-io/distilabel.svg?style=flat-square) |
| [FastDatasets](https://github.com/ZhuLinsen/FastDatasets) | A powerful tool for creating high-quality training datasets for Large Language Models. | ![GitHub Badge](https://img.shields.io/github/stars/ZhuLinsen/FastDatasets.svg?style=flat-square) |
| [fastdocparse](https://github.com/pranjalparmar/fastdocparse) | Extracts structured data from documents with an LLM, then grounds every field against the source text to flag hallucinated values automatically. | ![GitHub Badge](https://img.shields.io/github/stars/pranjalparmar/fastdocparse.svg?style=flat-square) |
**[⬆ back to ToC](#table-of-contents)**
@ -461,14 +619,17 @@ An awesome & curated list of the best LLMOps tools for developers.
| ------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- |
| [Comet](https://github.com/comet-ml/comet-examples) | Comet is an MLOps platform that offers experiment tracking, model production management, a model registry, and full data lineage from training straight through to production. Comet plays nicely with all your favorite tools, so you don't have to change your existing workflow. Comet Opik to confidently evaluate, test, and ship LLM applications with a suite of observability tools to calibrate language model outputs across your dev and production lifecycle! | ![GitHub Badge](https://img.shields.io/github/stars/comet-ml/comet-examples.svg?style=flat-square) |
| [ClearML](https://github.com/allegroai/clearml) | Auto-Magical CI/CD to streamline your ML workflow. Experiment Manager, MLOps and Data-Management. | ![GitHub Badge](https://img.shields.io/github/stars/allegroai/clearml.svg?style=flat-square) |
| [dstack](https://github.com/Dstack-TEE/dstack) | Open-source confidential AI framework for secure LLM deployment with data privacy, providing hardware-enforced isolation for production ML workloads. | ![GitHub Badge](https://img.shields.io/github/stars/Dstack-TEE/dstack.svg?style=flat-square) |
| [Hopsworks](https://github.com/logicalclocks/hopsworks) | Hopsworks is a MLOps platform for training and operating large and small ML systems, including fine-tuning and serving LLMs. Hopsworks includes both a feature store and vector database for RAG. | ![GitHub Badge](https://img.shields.io/github/stars/logicalclocks/hopsworks.svg?style=flat-square) |
| [OpenLLM](https://github.com/bentoml/OpenLLM) | An open platform for operating large language models (LLMs) in production. Fine-tune, serve, deploy, and monitor any LLMs with ease. | ![GitHub Badge](https://img.shields.io/github/stars/bentoml/OpenLLM.svg?style=flat-square) |
| [MLflow](https://github.com/mlflow/mlflow) | Open source platform for the machine learning lifecycle. | ![GitHub Badge](https://img.shields.io/github/stars/mlflow/mlflow.svg?style=flat-square) |
| [MLRun](https://github.com/mlrun/mlrun) | An open MLOps platform for quickly building and managing continuous ML applications across their lifecycle. | ![GitHub Badge](https://img.shields.io/github/stars/mlrun/mlrun.svg?style=flat-square) |
| [ModelFox](https://github.com/modelfoxdotdev/modelfox) | ModelFox is a platform for managing and deploying machine learning models. | ![GitHub Badge](https://img.shields.io/github/stars/modelfoxdotdev/modelfox.svg?style=flat-square) |
| [Kserve](https://github.com/kserve/kserve) | Standardized Serverless ML Inference Platform on Kubernetes | ![GitHub Badge](https://img.shields.io/github/stars/kserve/kserve.svg?style=flat-square) |
| [KubeStellar Console](https://console.kubestellar.io) | Open source AI-powered multi-cluster Kubernetes dashboard for managing LLM workloads across hybrid edge and cloud environments. GPU monitoring, benchmark streaming, real-time observability with 20+ CNCF integrations, and AI-guided cluster operations. CNCF Sandbox project. | ![GitHub Badge](https://img.shields.io/github/stars/kubestellar/console.svg?style=flat-square) |
| [Kubeflow](https://github.com/kubeflow/kubeflow) | Machine Learning Toolkit for Kubernetes. | ![GitHub Badge](https://img.shields.io/github/stars/kubeflow/kubeflow.svg?style=flat-square) |
| [PAI](https://github.com/microsoft/pai) | Resource scheduling and cluster management for AI. | ![GitHub Badge](https://img.shields.io/github/stars/microsoft/pai.svg?style=flat-square) |
| [piqc](https://github.com/paralleliq/piqc) | Open-source, read-only GPU waste scanner for Kubernetes inference clusters. Deploys in minutes, no write permissions required. | ![GitHub Badge](https://img.shields.io/github/stars/paralleliq/piqc.svg?style=flat-square) |
| [Polyaxon](https://github.com/polyaxon/polyaxon) | Machine Learning Management & Orchestration Platform. | ![GitHub Badge](https://img.shields.io/github/stars/polyaxon/polyaxon.svg?style=flat-square) |
| [Primehub](https://github.com/InfuseAI/primehub) | An effortless infrastructure for machine learning built on the top of Kubernetes. | ![GitHub Badge](https://img.shields.io/github/stars/InfuseAI/primehub.svg?style=flat-square) |
| [OpenModelZ](https://github.com/tensorchord/openmodelz) | One-click machine learning deployment (LLM, text-to-image and so on) at scale on any cluster (GCP, AWS, Lambda labs, your home lab, or even a single machine). | ![GitHub Badge](https://img.shields.io/github/stars/tensorchord/openmodelz.svg?style=flat-square) |
@ -486,8 +647,10 @@ An awesome & curated list of the best LLMOps tools for developers.
| [Airflow](https://airflow.apache.org/) | A platform to programmatically author, schedule and monitor workflows. | ![GitHub Badge](https://img.shields.io/github/stars/apache/airflow?style=flat-square) |
| [aqueduct](https://github.com/aqueducthq/aqueduct) | An Open-Source Platform for Production Data Science | ![GitHub Badge](https://img.shields.io/github/stars/aqueducthq/aqueduct.svg?style=flat-square) |
| [Argo Workflows](https://github.com/argoproj/argo-workflows) | Workflow engine for Kubernetes. | ![GitHub Badge](https://img.shields.io/github/stars/argoproj/argo-workflows.svg?style=flat-square) |
| [awaithumans](https://github.com/awaithumans/awaithumans) | One-function HITL primitive for AI agents. `await_human` blocks like a Promise; a human reviews via Slack/email/dashboard; the agent resumes with the typed response. Durable across restarts via Stripe-style idempotency keys. Temporal + LangGraph adapters included. | ![GitHub Badge](https://img.shields.io/github/stars/awaithumans/awaithumans.svg?style=flat-square) |
| [Flyte](https://github.com/flyteorg/flyte) | Kubernetes-native workflow automation platform for complex, mission-critical data and ML processes at scale. | ![GitHub Badge](https://img.shields.io/github/stars/flyteorg/flyte.svg?style=flat-square) |
| [Hamilton](https://github.com/dagworks-inc/hamilton) | A lightweight framework to represent ML/language model pipelines as a series of python functions. | ![GitHub Badge](https://img.shields.io/github/stars/dagworks-inc/hamilton.svg?style=flat-square) |
| [Heym](https://github.com/heymrun/heym) | Source-available, self-hosted AI workflow automation platform with a visual canvas for agent, RAG, and tool-using workflows. Includes MCP support, evals, traces, and cost tracking. | ![GitHub Badge](https://img.shields.io/github/stars/heymrun/heym.svg?style=flat-square) |
| [Kitaru](https://github.com/zenml-io/kitaru) | Durable execution layer for AI agents. Checkpoints, replay, resume, and observability primitives that make agent workflows persistent and replayable — no graph DSL required. | ![GitHub Badge](https://img.shields.io/github/stars/zenml-io/kitaru.svg?style=flat-square) |
| [Kubeflow Pipelines](https://github.com/kubeflow/pipelines) | Machine Learning Pipelines for Kubeflow. | ![GitHub Badge](https://img.shields.io/github/stars/kubeflow/pipelines.svg?style=flat-square) |
| [LangFlow](https://github.com/logspace-ai/langflow) | An effortless way to experiment and prototype LangChain flows with drag-and-drop components and a chat interface. | ![GitHub Badge](https://img.shields.io/github/stars/logspace-ai/langflow.svg?style=flat-square) |
@ -598,15 +761,22 @@ An awesome & curated list of the best LLMOps tools for developers.
| Project | Details | Repository |
| --------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------- |
| [AI Agent Token Cost Calculator](https://github.com/tinyopsstudio/ai-agent-token-cost-calculator) | Browser-only calculator for estimating monthly AI agent token spend, avoidable cost, and optimization payback. [Live calculator](https://tinyopsstudio.com/ai-agent-token-cost-calculator) | ![GitHub Badge](https://img.shields.io/github/stars/tinyopsstudio/ai-agent-token-cost-calculator.svg?style=flat-square) |
| [Entroly](https://github.com/juyterman1000/entroly) | Information-theoretic context optimization proxy. Cuts LLM token costs by 70–95% with zero accuracy loss using greedy submodular knapsack maximization. | ![GitHub Badge](https://img.shields.io/github/stars/juyterman1000/entroly.svg?style=flat-square) |
| [FeatherCNN](https://github.com/Tencent/FeatherCNN) | FeatherCNN is a high performance inference engine for convolutional neural networks. | ![GitHub Badge](https://img.shields.io/github/stars/Tencent/FeatherCNN.svg?style=flat-square) |
| [Forward](https://github.com/Tencent/Forward) | A library for high performance deep learning inference on NVIDIA GPUs. | ![GitHub Badge](https://img.shields.io/github/stars/Tencent/Forward.svg?style=flat-square) |
| [LangWatch](https://github.com/langwatch/langwatch) | LangWatch Optimization Studio is your laboratory to create, evaluate, and optimize your LLM workflows using DSPy optimizers | ![GitHub Badge](https://img.shields.io/github/stars/langwatch/langwatch.svg?style=flat-square) |
| [lean-ctx](https://github.com/yvgude/lean-ctx) | Context runtime and MCP server that reduces AI coding agent token costs via session caching, AST-aware compression, and shell output patterns. [Website](https://leanctx.com) | ![GitHub Badge](https://img.shields.io/github/stars/yvgude/lean-ctx.svg?style=flat-square) |
| [LLMSlim](https://github.com/Thanatos9404/llmslim) | Open-source context compression for RAG and agents: selects query-relevant units under a hard token budget (lexical, or an optional 22M cross-encoder), with a reproducible benchmark against LLMLingua-2 and LongLLMLingua. | ![GitHub Badge](https://img.shields.io/github/stars/Thanatos9404/llmslim.svg?style=flat-square) |
| [NCNN](https://github.com/Tencent/ncnn) | ncnn is a high-performance neural network inference framework optimized for the mobile platform. | ![GitHub Badge](https://img.shields.io/github/stars/Tencent/ncnn.svg?style=flat-square) |
| [PocketFlow](https://github.com/Tencent/PocketFlow) | use AutoML to do model compression. | ![GitHub Badge](https://img.shields.io/github/stars/Tencent/PocketFlow.svg?style=flat-square) |
| [subagent-tuner](https://github.com/Kruhlikau/subagent-tuner) | Claude Code mod for steering the cost and quality of coding-agent subagents: a Manual/Auto band, a pick pane, and a saved default that picks the model and effort per subagent, so reviews stay on a strong model and searches on a cheap one. | ![GitHub Badge](https://img.shields.io/github/stars/Kruhlikau/subagent-tuner.svg?style=flat-square) |
| [TensorFlow Model Optimization](https://github.com/tensorflow/model-optimization) | A suite of tools that users, both novice and advanced, can use to optimize machine learning models for deployment and execution. | ![GitHub Badge](https://img.shields.io/github/stars/tensorflow/model-optimization.svg?style=flat-square) |
| [TNN](https://github.com/Tencent/TNN) | A uniform deep learning inference framework for mobile, desktop and server. | ![GitHub Badge](https://img.shields.io/github/stars/Tencent/TNN.svg?style=flat-square) |
| [optimum-tpu](https://github.com/huggingface/optimum-tpu) | Google TPU optimizations for transformers models | ![GitHub Badge](https://img.shields.io/github/stars/huggingface/optimum-tpu.svg?style=flat-square) |
| [agent-opt](https://github.com/future-agi/agent-opt) | Automated optimization engine for improving agent workflows using feedback-driven iterative refinements. | ![GitHub Badge](https://img.shields.io/github/stars/future-agi/agent-opt?style=flat-square) |
| [cap-evolve](https://github.com/skillberry-ai/cap-evolve) | Optimize agentic capabilities with agents. Reflective optimizer for AI agent prompts, tool code, MCP tool surfaces, and skill packages; learning from evaluation traces, val-only significance gate with a sealed test split scored once. Adapters for τ²-bench, SWE-bench, SkillsBench. Zero runtime deps, Apache-2.0. | ![GitHub Badge](https://img.shields.io/github/stars/skillberry-ai/cap-evolve.svg?style=flat-square) |
| [reliopt](https://github.com/obielin/reliopt) | Reliability-constrained, multi-objective optimization for LLM/agent programs — contract-gated Pareto frontiers instead of a single blended score. | ![GitHub Badge](https://img.shields.io/github/stars/obielin/reliopt.svg?style=flat-square) |
**[⬆ back to ToC](#table-of-contents)**
@ -628,6 +798,7 @@ An awesome & curated list of the best LLMOps tools for developers.
| Project | Details | Repository |
| ------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ |
| [Awesome AI Coding Sandboxes](https://github.com/fhiltscher/awesome-ai-coding-sandboxes) | Security-posture-first list of sandboxes for running AI coding agents' code, ranked by isolation, egress control and secrets handling | ![GitHub Badge](https://img.shields.io/github/stars/fhiltscher/awesome-ai-coding-sandboxes.svg?style=flat-square) |
| [Awesome Argo](https://github.com/terrytangyuan/awesome-argo) | A curated list of awesome projects and resources related to Argo | ![GitHub Badge](https://img.shields.io/github/stars/terrytangyuan/awesome-argo.svg?style=flat-square) |
| [Awesome AutoDL](https://github.com/D-X-Y/Awesome-AutoDL) | Automated Deep Learning: Neural Architecture Search Is Not the End (a curated list of AutoDL resources and an in-depth analysis) | ![GitHub Badge](https://img.shields.io/github/stars/D-X-Y/Awesome-AutoDL.svg?style=flat-square) |
| [Awesome AutoML](https://github.com/windmaple/awesome-AutoML) | Curating a list of AutoML-related research, tools, projects and other resources | ![GitHub Badge](https://img.shields.io/github/stars/windmaple/awesome-AutoML.svg?style=flat-square) |
@ -639,6 +810,7 @@ An awesome & curated list of the best LLMOps tools for developers.
| [Awesome Open MLOps](https://github.com/fuzzylabs/awesome-open-mlops) | This is the Fuzzy Labs guide to the universe of free and open source MLOps tools. | ![GitHub Badge](https://img.shields.io/github/stars/fuzzylabs/awesome-open-mlops.svg?style=flat-square) |
| [Awesome Production Machine Learning](https://github.com/EthicalML/awesome-production-machine-learning) | A curated list of awesome open source libraries to deploy, monitor, version and scale your machine learning | ![GitHub Badge](https://img.shields.io/github/stars/EthicalML/awesome-production-machine-learning.svg?style=flat-square) |
| [Awesome Tensor Compilers](https://github.com/merrymercy/awesome-tensor-compilers) | A list of awesome compiler projects and papers for tensor computation and deep learning. | ![GitHub Badge](https://img.shields.io/github/stars/merrymercy/awesome-tensor-compilers.svg?style=flat-square) |
| [Best of Open-Source AI](https://github.com/archestack/best-of-ai) | A ranked list of open-source AI apps, model servers, gateways, vector databases and observability tools, scored from GitHub data. | ![GitHub Badge](https://img.shields.io/github/stars/archestack/best-of-ai.svg?style=flat-square) |
| [kelvins/awesome-mlops](https://github.com/kelvins/awesome-mlops) | A curated list of awesome MLOps tools. | ![GitHub Badge](https://img.shields.io/github/stars/kelvins/awesome-mlops.svg?style=flat-square) |
| [visenger/awesome-mlops](https://github.com/visenger/awesome-mlops) | Machine Learning Operations - An awesome list of references for MLOps | ![GitHub Badge](https://img.shields.io/github/stars/visenger/awesome-mlops.svg?style=flat-square) |
| [currentslab/awesome-vector-search](https://github.com/currentslab/awesome-vector-search) | A curated list of awesome vector search framework/engine, library, cloud service and research papers to vector similarity search. | ![GitHub Badge](https://img.shields.io/github/stars/currentslab/awesome-vector-search.svg?style=flat-square) |