by Mattbusel · MCP Server · ★ 50
tokio-prompt-orchestrator Production-grade, multi-core Tokio orchestration for LLM inference pipelines. Five-stage bounded-backpressure DAG with deduplication, circuit breakers, rate limiting, prompt injection/jailbreak detection, provider arbitrage (cheapest provider meeting your latency SLA), adaptive worker pool sizing, and an optional autonomous self-improving control loop. Supports Anthropic, OpenAI, llama.cpp, vLLM, and any custom backend. Exposes REST, WebSocket, SSE, MCP (Claude Desktop), Prometheus metrics, and OpenTelemetry distributed tracing.
| Stars | 50 |
| Forks | 5 |
| Language | Rust |
| Category | MCP Server |
| License | MIT |
| Quality Score | 63.7379725213552/100 |
| Open Issues | 2 |
| Last Updated | 2026-03-23 |
| Created | 2025-10-07 |
| Platforms | mcp, rust |
| Est. Tokens | ~20012k |
Explore other popular mcp server tools:
tokio-prompt-orchestrator is Multi-core, Tokio-native orchestration for LLM pipelines.. It is categorized as a MCP Server with 50 GitHub stars.
tokio-prompt-orchestrator is primarily written in Rust. It covers topics such as agent, async, backpressure.
You can find installation instructions and usage details in the tokio-prompt-orchestrator GitHub repository at github.com/Mattbusel/tokio-prompt-orchestrator. The project has 50 stars and 5 forks, indicating an active community.
tokio-prompt-orchestrator is released under the MIT license, making it free to use and modify according to the license terms.