EARLY ACCESS · Open-source GenAI for serverless

Infrastructure for
autonomous systems.

Substrai is an open-source framework ecosystem bridging LLM reasoning and production-grade orchestration. Built for reliability, observability, and deterministic scaling.

Explore the ecosystemView on GitHub
substrai@prod ~

❯ pip install lambdallm

# Setting up serverless LLM orchestration...

[info] Installing lambdallm + dependencies (4.8MB)

[ok] Core framework ready

[ok] DynamoDB state adapter configured

[ok] Bedrock model router initialized

→ Ready to deploy to Lambda · see docs at substrai.dev/frameworks/lambdallm

7
Repositories
MIT
Open License
100%
Serverless
AWS
Native

Core Frameworks

One stack. Five primitives.

Each repository solves one hard problem in the agentic stack — composable, independently deployable, MIT-licensed.

VIEW ALL ON GITHUB
v1.9.0

lambdallm

/ Orchestration

Serverless-native LLM orchestration for AWS Lambda. Sub-5MB core with cold-start-optimized execution and DynamoDB state persistence.

→CI/CD pipeline with pytest + coverage
→DynamoDB session state store
→Response streaming via Lambda API
→Structured output with Pydantic validation
→Input sanitization & injection defense
→Cost-aware model routing with fallback
→Integration tests with moto for DynamoDB
→Retry strategy with exponential backoff
→Request/response middleware pipeline
PythonAWS Lambda
v1.0.0

guardrailgraph

/ Safety

Composable AI safety pipelines for production GenAI. Graph-based validation with PII detection, toxicity scoring, and audit trails.

→CI/CD with pytest, mypy, ruff
→Async pipeline execution (asyncio.gather)
→Check result caching with content-hash
→Pipeline execution metrics (p50/p95/p99)
→FedRAMP compliance pack
→Webhook alert notifications (Slack, PagerDuty)
→Guardrail bypass mode with audit logging
→Hallucination detection via source grounding
→Multi-language toxicity detection (ES/FR/DE/JA)
PythonBedrock Guardrails
v1.0.0

costsentinel

/ Cost

Real-time GenAI cost governance and budget enforcement. Per-endpoint tracking with automatic model downgrade on budget breach.

→CI/CD with full test suite
→DynamoDB state backend (atomic counters)
→Environment-specific config overrides
→Version consistency enforcement
→Routing engine wired to middleware
→Rate limiter wired to interceptor flow
→Circuit breaker wired to middleware
→Anomaly detection wired for real-time alerts
→CLI deploy command for infrastructure provisioning
PythonCloudWatch
v1.2.0

promptops

/ Ops

Infrastructure-as-Code for prompt management. Git-based versioning, A/B testing, and automated regression testing for prompts.

→CI/CD with pytest + TypeScript tests
→Prompt diff engine (semantic change detection)
→Prompt linting rules for anti-patterns
→Response cache with TTL + version invalidation
→OpenTelemetry tracing for invocations
→Prompt migration CLI for version upgrades
→Prompt dependency graph (circular detection)
→Golden dataset snapshot testing with diffs
→Adversarial prompt test generator
PythonS3
v1.1.0

agentdeploy

/ Deploy

Zero-to-production agent deployment for AWS. Handles IAM, networking, monitoring, and canary deployments with rollback.

→CI/CD with pytest + integration tests
→WebSocket streaming for real-time responses
→Health check endpoint & readiness probes
→Graceful shutdown with request draining
→Agent composition (multi-agent orchestration)
→Per-tenant API key management with rotation
→Request rate limiting (sliding window)
→Session persistence with configurable TTL
→Conversation memory strategies (buffer/summary/hybrid)
PythonCDK
v1.2.0

evalforge

/ Eval

Automated LLM evaluation pipeline generator. Describe your use case and get complete evaluation infrastructure with metrics, synthetic test data, and drift detection.

→CI/CD with pytest and coverage
→Parallel metric execution (ThreadPool)
→Custom metric plugin system (@metric)
→HTML report generation with SVG charts
→Dataset versioning with content-hash
→CI/CD quality gate (exit code 0/1)
→Evaluation result comparison between runs
→LLM-powered synthetic test data generator
→Factual consistency metric (NLI-based scoring)
PythonStep Functions

Open Source

Community contributions.

Active contributions to industry-defining AI and cloud-native projects.

LlamaIndex✓ Merged

Fix AgentWorkflow state mutation leaks

★ 49.7k
LangChain● Open

Forward token details in streaming events

★ 137.7k
HuggingFace● Open

Fix consecutive system messages crash

★ 27.6k
MCP SDK● Open

Respect server capabilities in client

★ 23.1k
Apache Airflow● Open

Add BedrockRerankOperator for RAG

★ 45.6k
OpenTelemetry● Open

Fix Kafka partition recording bug

★ 780+
OpenSSF● Open

Model Signing spec governance update

★ 171
Kubernetes● Open

Add sort indicators to table headers

★ 5.2k
AWS CDK● Open

Configurable OpenSearch vector index params

★ 536

Technical Leadership

Architecture as the foundation of impact.

fig.01 — guardrailgraph topology

In the next decade, software will not be written manually — it will be orchestrated. Substrai was founded on the premise that if LLMs are the engine, the industry still lacks the chassis and the transmission.

The work here focuses on building the deterministic layers that allow probabilistic models to function safely in critical systems — formal verification, type-safe policy graphs, and reproducible runtime environments.

Open source is not just a distribution model. It is a governance strategy. By building Substrai in the open, we establish a standard for how autonomous systems should be deployed, audited, and secured.

Local-first
Develop offline. Deploy anywhere.
Verifiable
Every output is auditable.
Composable
Each primitive ships standalone.

Writing

Technical insights.

Deep dives into the engineering challenges behind agentic infrastructure.

2026-05-30Securing the AI Supply Chain: Model Signing and MCP Safety2026-05-28Contributing to LlamaIndex: Fixing State Leaks in Agent Workflows2026-05-26Building a CDK Construct for Bedrock AI Agents2026-05-20Why LangChain Fails in Lambda (And What Does)2026-05-18Cost-Aware GenAI: Model Routing for Serverless2026-05-15From Use Case to Evaluation Pipeline in 10 Minutes

Join the ecosystem

Build the substrate for the agentic future.

Star the repositories, open an issue, ship a PR — or just lurk in the Discord.

Browse the orgRead the writing