Self-hosted · Apache 2.0

Detect, evaluate & prevent costly AI incidents

Observability, protection & red teaming — built for RAG and agentic applications. Stop hallucinations, PII leaks, and unsafe outputs before production.

OpenTelemetry Langfuse + Grafana LiteLLM Gateway Qdrant · Ollama
The gap

Observability alone won't save you

Most teams stitch together separate tools — and critical failures slip through.

Observability

Shows what happened, but does not prevent it. Traces don't block harmful outputs.

Evaluations

Measure quality offline, but provide zero runtime protection for live agents.

Guardrails

Block narrow failure modes, but offer no release confidence or regression checks.

AgentGuard merges observability, protection, evaluation, and red teaming into one control layer.
No gaps. No costly production incidents.

Production-grade AI reliability

One platform, complete control

From runtime protection to automated red teaming.

Full-stack observability

End-to-end traces, retrieval latency, tool usage, and model behavior — all integrated with OpenTelemetry, Langfuse, Grafana, and Jaeger.

  • Traces for every RAG & agent step
  • Langfuse dashboards + Arize AX tracing
  • Real-time latency & token usage metrics

Built for real-world AI workflows

Customer support, internal copilots, agentic systems — keep incidents customer-invisible.

Customer support assistants

Reduce hallucinated pricing, refund, or policy answers before they become customer-visible incidents.

Internal knowledge copilots

Monitor retrieval quality, evaluate answer faithfulness, and reduce sensitive data exposure.

Agentic workflows

Trace tool usage, benchmark outcomes, catch costly action errors before production rollout.

Trusted by teams handling regulated, business-critical, or sensitive workflows.

Launch AgentGuard in minutes
# 1. Clone the repository
git clone https://github.com/glaborie/agentguard.git
cd agentguard

# 2. Configure environment
cp .env.example .env

# 3. Spin up the full stack (Langfuse, Qdrant, Ollama, Redis, Grafana...)
docker compose up -d

# 4. Pull local embedding model
docker compose exec ollama ollama pull nomic-embed-text

# 5. Ingest knowledge & query
pip install -r requirements.txt
python -m app.main ingest
python -m app.main query "Does the Starter plan include SAML SSO?"
→ Open WebUI: http://localhost:3100
→ Langfuse UI: http://localhost:3200
Docker-based | Self-hosted | Apache 2.0 📘 Architecture →

Apache 2.0 · Open Source

AgentGuard is built for AI engineers, platform teams, and technical product owners who need operational control, not just experimentation.

Python 3.13+
OpenTelemetry
LLM-judge guardrails