OpenAI Evals
Framework and registry for evaluating LLMs and LLM systems.
Visit OpenAI Evals →Related Evals tools
- SWE-bench — Benchmark of real GitHub issues for evaluating coding agents.
- Inspect AI — UK AISI framework for LLM and agent evaluations with sandboxed tools.
- promptfoo — CLI for testing prompts, agents and RAG with assertions and red-teaming.
- DeepEval — Pytest-like LLM evaluation framework with agentic metrics like task completion.
- Langfuse — Open-source LLM observability: traces, evals and prompt management for agents.
- Arize Phoenix — Open-source tracing and evaluation for LLM apps built on OpenTelemetry.
All Evals tools → · Best memory and evaluation tools for AI agents