Open-Source AI Agent Review

A structured comparative review of open-source AI agent projects, covering general-purpose assistants, coding agents, and agent frameworks.

What this is

An independent, code-level assessment of how different open-source agent projects are built, what they can do, and how they compare. Each agent is reviewed at a specific commit, with findings backed by evidence from the source code.

This is not a benchmark or a ranking. It's a technical review aimed at helping developers and researchers understand the landscape.

Agents reviewed

Category A — General-Purpose Agents

Agent	Language	Repo
OpenClaw	TypeScript	openclaw/openclaw
NullClaw	Zig	nullclaw/nullclaw
NanoClaw	TypeScript	qwibitai/nanoclaw
Nanobot	Python	HKUDS/nanobot
Hermes Agent	Python	NousResearch/hermes-agent
QM	TypeScript	yc-software/qm

Category B — Coding Agents

Agent	Language	Repo	Status
CLIO	Perl	SyntheticAutonomicMind/CLIO
Codex CLI	Rust	openai/codex
Gemini CLI	TypeScript	google-gemini/gemini-cli
Aider	Python	Aider-AI/aider
Cline	TypeScript	cline/cline
OpenHands	Python	OpenHands/OpenHands
Goose	Rust	block/goose
SWE-agent	Python	SWE-agent/SWE-agent	Maintenance mode
Plandex	Go	plandex-ai/plandex	Dormant since Oct 2025
Pi	TypeScript	badlogic/pi-mono
Oh My Pi	TypeScript + Rust	can1357/oh-my-pi
Plimsoll	Python	EdwardAThomson/plimsoll	Corpus author's own project (disclosed in review)

Category C — Agent Frameworks/Libraries

Agent	Language	Repo	Status
memU	Python	NevaMind-AI/memU
GBrain	TypeScript	garrytan/gbrain
AutoGPT	Python + TypeScript	Significant-Gravitas/AutoGPT
LangGraph	Python	langchain-ai/langgraph
CrewAI	Python	crewAIInc/crewAI
AutoGen	Python + C#	microsoft/autogen	Maintenance mode — see MAF
Microsoft Agent Framework	Python + C#	microsoft/agent-framework
Pydantic AI	Python	pydantic/pydantic-ai
Dorfl	TypeScript	wighawag/dorfl	No license file

Comparisons

Tier 1 — Identity Cards — Facts: repo metadata, languages, size, community
Tier 1 — Overview Tables — Side-by-side comparison tables
Tier 2 — Capabilities — Architecture, LLM integration, tools, security
Tier 3 — Opinions — Code quality, maturity, innovation, red flags

Harness leaderboard (in progress)

Separate from the code-level reviews above, we're running a controlled empirical
comparison: many agent harnesses on the same 20 SWE-bench Lite
bug-fix tasks, with the same fixed model (deepseek/deepseek-v4-pro via OpenRouter),
scored the same way (the official SWE-bench scorer). Holding the model and tasks
constant isolates the harness's contribution, most published comparisons vary both.

Current standing: mini-SWE-agent is the baseline at 10/20 resolved. Six
more agents (Hermes, Goose, Pi, OpenClaw, NullClaw, Nanobot) are containerized and
pass the smoke gate; full runs are in progress.
Want to help run it? See the collaborator runbook,
it walks through setup and running a chunk end-to-end with your own OpenRouter key.
Harness behavior findings — how each harness
behaves with a fixed model (edit mechanism, verification, failure modes) and what it
implies for harness design. The behavioral payload, beyond the scores.
Design + status: leaderboard plan,
harness registry,
results.

Recommendations

See RECOMMENDATIONS.md for practical guidance:

Which agent to use by organisation type (enterprise, SME, solo, researcher)
Adoption strategy (evaluate, pilot, scale)
Risk matrix (vendor lock-in, security, sustainability, licensing)
Quick decision tree

Methodology

See METHODOLOGY.md for the full review process, tier definitions, and per-agent checklist.

Review date

Reviews were conducted between 2026-04-10 and 2026-04-21. Each review notes the specific commit hash assessed. Findings reflect the state of the code at that point in time.

Contributing

If you spot an error or think a finding is unfair, please open an issue. We aim to be factual and even-handed.

License

Review content is released under CC BY 4.0. The reviewed agent codebases are subject to their own licenses.

agent-reviews

Specs

README