Agent Bench Scorecard

Turn coding-agent effectiveness into a comparable score on real repo tasks

Problem

Developers want to know whether their coding agent is actually getting better, but rely on vibes or model switching. People clean repos, add lint rules, and feed better context without any way to measure if it helps.

Solution

A local benchmark tool that pulls verifiable micro-tasks from a real repository, runs them across different agents or configurations, and records pass rate, time, token usage, and human interventions into a comparison report.

Users

Individual developers and small teams heavily using Claude Code, Cursor, or Copilot

MVP

Extract tasks from recent fix commits; run specified agents and collect results; local comparable scorecard; custom scoring weights; one-click shareable summary

Revenue

Free local version with paid team dashboard and longitudinal trend subscription

Why now

Coding agents are moving from novelty to daily use, yet effectiveness measurement is immature; indie developers can ship a lightweight benchmark fast