PromptMeter — a health check for prompt engineering

Reproducible adversarial tests show developers how a prompt really performs across models, beyond gut feel

Problem

Developers increasingly rely on prompts to drive agents that write, review, and debug code, but they only have a vague sense of prompt quality. They don't know how much worse a prompt gets on a cheaper fast model, or which parts are load-bearing across models.

Solution

A local prompt evaluation bench: paste a prompt, pick two or three models, and the tool generates adversarial test cases (edge cases, ambiguous instructions, malformed input), runs each model, and produces a difference report highlighting weak spots in the prompt and capability gaps between models.

Users

Coding-agent developers using multiple models, indie developers downgrading from frontier to cheaper models, and tech leads who need maintainable team prompts.

MVP

Paste a prompt and up to three model endpoints; ten built-in adversarial test templates; a side-by-side model difference table; prompt improvement suggestions; fully local execution with keys never leaving the user's machine.

Revenue

Free tier with daily test limits; paid subscription for more models, test templates, and report export; optional usage-based API billing.

Why now

Multi-model workflows and model downgrading are now common, yet lightweight prompt quality evaluation remains scarce, leaving an opening for indie developers.