A report card for AI coding
Tally reads your coding-agent sessions on your machine, redacts them, and grades you on the six habits that separate people who get twenty dollars of work back per dollar of tokens from people who get two. Then it prices every deliverable three ways, so you can say what the bill bought.
The one chart
Every deliverable Tally finds is priced at the hours a person in the right role would have needed, three ways: a freelance quote range, your internal loaded cost, and an agency rate. The AI spend sits beside them on the same scale.
| Way to get the same 140 hours | Cost |
|---|---|
| AI spend on deliverables, exact from token usage | $410 |
| Freelance quote range, marketplace rates with client fee | $9,800 – $21,000 |
| Internal loaded cost, your rate for the role | $16,500 |
| Agency rate | $34,000 |
Illustrative: one month, one engineer, 140 hours of senior-engineer work delivered. Not a customer's data. The dot is what the tokens cost, priced exactly from usage at the vendor's published rates. Value is an estimate of labor replaced, shown as a range with its assumptions stated, never presented as revenue.
How it works
It finds your session transcripts and splits them into episodes: one prompt and everything the agent did until the next. Each episode is priced exactly from its token usage.
Everything is parsed and redacted locally before anything is uploaded. Secrets, emails, phone numbers and card numbers become placeholders, and raw transcripts and source files never leave. People's names are removed on receipt, before any analysis runs.
The service groups episodes into tasks and deliverables, grades you on six dimensions, and tells you which two habits would move the grade. Your card has a link you can share, or not.
Six dimensions
Each dimension answers one question from the evidence in your sessions. Cut points are fixed and printed on every card until a peer benchmark exists, so a grade means the same thing for everyone.
How much value does a dollar of tokens return?
A ≥ $20 back per $1 · B ≥ $10 · C ≥ $4
Put AI on bigger, checkable deliverables; small chat-style asks cost nearly as much per prompt and return far less.
How much does the agent get done per prompt, and is big work right-sized?
A ≥ 12 actions per prompt · B ≥ 8 · C ≥ 5 · large tasks that keep being abandoned lower it
Delegate whole features, investigations and documents in one brief, then let the agent run. Anchors: about 5 actions per prompt for novices, 12 for experts.
How well do requests say what, where, and what done looks like?
A ≥ 2.2 of 3 on the rubric · corrections ≤ 5% · plans first ≥ 30% of tasks
State the goal, the files in scope, the constraints, and how you'll judge it done. Ask for a plan first on anything large.
Does work finish with proof, not just a judgment?
A ≥ 33% of tasks verified by tests, a commit or your confirmation · B ≥ 25% · C ≥ 15%
Ask for the check in the brief: tests must pass, show the diff, compare against the old output. Anchors: 15% verified for novices, 33% for experts.
How much spend leaks, and does troubled work get rescued?
A ≤ 5% leaked and ≤ 7% abandoned · recovery ≥ 60% of troubled tasks
When a task hits an error, stay with it and say what to check next. Anchors: novices abandon 19% of troubled sessions, experts 5–7%.
Is routine work on frontier models, and is oversight spent where it matters?
A ≤ 8% of spend routable to a cheaper model · B ≤ 15% · C ≤ 25%
Set subagents and well-specified chores to a cheaper model; auto-approve low-risk work and interrupt on exceptions.
Overall grade is the average of the six on a four-point scale: A at 3.5 and above, B at 2.75, C at 2.
The research behind the grades
Where a published number exists, we use it as the anchor for a grade. Where none does, the cut point is stated on the card as a default until a peer benchmark replaces it.
Agents shift worker effort from implementation to supervision, which especially benefits verifiable work and expert workers.Suproteem K. Sarkar, University of Chicago, AI Agents and Higher-Order Work (2026), from Cursor usage data
About five agent actions per prompt in novice sessions against about twelve in expert sessions, across every kind of work, in Anthropic's analysis of roughly 400,000 Claude Code sessions.
Anthropic, Agentic coding and returns to expertiseSuccess is split into judged and verified, where verified means passing tests, a commit or explicit confirmation. Verified success runs about 15% for novices and 33% for experts; that is where our A and C lines sit.
Anthropic, Agentic coding and returns to expertiseNovices abandon about 19% of troubled sessions, experts 5 to 7%, and experts recover troubled sessions to a verified result several times as often.
Anthropic, Agentic coding and returns to expertiseExpertise in these studies is read from how precisely people instruct, what verification they ask for, and how they correct. Experienced Cursor users ask fewer questions and are more likely to develop a plan in the first message.
Sarkar, AI Agents and Higher-Order WorkExperienced users auto-approve more and interrupt more: oversight by exception rather than approval of every step. We show both rates beside the routing grade.
Anthropic, Measuring agent autonomyTask value is benchmarked against freelance marketplace rates in the same research, and in Anthropic's internal study 27% of AI-assisted work would not have been done otherwise. We report labor replaced and work enabled as two numbers, and human-expert time as the unit, as METR does.
Anthropic, How AI is transforming work at Anthropic · METR, time horizonsAlso drawn on: Baumann et al., SWE-chat (6,000 public agent sessions; users push back in 44% of turns and 44% of agent code survives to a commit), which is why corrections are measured and why code survival is next on the list.
What executives are saying
The same three anxieties recur in the room: they cannot see the spend, they cannot prove the value, and the bill moves faster than the budget.
One of my engineers spent $40,000 on tokens last month, and I genuinely don't know whether I should stop him or should I go and tell everyone else to be like him.Vitaly Gordon, CEO, Faros AI · TechCrunch, June 2026
In April and May, I started hearing from companies: ‘Oh my god, we are 3x over our entire 2026 token budget and it's only April.’J.R. Storment, Executive Director, FinOps Foundation · TechCrunch, June 2026
Now the conversations are about, ‘hey, we're spending so much. What visibility do you have? What auditability do you have? What token controls do you have? What is the efficiency of your models?’Alexander Embiricos, Head of Enterprise, OpenAI · TechCrunch, June 2026
The best ROI comes from moving the broad middle from low to moderate usage, not pushing heavy users higher.Nicholas Arcolano, Head of Research, Jellyfish · TechCrunch, June 2026
For teams
A company gets the same grades for every person, ranked by efficiency and never by volume, plus the ledger a finance team can book against. Every value number carries its range, its confidence and the counterfactual it assumes.
Grades and task mix for everyone, with a coach, clone, lift or cap read for each person and the two habits that would move their grade.
Spend to date, run-rate, projected month end, days of runway, and the days that departed from a person's or project's baseline.
Every shipped outcome next to its freelance range, internal loaded cost and agency rate, with a frozen baseline and period-over-period change on every number.
Which prompts and delegated runs a cheaper model would have handled, what that saves, and a policy simulator to move the threshold and watch the number change.
A CSV by cost centre, person and month that finance can book, and ticket-anchored deliverables where a tracker exists.
Claude Code, Codex CLI, OpenCode and OMP map onto the same episode shape, so grades and spend compare across tools and across teams.
for individuals. Your own card, your own data.
for companies. Talk to us about scope and pricing.
Privacy, briefly
People's names are removed on receipt, before analysis, and the stored copy of your upload is overwritten with the redacted version. Redaction is best-effort by pattern and by judgment. You can delete everything on request. The full policy, in plain English: Privacy policy.
Questions
Cost is exact: every prompt is priced from its token usage at the vendor's published rates, and the report says whether that is list or billed. Value is an estimate: a model reads a summary of each task and judges the hours a person in the right role would have needed. We show it as a range, carry a confidence on every judgment, flag low-confidence items for a human, and let teams calibrate the hours against a labeled sample. Independent studies find single-digit throughput gains from AI coding tools, not the marketed multiples, and we would rather be believed than impressive.
The redacted episodes you upload (prompt text with placeholders, token counts, model and tool names, file names, timestamps, your machine account name) and what we compute from them: tasks, deliverables, judgments, scores and reports. We never receive raw transcripts, source code files or secrets, so we cannot keep them.
Yes. Ask from your account or by email and everything tied to you is deleted within 30 days, including cached judgments. De-identified, aggregated benchmark statistics (for example, the median correction rate across all users) are retained and cannot be traced back to you.
Claude Code, Codex CLI and OpenCode today; OMP is experimental. Each adapter maps that tool's session format onto the same episode shape, so grades stay comparable across tools. If you use something else, tell us on the signup form.
Free for individuals: your own card, as often as you like. Companies get per-person views, pacing, chargeback and the team dashboard as a paid pilot; request one and we will talk about scope and pricing.
Six dimensions, each anchored where possible to a published number: delegation to actions per prompt (about 5 novice, 12 expert), verification to verified-success rates (15% novice, 33% expert), hygiene to abandonment rates (19% versus 5–7%), all from Anthropic's study of 400,000 Claude Code sessions; oversight patterns from Anthropic's agent-autonomy research; specification and plan-first behavior from Sarkar's Cursor study. Where no number exists the cut point is a stated default, printed on the card. Every task is judged by a model, and on our own data five independent re-judgments agreed on 90–95% of tasks and produced identical grades.
For each task: expected hours saved, times the probability a person would have done the task at all, times a blended hourly rate for the role that would have done it, times the probability it actually landed. Tasks are then grouped into deliverables and judged once at the deliverable level, so hours are not double counted. The freelance, internal and agency prices on the ladder are those hours at published rate bands you can replace with your own.