Early preview. Purchases are test/demo and no real money moves. Join the waitlist

Kessance
Back to feed
DataEval setModuleSatisfaction guarantee

Tool-Use Eval Set

What it’s for: Measure how reliably an agent picks and calls the right tool on messy, realistic tasks, all written by hand rather than taken from user logs.

Satisfaction guarantee: Use licenses can be refunded for any reason within 7 days, even after download, once per creator. Build licenses are not covered. The refund ends the license.

Sold by @priya.dataDeclared by creator
Made withClaude Opus 5Claude Opus 5.5

Code excerpt

Eval set900 rows
taskgold_trace
Refund order 4417 if it arrived lateorders.get → refunds.create
Move Friday’s 1:1 to the next free slotcalendar.list → calendar.update
Which invoices over $5,000 are unpaid?invoices.search

From the free row sample

def grade(trace, gold):
    """Score a tool-use trace against the gold trace."""
    correct = sum(1 for a, b in zip(trace, gold) if a.tool == b.tool)
    extra = max(0, len(trace) - len(gold))
 
    # Penalize unnecessary calls more than missing ones.
    score = correct / len(gold) - 0.15 * extra
    return round(max(score, 0.0), 3)
Partial code: 132 more lines in this file, full source after purchase.

Versions

Earlier versions sell at their own price.

  1. v1.0

    Jul 30, 2026

    Initial 600 tasks.

    Not for sale

Tool-Use Eval Set

v1.1 · Use license

Dossier

What it is, how it was made, and what you need to run it.

Data details

Rows & schema
900 rows; task, tools[], gold_trace, rubric
Origin
Tasks and gold traces written by the creator, with Claude Opus 5 and Claude Opus 5.5 used to draft variants; every task reviewed by two annotators. No tasks come from real user logs.
Source license
Original work by the creator, sold only under Kessance’s Use and Build licenses; no separate open license.
Cutoff date
Tasks written July to September 2026; tool schemas current as of September 2026
Row sample
50 tasks with gold traces in sample/first-50-rows.jsonl

Process notes

Approach
Tasks and gold traces written by hand in Python; Claude Opus 5 and Claude Opus 5.5 drafted variants, and two annotators reviewed every task.
Prompts and settings
Visible after purchase

What’s included

The exact contents you receive. Each file is listed with its size and fingerprint, so you can check that your copy is complete.

PathSizeSHA-256
  • README.md6.2 KBd5db19d8
  • LICENSE.md1.8 KB33bb6e0b
  • DATASHEET.md11 KB652969d7
  • schema.json3.8 KB5335c6ec
  • data/151 MB2 files
    • train.parquet115 MB5cfd52c4
    • test.parquet35 MBff8fc1aa
  • sample/51 KB1 file
    • first-50-rows.jsonl51 KBd3d55a8c
7 files in 2 foldersTotal 151 MB

Text files and datasets in your copy carry your order’s marks, so their fingerprints differ from those shown here; your copy’s license file lists its exact fingerprints. Images, video and other binary files arrive unchanged.

License in plain words

For the Use license you have selected.

You can

  • Use the work in your own projects and in work you deliver to clients
  • Edit and adapt it to fit those projects
  • Use it commercially, for example in ads, products you sell or paid client work

You can’t

  • Resell, share or give away the work itself, even if you modified it
  • Ship it inside a product that other people download or resell as a template
  • Claim you created it or use it to train a competing AI model

Questions before you buy

Ask @priya.data anything about this work. Answers are public.

0/800

  • Robin S. · 14d

    Is there a datasheet that says how the data was made?

    Priya RamanCreator · 14d

    Yes. Every version ships a DATASHEET.md covering origin, how it was generated, what the AI did, human review and allowed uses.

More from @priya.data

Visit storefront
Priya Raman@priya.data · Sep 1Data

v2: 48,200 synthetic US-style receipts with line-item boxes. No real shoppers, no real stores.

Synthetic data48,200 rows
merchanttotal
Juniper Street Deli$21.86
Bayview Hardware$94.30
Prairie Fresh Market$63.41

From the free row sample

Synthetic US Retail Receipts, Labeled

Synthetic data · Project

48,200 AI-generated receipts from 1,900 fictional merchants, labeled for merchant, date, line items, tax and total, with line-item boxes.

What it’s for: Train or test receipt extraction on realistic US-style receipts without a single real shopper or store in them.

Nano Banana 2Claude Opus 5
SampleSatisfaction guarantee
$390
Available
Priya Raman@priya.data · 9dData

120,000 support conversations and not one real customer in them.

Synthetic data120,000 rows
turns[]intent
I was charged twice for the Pro plan this month.billing_dispute
How do I add a teammate to my workspace?add_user
The CSV export stops at 10,000 rows.bug_report

From the free row sample

Synthetic Support Conversations

Synthetic data · Module

120,000 multi-turn conversations across 40 SaaS product types, with intent, sentiment and resolution labels.

What it’s for: Fine-tune or evaluate support bots without exposing a single real customer conversation.

Claude Sonnet 5.5DeepSeek-V4-Pro
SampleSatisfaction guarantee
$149
Available
Priya Raman@priya.data · 6dData

Eval Forge builds an eval set from a brief, and I review a sample before it submits. This one grades SQL by the rows it returns, not by its text.

Eval set1,200 rows
questiondifficulty
Which station started the most rides in July?easy
Patients who missed two or more appointments this yearmedium
Revenue by genre for games released before 2024hard

From the free row sample

1,200 questions over three invented schemas (a bike-share, a clinic scheduler and a game store), graded on the rows a query returns rather than on its text.

What it’s for: Test how well an agent writes SQL against three fictional company schemas, with a gold query and expected result for every question.

Claude Opus 5.5DeepSeek-V4-Pro
SampleSatisfaction guaranteePublished by an agentEval Forge · for @priya.data
$39
Available

Related works

Mira Okafor@mira.codes · 11dCode

v2.0 is out. Rebuilt the forecasting views with Claude Opus 5.5 and then spent a week on the empty states nobody screenshots.

Ledger billing overview: MRR, ARR, churn and ARPA cards, a 12-month MRR chart, MRR movements and recent invoices

Ledger: Finance Dashboard Kit

Project kit · Project

What it’s for: Launch a polished SaaS billing dashboard in an afternoon instead of a sprint.

Next.jsTypeScriptTailwindPostgres
SampleSatisfaction guarantee
$129
Available
Mira Okafor@mira.codes · 8dCode

Fourteen pages, two languages, light and dark, and a Lighthouse score of 95 or better on every one. Northwind is the marketing site I kept rebuilding for clients, finished once.

Northwind home page in dark mode: the headline Know what to reorder before you run out above a demand forecast dashboard

Northwind: Complete SaaS Website

Page or landing · Project

What it’s for: A finished marketing site for a SaaS product: fourteen pages in English and Spanish, light and dark, ready to rename and deploy.

Next.jsTypeScriptTailwindMDX
SampleSatisfaction guarantee
$79
Available
Devon Hale@devon · 8dCode

Most startup ledgers I have audited were a balance column and good intentions. Tally is the boring version: double-entry journals, idempotent writes and 420 tests that prove every journal balances.

src/journal.ts

export interface Line {  account: AccountId;  amount: bigint; // minor units, positive = debit  currency: Currency;} // Every journal must balance per currency, and a retried request must not post twice.export async function postJournal(db: Db, input: { idempotencyKey: string; lines: Line[]; memo?: string }) {  for (const [currency, total] of sumByCurrency(input.lines)) {    if (total !== 0n) throw new UnbalancedJournal(currency, total);  }

Tally: Double-Entry Ledger Service

Backend or API · System

What it’s for: A small, correct ledger service for apps that move money: double-entry journals, idempotent writes and balances you can trust at any point in time.

TypeScriptNode.jsPostgresOpenAPI
Sample
$149
Available
Devon Hale@devon · 6dCode

Early access: webhook verification for 14 providers, plus a replay console that has already saved me twice.

src/verify.ts

import { createHmac, timingSafeEqual } from 'node:crypto'; export function verifySignature(secret: string, payload: string, header: string) {  const [timestamp, signature] = parseHeader(header);  if (Date.now() / 1000 - timestamp > 300) return false;   const expected = createHmac('sha256', secret)    .update(`${timestamp}.${payload}`)    .digest('hex');   return timingSafeEqual(Buffer.from(expected), Buffer.from(signature));}

Webhook Relay API

Backend or API · System

What it’s for: Receive, verify, retry and replay webhooks from any provider with one small service.

Node.jsHonoRedisPostgres
Sample
$49
Early access
Kenji Morita@kenji.builds · 12dCode

Pocket Delve is done: 30 floors, 24 enemies, 60 items, and builds for web, iOS and Android. Swap the sprites with the reskin guide and it is your game.

Pocket Delve key art: a hooded adventurer with a lantern at the top of a spiral stone stair that winds down into a dark well full of glowing eyes, with gold coins and a chest on the steps and the POCKET DELVE title above

What it’s for: A complete roguelite for web and mobile, finished and tested, that you can reskin and publish under your own studio name.

TypeScriptCanvasCapacitor
Sample
$79
Available
Ada Nwosu@ada.agents · 12dAgents

Ported to Claude Opus 5.5 and re-ran our eval: 31% fewer escalations. Tagging moved to Gemini 3.8 Flash, so a ticket still costs about a cent.

agent.yaml

name: support-triagemodel: claude-opus-5-5tagging_model: gemini-3.8-flashmax_steps: 6 tools:  - read_ticket  - add_internal_note  - post_slack_message # Escalate anything mentioning refunds over $500.policies:

Support Triage Agent

Agent config · System

What it’s for: Sort, tag and draft replies for incoming support tickets so humans only handle the hard ones.

Agent configPrompt packEvals
Sample
$49
Available