Coding-agent test kit · Version 1 · October 21, 2026
Compare agents on the same work.
Try two agents on three small, deliberately broken JavaScript tasks. Each comes with public acceptance tests, a fixed prompt and a blank results template. You can inspect and run the kit locally with Node.js 22 or newer, without installing dependencies.
This is a reproducible test kit. AgentBoards has checked that the starter tasks fail and reference fixes pass. We have not run competing agents on these tasks or published a winner. Small public tests do not establish production reliability.
Your personal comparison workspace
Choose a pair, copy one task, then record what happened after running it locally. No account needed. Keep up to eight task records in this browser; clearing site data removes them. Results do not sync across devices.
Use two different agents. Setup links contain only the tool choices and task. Results and settings stay on this device and are not sent to analytics.
Fix invoice quantities
Use a fresh copy of AgentBoards Comparison Kit v1. Fix invoice.mjs only. Return the sum of priceCents × quantity. Reject values that are not nonnegative safe integers with RangeError. Keep the input unchanged.
Run node --test invoice.test.mjs before and after your edit. Keep the tests unchanged. Work within ten minutes and at most three user turns. Show the diff and exact test output, including any failures.
Start each agent and each task in a fresh copy of the kit, with a new session. Use the same files and exact task prompt.
Use the same permissions and a ten-minute limit with at most three user turns. Record exact agent version, model ID, reasoning effort, operating system and any human intervention.
Run the matching test command before and after the edit. Keep tests unchanged. Save the actual diff and output, including failures and incomplete attempts.
Record elapsed wall-clock time and actual billed cost. Use null when cost is unknown; a subscription is not evidence of zero per-run cost. Repeat trials before drawing broader conclusions.
Three tasks you can reproduce
Task 1
Fix invoice quantities
Fix invoice.mjs only. Return the sum of priceCents × quantity. Reject values that are not nonnegative safe integers with RangeError. Keep the input unchanged.
node --test invoice.test.mjs
Checks: Quantities, empty input, zero quantity, invalid values and immutable objects.
Task 2
Fix pagination boundaries
Fix pagination.mjs only. Use zero-based page indices and a positive integer page size. Reject invalid arguments with RangeError, return an empty array beyond the last page and keep input unchanged.
node --test pagination.test.mjs
Checks: No skipped or repeated rows, the last partial page, empty input, invalid arguments and immutable arrays.
Task 3
Preserve useful URL parameters
Fix campaign-url.mjs only. Require an absolute HTTP(S) URL. Remove all utm_ query keys, ignoring case. Preserve other keys, repeated values and the hash. Return the serialized URL.
Prioritize correct output, unchanged tests and the amount of supervision required. Compare time and cost only for runs with equivalent settings. A failed attempt is useful evidence: keep it in the record.