Skip to content
agentboards
Browse sections Compare
← Compare agents

Coding-agent test kit · Version 1 · October 21, 2026

Compare agents on the same work.

Try two agents on three small, deliberately broken JavaScript tasks. Each comes with public acceptance tests, a fixed prompt and a blank results template. You can inspect and run the kit locally with Node.js 22 or newer, without installing dependencies.

This is a reproducible test kit. AgentBoards has checked that the starter tasks fail and reference fixes pass. We have not run competing agents on these tasks or published a winner. Small public tests do not establish production reliability.

Your personal comparison workspace

Choose a pair, copy one task, then record what happened after running it locally. No account needed. Keep up to eight task records in this browser; clearing site data removes them. Results do not sync across devices.

Use two different agents. Setup links contain only the tool choices and task. Results and settings stay on this device and are not sent to analytics.

Fix invoice quantities

Use a fresh copy of AgentBoards Comparison Kit v1. Fix invoice.mjs only. Return the sum of priceCents × quantity. Reject values that are not nonnegative safe integers with RangeError. Keep the input unchanged. Run node --test invoice.test.mjs before and after your edit. Keep the tests unchanged. Work within ten minutes and at most three user turns. Show the diff and exact test output, including any failures.

Review their specs →

Record your observations below. Nothing is marked passed until you choose it. These are your notes, not a published benchmark or a calculated winner.

Bito AI Code Review Agent
Record settings, time and cost

Leave unknown values blank. A subscription does not establish zero per-run cost. Use the same settings and preserve test output in your local kit.

Sourcery
Record settings, time and cost

Leave unknown values blank. A subscription does not establish zero per-run cost. Use the same settings and preserve test output in your local kit.

Open shareable setup

Keep the comparison fair

  1. Start each agent and each task in a fresh copy of the kit, with a new session. Use the same files and exact task prompt.
  2. Use the same permissions and a ten-minute limit with at most three user turns. Record exact agent version, model ID, reasoning effort, operating system and any human intervention.
  3. Run the matching test command before and after the edit. Keep tests unchanged. Save the actual diff and output, including failures and incomplete attempts.
  4. Record elapsed wall-clock time and actual billed cost. Use null when cost is unknown; a subscription is not evidence of zero per-run cost. Repeat trials before drawing broader conclusions.

Three tasks you can reproduce

Task 1

Fix invoice quantities

Fix invoice.mjs only. Return the sum of priceCents × quantity. Reject values that are not nonnegative safe integers with RangeError. Keep the input unchanged.

node --test invoice.test.mjs

Checks: Quantities, empty input, zero quantity, invalid values and immutable objects.

Task 2

Fix pagination boundaries

Fix pagination.mjs only. Use zero-based page indices and a positive integer page size. Reject invalid arguments with RangeError, return an empty array beyond the last page and keep input unchanged.

node --test pagination.test.mjs

Checks: No skipped or repeated rows, the last partial page, empty input, invalid arguments and immutable arrays.

Task 3

Preserve useful URL parameters

Fix campaign-url.mjs only. Require an absolute HTTP(S) URL. Remove all utm_ query keys, ignoring case. Preserve other keys, repeated values and the hash. Return the serialized URL.

node --test campaign-url.test.mjs

Checks: Mixed query parameters, repeated tracking keys, fragments and invalid URL schemes.

Turn the results into a decision

Prioritize correct output, unchanged tests and the amount of supervision required. Compare time and cost only for runs with equivalent settings. A failed attempt is useful evidence: keep it in the record.

Request the weekly AgentBoards briefing

Optional weekly news and meaningful tool updates. This is a general briefing, not an alert for each saved item. Confirm your subscription by email.

Confirm by email for a weekly briefing on new agents and meaningful updates. Unsubscribe at any time.

Loading agent listings… Models and guides are ready.
↑↓ navigate · Enter open · Esc closeModels & guides