Pywen

The deciding trait is the audience. This is built first as an arena for putting different agents under identical conditions, and second as something you would actually work in, and you can feel that ordering in every part of it. For a researcher or anyone choosing between agents on evidence, that is a real and rare offering.

For daily engineering it is thin: fewer conveniences, less polish and no ecosystem around it. Pick it if you are running the comparison. Pick a maintained terminal agent if you are trying to finish a task.

Alignment is not equivalence and the row does not claim otherwise, which is where the problem starts. If the replicated agent diverges from the original in any respect, every result attributed to that agent belongs to this reimplementation instead, and no version pinning, fidelity test or divergence report appears anywhere. The original also changes without notice.

What it does right is name what it copied. A reimplementation that says whose behaviour it is approximating is at least falsifiable by anyone willing to diff the two.

1. Holding the tool layer constant is exactly right. Most published agent comparisons vary the scaffold and the harness together and then attribute the difference to the agent, which is not a controlled experiment. Fixing the interface removes one confound. 2. Trajectory recording provides the audit trail such a comparison requires.

2. None of it has been used, or at least no result is reported: no task set, no participants, no numbers. A fair arena with no matches played is a proposal for a methodology rather than a contribution to one.

Laboratory-origin tooling has a distinctive curve. Quality is often high because the authors are pursuing a question rather than a quarter, and continuity is poor because the people move when the project that funded them ends. There is no company to acquire and no revenue to protect, which simplifies the analysis considerably.

Moat: none, though the ecosystem position it claims would be valuable if anyone adopted it. Likely path: a paper, then quiet. Position: cite it, borrow the interface idea, do not build a workflow on top of it.

Those three words are the reason this got a second reading from me. An approval step and a recorded trajectory are the two artefacts a security questionnaire asks for, and finding them stated in a project of this size is unusual enough to note.

They are still local artefacts. There is no console aggregating them across sixty machines, no identity integration, no retention policy and no unattended run to measure, so what exists is the raw material for governance rather than governance. Not yet, and the condition would be shipping those trajectories somewhere we control.

The licence is the good part and the skills system is the second good part: reusable capability injection means my own additions are units the platform understands rather than patches I maintain against a moving tree. Python, readable, forkable.

Everything else is closed by omission. No install command anywhere, so I clone and work it out. No MCP client, so the servers I run are invisible. No local endpoint, so a platform built for reproducible comparison sends every token to an API whose weights change under it, which is a strange foundation for reproducibility.