El Profesor

Aider documents its design. 1. Context comes from a repository map built from a tree-sitter parse and ranked by graph centrality, so the files shown to the model are chosen by reference structure, not embedding similarity. 2. Edit formats are chosen per model, which is why the polyglot leaderboard exists: it measures which model follows which format. 3. The SWE-bench Lite 26.3% and full 18.9% figures are self-reported from 2024 with the models of the time and read as history.

The consequence: Aider's numbers are comparable across models and not against other agents. The observation: publishing the harness is worth more than the score, and most vendors here publish neither.

1. Offloading tool results to files and referring to them by path is the correct treatment of large observations, because it decouples what the agent can access from what it must currently hold. 2. Summarisation of long threads is applied as a documented stage rather than left to a prompt instruction, which makes the loss of detail a design decision instead of an accident.

2. Checkpointing under the graph runtime means a run has resumable states, so failure analysis starts from a recorded position. 4. No evaluation is published and none is claimed.

1. This is the correct direction of dependency. The agent is not asked to judge whether code is unsafe; a deterministic scanner makes that call and the agent is handed the result, so the finding is reproducible by anyone with the same tool and version. 2. Converting findings into ordered tasks turns a report into a plan without inserting a generative step between them.

2. No benchmark is published and none is needed, because the capability being claimed is integration, and integration is demonstrated by naming the three programs it integrates.

The cleanest methodology here. 1. One tool, bash, executed with subprocess.run, each action independent, so there is no hidden state between steps. 2. A linear history; every step appends, nothing is summarized, so the transcript is the full context. 3. No use of the model's tool-calling interface, which removes a vendor-specific variable. 4. The above 74% on SWE-bench Verified is reported with Gemini 3 Pro on the bash-only setup.

The consequence is that the score is attributable: change the model and nothing else moves. The observation: when the scaffold is this small, the score is the model's.

1. The knowledge pipeline handles ingestion, chunking and retrieval as its own concern rather than as nodes scattered through a graph, which keeps the retrieval configuration inspectable in one place. 2. Model management is separate again, so provider changes do not touch application logic. 3. Completed workflows are published as OpenAPI endpoints, which turns the visual artefact into a service any client can call.

That third property is the architecturally important one: the canvas is the authoring environment, not the runtime interface. Verification remains whatever node the author adds. No benchmark is published.

1. Context is supplied through typed dependency injection, so what an agent may read is declared in a signature rather than assembled inside a prompt. 2. Actions are tools whose arguments are validated before the function body executes. 3. Verification is doubled. A structured result is parsed and checked against its declared type before reaching the caller, and the evaluation package asserts behaviour the way a test suite asserts code.

No benchmark is published, which is consistent. A library whose entire claim is type discipline gains nothing from a leaderboard, and reviewing a project that declines to offer one is a small relief.

The design is unusually legible. 1. Agent work is committed to its own branch, so inspection uses tools the reader already trusts rather than a bespoke viewer. 2. Acceptance is split into two distinct operations, one that brings the history across and one that brings only the resulting state, and distinguishing them is a real semantic choice most tools elide. 3. Checking out an agent's environment lets a human continue from where it stopped.

The design is agnostic about which agent produced the commits, so it should survive both model and vendor churn without modification.

1. There is an eval framework for the reviews themselves, which is the rarest thing on this board: a tool that treats its own output as something to be measured rather than asserted. A review agent without one is a source of opinions with no error rate. 2. That framing also makes the Skill the unit of evaluation, so a team can improve one rule without disturbing the others.

2. No results from that framework are published, so the apparatus exists and the numbers do not. Still, the apparatus is the harder half.

1. Every ACP message can be inspected in a traffic log. That is an unusual and valuable property: the protocol between the client and the agent is normally opaque, and here the JSON-RPC exchange is available to the person debugging it, which makes a disagreement about whose fault a failure is answerable rather than rhetorical. 2. It also makes the extension a usable instrument for studying agent behaviour, which is not what it was built for.

2. There is nothing to benchmark, since the extension contributes no capability of its own. Its correctness is protocol conformance and that is checkable.

1. Emitting telemetry in a standard format rather than a bespoke log means the questions that matter, how often a tool is called, how many tokens a prompt consumes, how long a turn takes, are answerable with instrumentation an organisation already runs. 2. That is the difference between a vendor's published number and a number a reader can generate on their own workload.

2. Modelling the whole thing on an established editor protocol is the second sound decision: the failure modes of a stdio server negotiating with a client are twenty years catalogued rather than newly invented.

1. The layering is the contribution. Provider differences are normalised into a neutral stream at the bottom, the middle owns messages, tools, events and the loop, and only the top layer knows it is a coding tool. Each layer can be read without the others, which is the definition of a successful decomposition.

2. That structure also makes the middle reusable for agents that are not about code. 3. No evaluation is published and none is needed, since the claim is pedagogical and the evidence is the source.

1. A percentage accompanied by its denominator, its repetition count and its inference cost is a measurement; a percentage on its own is a marketing claim. This is the former. 2. Repeating each of the eighty-nine tasks five times is the minimum defence against variance in a stochastic harness, and reporting the trial total rather than a task pass rate lets a reader recompute the figure.

2. The evaluation is run continuously across model families rather than pinned to one, which measures the harness instead of the model. That is the correct unit of analysis for a tool of this shape.

1. The interesting number is published rather than implied: a core prompt under 270 tokens and a fixed harness around 1,000. Stating that figure invites verification, which is the point of stating it. 2. Project context is layered in on request rather than gathered in advance, so the window holds what the task needed instead of what the tool guessed.

2. That design makes the token cost of the scaffolding a known constant instead of a mystery, and a known constant is what lets a reader reason about cost per task at all.

1. The oracle is external to the model. A verdict is the exit status of a command the user wrote, not an assessment produced by the system being assessed, which is the distinction most tools in this category collapse. 2. A baseline mode exercises the checks themselves before any work is attempted, so a suite that passes trivially is caught in advance.

2. Every attempt is recorded to a structured log, which makes a run reproducible and a failure rate computable. This is what evaluation discipline looks like applied to a product.

1. Treating model choice as a property of the task rather than of the session is the correct decomposition. Exploration is cheap and repetitive, synthesis is expensive and rare, and vision is a separate capability entirely; paying one price for all three has always been an accident of interface design rather than a decision.

2. What is not described is how the roles hand off: what an exploration model returns to the main agent, and in what form, determines whether the split saves cost or loses information.

3. No evaluation accompanies the arrangement, so the saving is asserted. The design will survive model changes, which is its strongest property.

1. This is the strongest efficiency argument on the board, and it does not depend on a benchmark, because the saving follows from the shape of the protocol rather than from a measurement somebody chose. Ten conditional steps expressed as one program is ten fewer inferences, whatever the model.

2. The trade is legibility: a tool call is inspectable before it executes, and a program is not, so what is bought in latency is paid in reviewability. 3. The project states the mechanism and leaves the reader to price that exchange, which is honest.

1. This is the most defensible design decision on the board. Because the orchestration layer records and replays deterministically, a failed run can be re-executed to the exact step that went wrong, which is the property every other agent system substitutes logs for. 2. It also survives model changes, since the replay concerns control flow rather than generation.

2. No evaluation is published, and the architectural claim does not require one, because reproducibility is verifiable by using it.

1. Verification is part of the loop rather than an afterthought: the agent runs the test suite, compiles, or loads the page before reporting. 2. More importantly, when no check can be run it says so with an explicit marker rather than presenting an unchecked result in the same voice as a checked one.

2. That distinction, between demonstrated and asserted, is the one this whole category blurs, and seeing it encoded in output formatting is worth more than a benchmark. No benchmark is published, and after that, none is missed.

1. Making the system prompt visible is the choice most products decline, and it changes the epistemics of the whole tool: a surprising answer becomes a question about an input the reader can see. 2. Context is an editable item rather than an accumulated transcript, so the model's view is constructed on purpose.

2. No evaluation is offered and none is claimed, which is consistent, because the argument here concerns control rather than capability. The design should also outlast model changes for the same reason: nothing in it depends on a particular model behaving in a particular way.

1. This is the strongest reproducibility property on the board. Because every side effect passes through the runtime as a recorded host call, a past execution is fully determined by its log, and replay requires no provider, no network and no spend.

2. The consequence is that regression testing an agent becomes ordinary software testing, which is the thing this field has been unable to do. 3. No published evaluation exists, and none is needed for the claim, because a reader can verify it by running a checkpoint twice.

1. A graph whose nodes may be hand-written code or model calls, so the deterministic parts stay deterministic and the model is confined to the nodes that need it. 2. Durable execution: state is persisted at each step and a run resumes from the last checkpoint after a crash, which makes a long run reproducible from its history. 3. Streaming. 4. Memory in working and cross-session tiers.

The consequence is that verification can be placed at any node as ordinary code, which is the property most agent frameworks lack. The documentation calls itself very low-level, which is accurate and rare.

1. Control flow is delegated to the model, which chooses among tools rather than traversing a graph an engineer drew, so behaviour improves with model capability and cannot be constrained the way an explicit graph can. 2. Determinism is recovered through hooks and steering, which are the documented points where code re-enters the loop. 3. Multi-agent structure is offered as named patterns, swarm and agent-as-tool, rather than as bespoke wiring.

No benchmark accompanies the SDK. The architecture is an explicit bet that model capability keeps rising, which is a defensible bet and worth naming as one.

1. The governance layer intercepts model calls and tool-protocol traffic at the wire rather than instrumenting an agent, so any agent placed in a workspace is observable without modification. That is the correct architectural choice and it is why this design survives the agents themselves being replaced. 2. Environments declared as code make a run reproducible, which is the precondition for any claim about what an agent did.

No benchmark is published, and none would be meaningful for a substrate. The observation: the design's value grows as the agents inside it churn, which is a rare property.

1. The diagram is a front end over a Python runtime, so nodes are executed objects rather than generated code, which keeps behaviour identical between the editor and a deployment. 2. Wiring covers models, vector stores and tools, making retrieval an explicit node rather than hidden middleware. 3. A finished graph can be published so its steps become callable tools for outside clients.

No benchmark accompanies any of this, so throughput and accuracy claims are absent rather than unverified. One observation: a builder whose steps become somebody else's tools has inverted the usual dependency.

OpenHands is the one tool on this board whose headline claim can be checked. The 71.8 percent on SWE-bench Verified with GPT-5 is a public leaderboard submission dated 2025-08-07 and checked by the benchmark maintainers, so it is documented rather than self-reported, and comparable with every other entry that went through the same harness.

Two caveats. 1. A scaffold-plus-model result attributes the score to the pair; swap the model and the number is no longer yours. 2. Verified is the curated subset, so the figure says nothing about the harder residue. The observation: the scaffold is the reproducible half, and the half most vendors decline to publish.

1. Routing is a design decision made visible. A dedicated agent selects among named specialists, so the attribution question, which component produced this answer, has an answer, and a user can correct a bad route rather than rewriting a system prompt. 2. Separating the framework from the product built on it means the architecture is reusable outside this editor, which is a strong signal that the boundaries are real.

2. No evaluation is published and no capability claim needs one, since the product asserts structure rather than performance.

The notable decision is packaging. 1. An agent definition is stored and retrieved through any OCI registry, which means immutable tags, content addressing and the mirroring infrastructure organisations already operate, none of it built for this purpose. 2. Structured tools for memory, retrieval and language-server queries are provided rather than improvised per project, so capability is declared instead of prompted.

The design should survive model turnover, because the file names a provider and nothing in the packaging depends on which one. No evaluation is published, which is appropriate for a substrate rather than an agent.

1. Publishing the data path is the rarest thing in this catalogue. Most listings describe features and leave the reader to infer where the request went; this one states the destination, states the exception, and states that the exception is shown in the interface before it takes effect. 2. Requiring an explicit dialog before reading a credential source turns an implicit capability into an observed event.

2. None of it is verified by a third party, and disclosure is not proof. It is, however, a falsifiable claim, which is more than an unfalsifiable one.

1. Distinguishing the observability channel from the context channel is the correct decomposition, and it is the one most delegation implementations collapse: they either hide the subordinate work entirely or paste all of it back into the parent window. 2. Keeping the transcript visible to a human while withholding it from the parent prompt preserves auditability without paying for it in tokens.

2. The efficiency claim implied by that choice is not quantified anywhere, so the reader has a principled design and no measurement of what it saves.

1. Naming the six operations an agent may perform on files, and putting them over a virtual filesystem scoped to a conversation, converts an unbounded capability into an enumerable one. That is the correct direction: capability as a closed set rather than as a shell.

2. A plan artefact means intent is inspectable before execution rather than reconstructed afterwards. 3. No evaluation is published, so whether these defaults improve task completion over an unstructured agent is unmeasured, and the design argument stands on its own.

1. Replay-based durability is a well-studied execution model with a documented constraint, and applying it here forces a separation most agent frameworks never make: deterministic control flow on one side, non-deterministic model output recorded as an activity result on the other. 2. That separation is exactly what makes a run reconstructible rather than merely retried.

2. It also means the framework's correctness depends on authors respecting the constraint, which is a discipline the type system cannot enforce. No evaluation is published, and the claim under test would be an operational one rather than a capability score.

1. Making the investigating subagents read-only and concurrent is the correct decomposition: retrieval parallelises safely, mutation does not, and the design encodes that distinction instead of relying on instruction. 2. Bounding each delegated task means the coordinator's context grows by results rather than by transcripts, which is the difference between a system that degrades slowly and one that degrades suddenly.

2. No evaluation is published and no capability claim requires one, since the argument being made is about structure.

1. The unit of work is an issue, and a working tree is created for it, which means context is scoped by the thing being fixed rather than by whichever files were open. That is a principled choice: the boundary of the task and the boundary of the checkout coincide. 2. Diffs and pull-request context sit in the same view as the terminal that produced them.

2. Verification therefore happens where the work happened, without an export step, which removes the most common place for a reviewer to lose the thread. No benchmark is published, and none is claimed, so nothing here invites comparison to a leaderboard.

1. A durable ordered event log is the difference between an agent run you can study and one you can only remember. Replay from a database makes a past run reproducible as an object, which is the precondition for any serious analysis of failures. 2. Naming two analytic stores rather than one proprietary sink keeps the data in systems that already have query tools.

2. A machine-readable specification means the contract can be validated automatically instead of being trusted, which is rare on this board.

1. The phases are separated and named: plan, then execute, then verify, each with its own workers, which means a failure can be attributed to a stage instead of to the run. 2. Branch advance verification is the detail worth noting, because it checks that work produced movement rather than trusting that a process exited cleanly.

2. Retry backoff is described as deterministic, which makes a failing run reproducible, and reproducibility is the property that separates an engineered scheduler from a hopeful one. No benchmark is published, and none is needed for a claim of this shape.

1. Profiling and evaluation are treated as first-class rather than as a logging afterthought, which is the correct emphasis for a field where most capability claims are anecdotes. 2. Being framework-agnostic makes comparisons across implementations possible, and comparability is the scarce good here.

2. No numbers are published about the toolkit itself: no overhead figure, no accuracy claim for the tracing, no ablation of the accelerators. A measurement layer asking to be trusted on assertion is an irony the documentation does not address.

1. Conditional activation is the right answer to the problem every framework eventually hits: static instructions grow monotonically, and each addition dilutes the rest. Making the set a function of context bounds that growth by construction. 2. Rendering an agent's reasoning as an observable computation graph gives the same treatment to control flow, since the path taken becomes an object rather than a narrative.

2. Neither claim is accompanied by measurement. The reader gets two principled mechanisms and no figure for what either saves or catches.

1. The trust boundary is drawn where it should be. Orchestration travels over gRPC with mutual TLS, and terminal bytes travel through a relay that holds no state, so the component with the widest reach carries the least to steal. 2. One Rust core serves the web, desktop and iOS clients, which means three surfaces cannot drift into three behaviours.

2. Each pod holds private credentials in its own working tree, so the isolation claim is enforced by construction rather than asserted in prose. No benchmark accompanies any of this, and none is claimed, which is the correct pairing.

1. Subagent identity is runtime-owned rather than model-supplied: synchronous children, asynchronous children and workflow children all receive continuation ids from the runtime. That places the addressing scheme outside the text the model produces, which removes an entire class of hallucinated-reference failures. 2. The interface contracts are declared stable, so an external client can be written against them without tracking releases.

2. No evaluation is offered, and the design does not require one, since every claim here is about structure rather than capability. Structure of this kind usually outlives the model it was built for.

1. Tool definitions are checked by the compiler. A macro turns an annotated struct into the tool schema, so the mismatch between a declared parameter and the code behind it becomes a build error rather than a runtime surprise, which is a class of failure most frameworks handle with validation at call time. 2. That moves verification earlier than any prompt-level guard can reach.

2. The cost is that a tool cannot be defined at runtime, and no evaluation is offered on any of it. This is a design argument, and the argument is sound on the axis it chooses.

1. Choosing an established formalism means the failure modes are already catalogued in forty years of literature, which is a better foundation than an invented control flow. 2. Composition is declared through task assignment rather than through prompt convention, so the structure of a system is visible in code instead of inferred from behaviour.

2. No benchmark accompanies any of this, and the authors make no capability claim that would need one. That is the correct pairing: an architectural argument, published as an architectural argument.

1. This is the architecturally serious answer to the hallucination problem. The model does not rewrite text; a deterministic recipe transforms a parsed tree that retains formatting and type information, and the language model's role is selecting and shaping the recipe. 2. The output is therefore reproducible, which almost nothing else on this board can claim.

2. No benchmark is published, and none is needed for determinism, though the selection step, where the model actually decides, is the part left unmeasured.

1. Context is modelled explicitly. A session exists in original, working and offload states, and compaction proceeds through six described levels, which means degradation is staged and legible instead of arriving as a single truncation nobody can reconstruct afterwards. 2. Long-term memory is extracted asynchronously into an external store and injected per user, separating retention from the request path.

2. Sub-agents receive their own windows, so delegation is also context isolation. No evaluation is published and none is claimed.

1. Checkpoints are git-based, so the record of what an agent changed is the same record everything else in the repository uses, and reverting is an operation the team already understands rather than a proprietary undo stack. That is the correct choice and it is not the common one.

2. The loop interceptor runs after inference, which places the intervention point where a caller can inspect what the model produced before anything acts on it. Between those two, an embedder can both prevent and undo. 3. No evaluation accompanies the library, and none is needed for claims about structure.

1. A single named state passed between nodes is the decision that makes everything else analysable. What a node may read and what it may write become properties of a type rather than of a prompt, which means a reviewer can reason about information flow without executing anything at all.

2. Persistence is a separate module rather than an assumption, so a graph can be durable or not without changing shape. 3. No evaluation is published and none is needed: this is a port of a specified model, and the interesting question is fidelity to that specification, which is answered by reading.

1. A declared maximum number of turns is the simplest correct answer to non-termination, and it is stated as a parameter rather than buried as a default, which means the bound is a design decision the author makes rather than one they inherit. 2. Guardrails placed on both ends of the loop give validation a defined position instead of scattering checks through tool bodies.

2. In a statically typed language those boundaries carry compile-time meaning, which is the strongest form this idea takes. No evaluation is published, and the claims are structural.

1. Separating orchestration from inference is the principled decision here. A routing choice made by an expression is reproducible and inspectable; one made by a model is neither, and the difference compounds across long pipelines. 2. Parallelism is declared in two documented shapes, static groups and for-each fan-out, so concurrency is a property of the definition rather than an emergent behaviour.

2. No evaluation accompanies any of this, and none is owed. The project asserts a structure, not a capability, and a structural argument is assessed by reading it.

Zed publishes no benchmark, which spares the reader a methodology audit. The principled property is separation. 1. The Agent Client Protocol hosts Claude, Codex and Gemini as external agents in the same panel, so the editor's value does not depend on its own loop being best. 2. Subagents get their own context window, which bounds what one task can read. 3. Edits are text replacements the user sees before they land, so verification is visual and immediate rather than deferred to a test run.

The observation: an editor that can host its competitors has hedged its own agent, which is either humility or strategy, and works either way.

Two distinct mechanisms are worth separating. 1. The workflow builder takes a description and emits a graph, so the model output is a reviewable artefact a human edits before anything runs; the risk of a bad generation is bounded by that review. 2. A chat hub fronts multiple models, which is routing, not orchestration, and should not be confused with the first.

Guardrails are named as a capability but not specified, so the reader cannot tell whether they are output filters, schema validation or approval gates. No benchmark accompanies any of it.

Two design decisions are worth naming. 1. Forking a session turns the usual complaint about non-determinism into an experiment: the same prefix, two continuations, compared directly, which is how one would actually study a prompt change rather than argue about it. 2. Output is available as structured data in three shapes, so the agent's result can be consumed by a program instead of read by a person.

No evaluation is published and the project is two months old, so the documentation is young. The architecture nonetheless assumes machine consumption, which is the assumption that ages best.

The structure is layered and documented. 1. A unified LLM API abstracts the providers. 2. An agent loop sits on it. 3. A TUI and the coding-agent CLI sit on the loop, so each layer is usable without the one above. 4. Sessions are trees: a conversation can branch, be navigated, and be compacted, with abandoned branches summarised rather than dropped, which is a principled answer to context growth.

No benchmark is published and no capability claim is made that would need one. The observation: a session tree is version control for a conversation, and the rest of the board still uses a scrollback.

1. Context gathering is explicit: the chat buffer holds the references the user added, so the model's inputs are visible text rather than a retrieval decision made elsewhere, and the token cost of a turn can be read off the screen. 2. Edit application rewrites the buffer directly, keeping the editor's own undo as the rollback mechanism, which is a better safety net than most bespoke diff appliers.

No benchmark is claimed, and the documentation site is the only source for any of this. The observation: designs that reuse the editor's primitives tend to survive model changes, because they never depended on the model.

The design is a thin harness on a thick runtime. 1. The agent loop runs on LangGraph, which supplies persistence, durable execution and human-in-the-loop as inherited properties rather than features of the harness. 2. Tools are plain functions whose docstrings become the schema, so the model's view of a tool is whatever the author wrote. 3. Middleware wraps the loop for guardrails, retries, routing and per-tool policies.

Item 3 is where verification lives, which means it lives wherever the developer puts it. Nothing is verified by default. The documentation is complete on mechanics and silent on measured outcomes; there is no benchmark to audit, which is at least an honest silence.

1. Two mechanisms do the work here. An audit trail described as tamper-evident makes after-the-fact reconstruction meaningful rather than merely available, and an escalation queue gives an autonomous run a defined point at which a person must act. 2. Together they turn unattended execution into something reviewable, which is the precondition for trusting it at all.

2. No evaluation is published and none is claimed, so the argument is structural. The structure is the strongest on this row.

1. The architecture answers the question most autonomous products dodge, which is how the agent knows it succeeded. A full development environment means the build, the test suite and the dependencies are present, so correctness is demonstrated by running things rather than inferred from a diff.

2. That also makes the run reproducible for a reviewer, since the environment is defined rather than incidental. 3. No benchmark is published, so throughput and success rate remain unstated, and the architectural argument stands alone.

The interesting property is the bound rather than the roles. Separating execution, planning and research lets each sub-agent carry only the material its task requires, so a long research detour does not enlarge the context every subsequent edit is charged against. Cost per useful edit stays closer to flat as a session lengthens, which is the failure mode of single-context agents.

What the record does not describe is how the plan is validated before execution begins, or how results return to the planner. No benchmark is published.

1. The number is reproducible because the conditions are stated. Anyone can send the same greeting and count, which is the difference between a measurement and an advertisement, and it is a low bar most efficiency claims fail to clear. 2. The comparison figure is not produced the same way.

2. The range attributed to other harnesses is quoted without a method, a version or a named tool, so the ratio a reader takes away is one careful number divided by an estimate. The measured half is exemplary. The comparative half is ordinary marketing arithmetic.

1. The integration contract is the terminal itself: any agent that runs in one runs here, with no adapter per vendor and no capability negotiation. That is a deliberate lowest common denominator, and it is why the supported list is open-ended rather than a table someone has to maintain. 2. The cost of that choice is symmetrical. Nothing inside a pane can be inspected structurally, because the interface is a stream of characters.

2. No evaluation exists and none is owed. The claim is compatibility, and compatibility is demonstrated by the absence of an integration step.

1. Consensus is a defensible proxy. Two models trained on overlapping corpora may agree on a wrong answer, so agreement bounds independent error only to the extent that the reviewers are genuinely independent, and no analysis of that independence is offered. 2. Disagreement is treated as a signal, which is the sounder half of the design.

2. The measurement that would settle this is available and unperformed: a set of pull requests with known defects, reviewed by one model and by the panel, reporting how many defects each caught. Until that exists, the claim is that several opinions beat one, which is plausible and not evidence.

1. The capability model is additive rather than subtractive. Web access is not present until a bridge is enabled, so the default configuration has a smaller surface than the documented one, which is the correct order and the opposite of what most products in this category do.

2. The wider architectural bet is that capability should arrive through published protocols rather than bespoke integrations, which means the tool ages with the ecosystem instead of against it.

3. No evaluation is published and none is claimed.

1. Naming a reranker as a first-class model type is the notable decision. Most frameworks treat retrieval as embedding plus similarity and leave reranking to whoever notices the recall problem; declaring it in the type system means the two-stage design is the default rather than an optimisation somebody adds later.

2. Subagents and skills are described as framework features, not as prompt conventions, which puts composition in code where it can be tested. 3. No evaluation accompanies any of it. The project makes structural claims and publishes structural evidence, which is self-consistent.

This is the design most agent products describe and few encode. Phases are named and ordered, and the implementation phase repeats until the test suite passes, which makes the exit condition mechanical rather than a judgement the model makes about its own output. An approval gate stands between validation and the pull request, so a human decision is a structural element and not a setting.

No benchmark is published, and none is needed to evaluate this. The architecture is legible from the workflow definition itself, which is a stronger form of evidence than a score.

1. Keeping secrets out of the context window is a structural answer to prompt injection rather than a filtering one, and structural answers survive model changes while filters do not. 2. Isolating each tool server in a separate container makes the trust boundary match the process boundary, so a compromised server cannot read the conversation it was called from. 3. Per-conversation sandboxing gives the same property to the workspace.

2. All three are asserted in project documentation and none is accompanied by a published threat model or an audit, so a reader should record them as documented intent rather than verified behaviour.

1. This is how a number should be published. The harness is the official one, the resolved count is given as 168 of 300, the patch generation rate is separated from the resolution rate, and the configuration names the model, the month, the closed six-tool set and the absence of network access and sub-agents.

2. Separating patch generation from resolution is the detail that matters, because it distinguishes failing to produce a patch from producing one that does not work. 3. The configuration is restricted, so the figure is a lower bound on the shipped product rather than a ceiling.

1. The control flow is stated plainly: investigate, propose a plan, wait for approval, then edit, test and commit. Placing the gate at the plan rather than at each edit is a deliberate trade of granularity for tempo, and the documentation says which it chose. 2. Any turn can be undone, so the loop is reversible as well as gated.

2. Memory is separated into short-term session state and long-term cross-project state, which is a distinction most tools collapse. No benchmark is published and none is asserted.

1. Most implementations conflate these three, which is why they cannot answer what a given agent is permitted to do independently of what it happens to be doing. Separating durable identity from durable resources from ephemeral context makes each question answerable on its own. 2. The workspace is defined as both a file and an execution boundary, so the security scope and the memory scope coincide rather than cross-cutting.

2. No evaluation is published. The claim here is structural and the correct reading is that the decomposition is principled, not that it has been measured.

1. Filesystem confinement uses Landlock alongside container mounts, fixed when the sandbox is created, so a running agent cannot widen its own boundary without a restart. 2. Process limits come from the container runtime security context, with capabilities dropped at the entrypoint. 3. Egress passes a gateway operating at layer four or, with the protocol set to rest, inspecting HTTP, and each rule names which executables may reach that destination. 4. An unlisted destination is blocked and raised to the operator, whose approval survives only for that instance.

Enforcement below the agent rather than inside its instructions is the correct place for it.

1. Isolation is delegated to a primitive that already works. Worktrees give each task its own checkout with shared object storage, which is cheaper than containers and correct for the property being protected. 2. Persisting state as line-delimited records under version control means a run's history is inspected with the same tools as the code, and survives the program that wrote it.

2. There is no database and no service, so the design has no component whose absence breaks replay. 4. No evaluation is published, and the row claims none.

1. Moving approval from a terminal prompt into a callback is the change that makes an interactive agent programmable, because a decision a human makes by looking becomes a decision a program makes by policy. 2. Surfacing tool calls and approvals in the stream, rather than in a summary afterwards, means the policy sees them in time to act.

2. The design shifts the burden rather than removing it: an approval policy expressed in code is only as good as the code, and nothing offers a default worth inheriting. No evaluation accompanies any of this, and none is claimed, which for an interface specification is the correct pairing.

1. The structure is the argument. Each stage produces a document the next stage consumes, so the system's reasoning is externalised at four points rather than held in a context window, and a wrong turn is visible in a file before it becomes visible in code.

2. Test-driven construction supplies the verification the other stages lack.

3. Whether the tests are written before the implementation by the same agent that then satisfies them is the question the documentation does not answer, and it decides whether this is verification or tautology. No evaluation is published. The architecture is the most disciplined on this shelf and the least measured.

1. Orchestration is offered as named patterns rather than improvisation: sequential, concurrent, handoff, and group collaboration, each with defined control flow. 2. Execution is checkpointed, and a stored state can be revisited and re-run, which turns debugging a nondeterministic system into an experiment you can repeat. 3. Human intervention is a supported step rather than an interruption. 4. Instrumentation follows the OpenTelemetry standard, so traces are portable.

Replay from a prior state is the property research on agent behaviour actually needs, and very few of these frameworks provide it.

1. The concurrency primitives are chosen rather than improvised: per-path file locks and background subagents built on channels rather than hand-rolled promises. Locking at path granularity is the correct unit for this problem, because it permits parallelism exactly where two agents are not in conflict and serialises them precisely where they are.

2. That is a stronger guarantee than most parallel agent tools offer, and it is stated as a property rather than a hope. 3. Nothing is measured. A harness whose single stated goal is speed invites a latency figure, and none is published, which is a missed opportunity rather than a flaw.

1. Separating the process that holds state from the process that draws it is the correct decomposition. The consequence is testable: a session's lifetime is a property of the daemon, not of a window, so a crash in the interface is a cosmetic event rather than a lost run.

2. Restoring conversations after a daemon restart is the stronger claim, since it requires the transcript to be persisted rather than held in memory, and nothing describes where or in what form. 3. No evaluation accompanies any of this and none is needed: the claim is structural and can be verified by anyone in ten minutes.

1. Context is gathered explicitly. Retrieval over local files is its own tool the model invokes, so what entered the window is visible in the transcript instead of assembled invisibly. 2. Changes are applied incrementally through a patch tool, with a separate faster path for bulk edits. 3. Guidance is separated from instructions through a lessons mechanism that matches on keywords, tools and patterns.

No benchmark accompanies any of this, and the release history instead documents dated capability additions from 2023 onward. For a tool of this size that is the appropriate evidence.

1. Naming the stages is what makes this analysable. A snapshot taken at an arbitrary moment is a guess about consistency; a snapshot taken at an enumerated boundary has a stated invariant, and a reader can reason about what is true at each one. 2. Designating a single fork point rather than allowing forks anywhere is the conservative and correct choice.

2. Persistence is offered through two storage engines with different durability characteristics, and the documentation does not state which guarantees hold under each. No evaluation is published.

1. Executing in a throwaway environment makes each run independent, so a failure cannot contaminate the next task through leftover state. That property is what makes repeated attempts interpretable.

2. Running the tests before proposing the change places verification inside the loop rather than after it, which is the distinction between an agent that works and one that reports.

3. The screenshot is a visual artefact attached to the claim. It is weak evidence and it is evidence, which is more than most rows offer. 4. No evaluation is published.

1. Defining completion as a git reference is the most disciplined choice on this row. A branch exists or it does not, it can be diffed, and it survives the system that produced it, whereas a conversation log requires interpretation to decide whether anything was accomplished. 2. Layering the optional integrations above that guarantee keeps the core contract small enough to reason about when one of them fails.

2. No evaluation is published, which is consistent with a claim about mechanism rather than capability.

1. Holding the model constant and varying the harness is the right experiment, because it isolates the variable the product actually controls. Most published numbers confound the two and are therefore uninformative about the tool. 2. Using the official verifier rather than a self-authored one closes the second common escape.

2. Per-task publication is the third good decision: an aggregate hides which tasks a harness fails, and a reader comparing harnesses cares about precisely that distribution. The result is a set of numbers a sceptic can argue with, which is a lower bar than it should be and one almost nobody clears.

1. This is the correct primitive and it is under-used. Two agents editing one checkout is a race with no referee; two agents in separate worktrees cannot collide, because the isolation is enforced by the version control system rather than by the orchestrator's good intentions.

2. It also makes the result reviewable: each unit of parallel work arrives as a branch, which is the artefact a team already knows how to inspect. 3. No evaluation accompanies the design and none is needed, because the guarantee comes from a tool whose semantics are documented elsewhere.

1. Substitution is the structurally correct answer to prompt-level secret leakage. If the value is never in the context, no jailbreak, transcript or log can disclose it, which is a guarantee rather than a mitigation. 2. Enforcing at the network layer rather than inside the prompt places the control below the component that can be talked out of things, so refusal does not depend on persuasion.

2. Domain knowledge is supplied as curated rulebooks rather than assumed from pretraining, which makes the knowledge auditable.

1. Retrieval is the claim and the structure is stated: summary nodes in a high-fanout directed acyclic graph, with any earlier message reachable regardless of how many compactions have intervened. A graph with high fanout keeps path lengths short, which is the property that makes deep history cheap to reach rather than merely present.

2. There is a paper, which puts this ahead of most of the board. 3. What the paper does not appear to carry into the product is a measurement: no figure is offered for retrieval accuracy, and an architecture that guarantees a message is reachable says nothing about whether it is found.

1. This is the correct fix for a well-documented failure. An agent addressing a file by line offsets corrupts it whenever the file has moved since it was read, and the corruption is silent, which is the worst property a bug can have. Hashing the anchor converts it into a rejected operation, and a rejection is something a caller can handle.

2. Syntax-tree inspection sits beside it, so structural questions are answered by a parse rather than a regular expression. 3. Neither choice arrives with a measurement, and no benchmark of any kind is published here. Defensible from first principles, which is the weakest available form of evidence.

1. Declaring a budget before the window fills is the difference between a designed context and an emergent one, and it is the single decision that most determines cost per useful edit. 2. Naming drift as something the runtime resists implies a comparison against a stated goal at each step, which is a verification posture rather than a prompt.

2. None of it is quantified. There is no published measurement of how much a budget saves or how often drift is caught, so these are documented design commitments and not demonstrated results.

1. The concurrency model is actors, supplied by an existing library, and distribution is handled by a dataflow runtime borrowed rather than written. Both choices mean the failure modes are already documented outside this project. 2. The agent pattern is ReAct, named explicitly, which lets a reader locate the design in the literature instead of inferring it.

2. Six collaboration modes are enumerated without any guidance on selection, which is the gap: a taxonomy is not a method. No evaluation is published and none is claimed.

1. This is the right kind of borrowing. Agent frameworks routinely reinvent persistence badly, and this one declines to, taking checkpointing and state management from a system where both have been under production load for years. The agent layer is consequently small enough to reason about.

2. The cost is that the design is only available to people already operating that system, which is a narrow audience by construction. 3. No evaluation is published and none is required, because the claim is structural rather than about capability, and a structural claim is checked by reading the code.

1. This is the measurement design most harnesses omit. Recording what a run produced on the run's own object means a question such as how many attempts produced a merged change becomes answerable by query rather than by memory, across every task the system has executed since it was installed.

2. Token usage in the same record is the part that matters most, since it lets cost be attributed to an outcome instead of to a month. 3. No benchmark accompanies any of it and none is claimed, which is appropriate: the contribution is bookkeeping, and bookkeeping is verified by reading the schema.

1. Treating the computer as an interface rather than an ambient capability is the correct abstraction, and it is the one most harnesses skip: substituting the execution environment becomes a configuration change instead of a rewrite. 2. Excluding the harness's own credentials and source from the agent's view closes the most obvious self-escalation path, where an agent reads the keys that drive it.

2. No evaluation is published. The claim is architectural and can be verified by reading the protocol rather than by running a benchmark.

1. The interesting claim is not the agent loop but the bookkeeping around it: response identifiers and tool results threaded across turns, prompt ordering held apart from delivery, and replay after a dropped connection. These are the parts rewritten badly in most integrations. 2. Events are ordered and typed, so a consumer reconstructs state from the stream rather than by inference.

2. No evaluation is offered and no capability is claimed, which is the correct pairing for a component whose entire contribution is an interface.

1. The separation is unusual and principled. Signals carry events inward, actions transform state, and directives describe what should happen without doing it, so the part that decides can be exercised in a test without a network. 2. That property survives a model change, because the model sits outside the decision boundary rather than inside it.

2. No evaluation is published and there is no benchmark to audit, which is consistent with a library that makes no capability claim. The design is the claim, and it holds.

1. Every framework that attaches dozens of tools pays for their descriptions on every turn, and almost all of them ignore the cost. Grouping capabilities and deferring schema retrieval until selection converts a fixed per-turn tax into an occasional one.

2. The completion runner accounts usage across nested agent calls, so that saving is measurable rather than asserted. 3. No measurement is published, which is a pity, because this is one of the few designs here whose benefit could be shown with a single token count.

1. Built-in tools: Read, Write, Edit, Glob, Grep, Bash, WebSearch, WebFetch, so edits are explicit file operations, not diffs the caller must apply. 2. Subagents run subtasks in isolated context, which bounds what each one can confuse. 3. Hooks run user code at lifecycle points, which is where verification belongs. 4. Sessions resume or fork. Other languages drive the CLI with -p and --output-format json, so the harness is language-independent in practice.

No benchmark is published for the SDK as such. The consequence is that the loop's quality is the CLI's, whatever that is. The observation: this is the harness, published.

The architecture is documented and legible. 1. The gateway process stays on the host; native plugins and MCP tools run in-process with it. 2. Tool execution is routed to a backend chosen from Docker, Podman, SSH or OpenShell, scoped per agent, per session or shared. 3. The browser runs in its own container on a dedicated network. 4. MCP is served and consumed.

No benchmark is published, which is appropriate for a harness whose capability is whichever model it routes to. The observation: a system that lets four different backends execute the same tool call has already survived one model change and will survive the next.

The instructive property is observability. 1. Each request is recorded with the route that actually served it, so a claim about which model answered is checkable after the fact rather than inferred. 2. Latency and token counts sit beside it, which makes a comparison between two providers an experiment a reader can run on their own workload. 3. No benchmark is published, and none is needed, because the tool ships the instrument instead of the result.

That is the right order. Most projects publish a number and withhold the harness that produced it.

1. Releasing an evaluation openly is a genuine good, because a benchmark nobody can inspect is not evidence, and most vendors on this board publish scores without publishing the harness. 2. The authorship problem does not disappear by being open: a suite designed alongside a model family will reflect the tasks its designers found interesting.

2. The correct reading is that this is a useful instrument and not an independent one, and the distinction between the two is the distinction the reader has to maintain themselves.

1. Removing an explicit indexing phase is a real usability gain, since a stale index is the most common reason retrieval quality degrades without anyone noticing. 2. The cost has to appear somewhere, either as latency in the query or as work done incrementally on a file change, and the documentation does not say which.

2. Parent and child task recovery is the other structural addition, and it is the one that matters for delegation, because a subtask that fails without a defined return path leaves a parent waiting on nothing.

Three points. 1. The contract is published as a machine-readable schema in-tree with browsable docs beside the running process, so a client is generated rather than guessed. 2. Four endpoints, message list, message post, status, and an event stream, is a small surface, and small surfaces are exhaustively testable. 3. Nothing is claimed on any evaluation, which is correct for a transport.

The principled choice is refusing to own context assembly or edit application. Correctness here reduces to whether the emitted stream matches the declared schema, which a reader can verify in an afternoon rather than trust.

1. Capability is assembled rather than monolithic. Planner, Data Analyst and Security Analyst are separate documented units combined into flows, which makes the division of labour inspectable in a way a single system prompt is not. 2. The same definition runs from three entry points, so behaviour should not depend on where it was launched.

2. Nothing is published on evaluation. No benchmark, no measurement of whether decomposition beats a single pass, and no stated criterion for a completed flow. The architecture is legible; its effectiveness is asserted.

1. In-process tools remove a serialisation boundary and a subprocess from every step, which matters because the file operations dominate turn count in any real session. 2. It also removes a substitution point: a built-in cannot be swapped for an instrumented version without recompiling, so measurement of the core loop is harder for anyone but the author.

2. The external protocol is still supported for everything else, so the choice is scoped rather than dogmatic. That distinction is the part worth noting, and it is stated clearly.

1. Naming the stages is more than presentation. A loop with an explicit Observe step commits the design to checking results before continuing, which is the distinction between an agent and a sequence of hopeful calls. 2. Making the stages interruptible puts the correction point where it belongs, before the next action rather than after the transcript.

2. What Observe actually checks is not specified: whether it reads test output, compiler diagnostics or only a tool's return value changes the claim entirely. The structure is documented and the semantics are not, and no evaluation accompanies either. Auto model selection is asserted on the same terms.

1. The division is principled. A hosted index can afford to read a repository no laptop could hold, and a language server on the machine answers precisely the questions that must reflect the file as it is right now. Using one for breadth and the other for currency is the correct allocation of each mechanism's strength.

2. The stated scale is a capacity claim rather than a quality claim, and no retrieval accuracy is reported. 3. Feeding diagnostics into the edit path is the detail worth copying, since it makes the compiler an input rather than an afterthought.

1. The stated failure hypothesis is specific and testable: analysis output, compiler errors and file contents accumulate in a single window until planning quality falls. Naming the mechanism is better than the usual appeal to context length. 2. The remedy is structural rather than prompted. Subagents are enforced, one per atomic edit, so tool output terminates in a process that is discarded instead of returning to the planner.

2. Nothing is measured. The design argument is coherent, and coherence is not evidence; a context-degradation claim is unusually easy to test and untested here.

1. Three prompt layers are exposed for editing, which is unusually transparent and has a methodological consequence: two installations of this tool are not the same instrument. Any claim about its review quality is a claim about one configuration, and cannot be transferred between teams. 2. The documentation does not pretend otherwise.

2. No evaluation is published and none is claimed. For a review tool this is the important absence, because review quality is measurable in a way that agent capability often is not: a held-out set of pull requests with known defects would settle it. The design is legible; the performance is unmeasured.

1. Persistent agent memory is usually a vector store nobody can read. Writing sessions and knowledge as JSONL and Markdown makes the context an inspectable artefact: a researcher can diff it, grep it and say precisely what the agent was carrying when it answered. 2. That property is worth more than any retrieval trick.

2. What is not described is selection: how much of a Mate's accumulated memory enters a given prompt, and on what basis. An inspectable store with an undocumented retrieval policy is legible at rest and opaque in use. No evaluation of the memory's effect on output is published, and none is claimed.

SWE-rebench, June 2026, 61.6 percent resolved and 72.7 percent pass@5, is self-reported on a set that draws fresh tasks each cycle, so results move run to run by the organizer's own account, and pass@5 is a five-attempt figure that should not be read beside single-attempt scores. The earlier 53.6 percent on SWE-bench Verified was a single run from January 2025.

The interesting part is verification: agentic debugging sets breakpoints and steps through code in the IDE's debugger, a richer signal than test output. The observation: the benchmark measures the model, the debugger measures the harness, and only the second is Junie's.

1. The verification design is unusually explicit for this category: high-risk operations can be routed through a mode that plans, executes and then checks, which makes intent, action and outcome three separate recorded things rather than one narrative. 2. Evidence bundles give that record a shape somebody other than the author can read.

2. A regression suite for the harness itself is shipped and runnable, which is the closest thing to a published evaluation on this row, and it measures the tool rather than the model.

1. Building on Pydantic AI is the most consequential decision on this row. Tool arguments are validated against declared types by a library maintained elsewhere, which eliminates the family of failures where a model returns almost-correct structure and the agent proceeds anyway. 2. The protocol layer is external too, so tool definitions are not this project's invention.

2. Neither choice is novel, and that is the argument in its favour: the design borrows components whose failure modes are already understood. No evaluation is published, and none is claimed.

1. Review context comes from a codebase index rather than the diff alone, which is the correct design: a change is only wrong relative to code that is not in the change. 2. No measurement accompanies it. Precision and recall on review comments are measurable, the ground truth exists in every merged pull request, and none of it is published.

2. The absence matters because every claim the product makes is about comment quality, and comment quality is exactly what an index cannot guarantee on its own.

1. The interface reads the same files the underlying agent writes, which means there is one representation of a session rather than two that must be reconciled. Divergence between a front end's model and the engine's model is a common and tedious source of defects, and this design removes it by refusing to have a second model.

2. The consequence is that the front end can be replaced or removed without migration. 3. No evaluation is published and none is claimed, which suits a project asserting only an interface.

1. Moving the plan out of the context window into a file on disk is the single most effective answer to a short window, because the state that must persist stops competing with the state that must be read. 2. Summarising against a declared budget rather than truncating at a limit keeps the decision about what to lose explicit.

2. Each of these is a known technique applied deliberately to a stated constraint, which is better engineering than most of this board. No evaluation quantifies any of it, so the constraint is named and the benefit is not measured.

The architecture addresses the failure modes that actually end long sessions. 1. Context is compacted automatically as it grows, so degradation from an overfull window is handled by the scaffold rather than by the user noticing. 2. Repetition is detected and interrupted, which targets the loop where an agent alternates between two wrong edits indefinitely. Both are bounding conditions, and bounding conditions are what separate a harness from a chat client.

No evaluation accompanies either claim, and the iterative mode is labelled experimental by its own author, which is the appropriate epistemic status for a mechanism this hard to test.

1. Isolation is achieved structurally rather than by policy. A separate working tree per session means two agents cannot observe each other's uncommitted state, which removes an entire class of interference without a lock. 2. A sandbox option and a browser view sit beside the session, so the observation surface and the execution surface are the same window.

2. No evaluation is published and none is required, since the claim is organisational rather than capability-bearing. Written in Go and shipped as a prebuilt binary, the tool keeps no runtime of its own between the user and the agent.

1. Instructions are assembled from files on disk and injected into the system prompt, which means the behaviour of the agent is inspectable and diffable rather than embedded in a binary. Anyone reproducing a result can read what the model was told. 2. Chat participants are addressable individually, so a request goes to a named component rather than a general handler.

2. No evaluation is published and none is claimed. The design asserts structure, and the structure is visible on the filesystem, which is the honest form of that claim.

Two points. 1. General-purpose, explore and plan subagents execute as separate child sessions, which bounds what a single task can accumulate and makes context growth a function of the subtask rather than the whole conversation. That is the correct containment for the failure mode where long sessions degrade. 2. Speaking a published agent protocol means the loop can be hosted by editors the vendor does not own, so interoperability does not depend on this vendor winning.

No evaluation is published. The design decisions are legible and should survive a model generation, which is more than the marketing needs them to.

1. The system is authored by the coding agent and then verified by the CLI, which is an unusual and defensible division: generation is probabilistic, import is not. Apps, tables, workflows and permissions land as files, so the artefact under review is text a human can read rather than state inside a running service. 2. That makes the whole configuration diffable, which is the property most agent platforms give up first.

2. Verification is described as an import step, and no evaluation of the generated systems is published. The design is sound; its accuracy is untested in public.

1. Treating evidence as a first-class object rather than a paragraph in a summary is the decision that distinguishes an auditable system from a narrated one, because the artefact supporting a claim is stored beside the claim. 2. Carrying review state in the same model means approval is a recorded transition rather than a message somebody remembers sending.

2. Nothing enforces that an agent produce evidence before a task advances, so the completeness of the chain rests on convention. The model is right and the guarantee is absent.

1. Declaring acceptance criteria as part of the plan is the right structural move, because it fixes the definition of done before the work rather than after it, when the answer is easier to rationalise. 2. Verification by a separate agent that observes a running application is a stronger form of evidence than an assertion from the agent that wrote the code.

2. A screenshot is evidence a human still has to read, so what is automated is collection rather than judgement. The distinction matters and is not made in the documentation.

1. The harness exposes four primitives, run, schedule, watch and inspect, and calls them durable, which is a specific claim: a session outlives the connection that started it. 2. Delegation is shown as a run graph, so a parent session's children are visible as structure rather than inferred from a log, and attribution has an answer.

2. No evaluation is published and none is owed, since the design claims durability rather than capability. The unusual property is that one session is addressable from several unrelated clients, which requires state to live in the harness rather than in any of them.

1. Stating the loop as a sequence rather than describing capabilities is a meaningful editorial choice, because it tells a reader where verification sits relative to the edit, which is the question that determines whether an agent can catch itself. 2. Naming test and lint execution as the tool's purpose puts the check outside the model.

2. What is absent is any measurement of how often the check is invoked or acted upon. The design permits verification; nothing published demonstrates that it happens.

1. Displaying context consumption and cost during a run turns two hidden quantities into observable ones, which is what allows a user to form any model at all of why a session degraded.

2. Branching from an arbitrary earlier message converts the conversation from a linear transcript into a tree, so an unproductive path can be abandoned without discarding what preceded it.

3. That second property is the more consequential: it makes exploration cheap and makes a wrong turn recoverable, which is the closest thing to experimental control this category offers.

1. The context-gathering design is the interesting part. Where most agents in this class locate a symbol by searching text and hoping, this one asks a language server, which returns the definition the compiler would agree with. That converts a probabilistic step into a deterministic one.

2. The consequence is fewer tokens spent on candidate files that were never relevant, and fewer edits made against the wrong definition of an overloaded name. 3. No evaluation quantifies the improvement, and the mechanism is sound enough that the absence is disappointing rather than disqualifying.

1. Capability is assembled from three sources: Python functions, MCP servers and named skills, all managed by the agent rather than fixed at construction. 2. Every tool call runs in an isolated environment, and four backends are documented: Docker, E2B, Kubernetes and a Daytona workspace, which lets the isolation level match the deployment instead of forcing one choice.

2. Verification is not described. Nothing in the documentation defines how an agent decides a task succeeded, and no benchmark is published, so the capability claim rests on the workspace documentation and the examples beside it.

1. Most agents build their own picture of a repository by searching it, which means n agents perform n incompatible analyses. Serving a graph through a protocol makes that picture a shared artefact instead: the same structure, the same version, available to whichever CLI is asking. 2. The design also survives model changes, since a graph is not a prompt.

2. What is undocumented is how the graph is built: which languages are parsed, how it is kept current as files change, and what happens when it is stale. A shared context structure is only an improvement while it is correct, and no evaluation of its accuracy is published.

1. Context persists as conversation. Societies retain chat history, tool outputs and accumulated knowledge across many turns instead of reassembling a prompt each time. 2. Planning is conversational, not procedural. The founding paper, accepted at NeurIPS 2023, has an assistant and a user role talk a task into steps, and the later Workforce module places a coordinator over specialised members. 3. Verification is domain-specific, and the group's CRAB benchmark measures agents across Ubuntu and Android environments rather than on a coding leaderboard.

The observation: this design predates two generations of models and has needed no rewrite, which is better evidence than a score.

1. The figure is precise and unaccompanied. A hit rate near ninety-nine per cent requires three declarations to be meaningful: what counts as a long session, what the workload was, and how a hit was counted. None appears. 2. Prefix caching is nonetheless the right architectural bet, because it reduces cost without changing what the model sees.

2. The design is also provider-shaped: the cache belongs to one vendor's API, so the efficiency argument travels only as far as that vendor does. That is a legitimate engineering choice, and it makes the number unportable, which is more useful to know about it than its size.

1. Verification is offered at three levels rather than one: read the change as a diff, run the program that contains the change, or send a comment back and have the agent revise. Most tools in this class provide the first and stop. 2. The comment channel matters because it closes the loop inside the same context rather than through a new prompt.

2. No evaluation is published and no capability is claimed, so the design stands on structure alone. The structure is coherent.

1. Capability discovery is registry-derived. Endpoints registered by a service become callable tools automatically, and the cards published for other agents are generated from that same registry, so a single definition feeds both directions. 2. Deterministic work is separated into flows whose steps are checkpointed, so a crash resumes at the last completed step rather than at the beginning. 3. Memory is an interface with a durable store behind it by default.

Deriving the interface from the code rather than a parallel schema is the design decision that will still look correct in three years.

1. Choosing the task as the durable boundary is the decision that separates this from a transcript store, because it keeps an artefact beside the reasoning that produced it and makes the pair retrievable together. 2. Giving each agent its own memory, skills and workspace means context is scoped by role rather than accumulated globally, which bounds what any single run can contaminate.

2. None of this is evaluated, and the interesting measurement would be retrieval quality over a long-lived workspace, which no project in this category publishes.

The headline number, Grok Code Fast from 6.7% to 68.3%, comes from the author's post, and the methodology is stated. 1. Fixtures are real files with a mechanical mutation, such as a removed guard clause, to be restored. 2. Three runs of 180 tasks, sixteen models, three edit formats, a fresh session and four tools each time. 3. Success is comparison against the original file after formatting. Total spend is disclosed.

This is reproducible and self-reported, not independent, and the fixtures favour edits that reverse a change exactly, which hashline is good at. The observation: the post's own table shows one model losing ground, and reports it anyway.

1. Taint tracking is a named method with a literature, a false-positive profile and results a second tool can reproduce, which is a different epistemic class from asking a model whether code looks unsafe. Naming the technique rather than the outcome is the part worth crediting.

2. Grounding search in a language server means symbol resolution comes from a parser rather than from text similarity. 3. No accuracy figures accompany either, so the methods are stated and their performance on real repositories is not.

1. Treating coding harnesses as interchangeable behind a single call is an elegant abstraction and a strong claim, because those harnesses differ in context strategy, edit format and verification behaviour. 2. Swapping one for another should therefore change results in ways the interface hides.

2. No comparison is published: no task set run across the six options, no success rates, no cost differences. The abstraction is offered on the assumption that the choice does not matter, which is exactly the assumption worth testing.

1. Owning session state locally rather than deferring to a provider's thread abstraction is the decision that lets the same conversation survive a provider swap, and it is the reason this design should outlast individual API shapes. 2. Serialising prompts through a single actor gives the ordering guarantee that most implementations approximate with a lock and get wrong under cancellation.

2. Compaction is described without a stated policy for what survives it, which is the detail that determines whether a long session degrades gracefully. No measurement accompanies any of this.

The architecture is small enough to state fully. 1. A single Node host routes each message user to messaging group to agent group to session. 2. It is written to the session's inbound.db and the container is woken. 3. Inside, a Bun runner on the Claude Agent SDK polls inbound.db, works, and writes to outbound.db. 4. Each file has exactly one writer, so there is no IPC. 5. A sweep every sixty seconds detects stale sessions and fires due tasks.

Verification is absent; the agent's word is final. The observation: replacing a message broker with two files and a poll is correct at one user, and the design admits it.

1. Making history addressable turns a conversation into an experiment: the same starting point can be run twice with different instructions, and the difference is attributable to the change rather than to drift. 2. Rewinding the workspace alongside the transcript is the part most implementations omit, and without it a replay restores the words while leaving the files where the failure left them.

2. Nothing is published about what the rewind covers or how far back it holds. The mechanism is documented; its guarantees are not.

1. Attribution embedded in version control history is durable in a way that an application log is not: it survives the tool being uninstalled, and it answers the accountability question inside the system reviewers already use. 2. That is the correct place to record it, and almost nobody in this category does.

2. Parallel work is fanned into separate sandboxes rather than threads in one environment, which keeps side effects from interleaving. No evaluation is published, so the architecture is documented and the throughput claim is untested.

1. Binding state to the project rather than to the client is the correct choice, and it has a testable consequence: a session started in one entry point is legible from another, which makes the runtime a shared substrate instead of three products with a family resemblance.

2. The unit of backup becomes obvious as a side effect.

3. What is not specified is the format or the concurrency model: two entry points addressing one project's state at once is the obvious question, and the documentation does not raise it. No evaluation accompanies the design, and none is claimed.

1. Guardrails are applied at both ends of the call, input and output, which is the correct symmetry and one many frameworks break by validating only what the model returns. 2. Checkpoint and restore turn a long trajectory into a resumable object, so a failure at step forty does not require repeating steps one through thirty-nine.

2. Memory is split into a short-term store and a vector-backed long-term store, a separation that is conventional and correctly conventional. No evaluation is published, and the project claims none.

The design choice worth studying is durability of context. Most assistants reconstruct their working set per request by searching transcripts. This one maintains a standing index: sources are written as linked notes with relationships stated rather than inferred, and the meeting recorder captures microphone and speaker, transcribes live, then writes a summary back into the same structure so later work inherits it.

The consequence is traceability. A wrong answer leads back to a specific note, and that note can be corrected by hand, which similarity search does not offer. No evaluation of retrieval quality is published, so the advantage is architectural rather than measured.

1. State is a database. Conversations, data and code live in a local libSQL store rather than in scattered files or a vendor's cloud, which makes the session history queryable by the user and inspectable after the fact. Very few tools in this category treat their own history as structured data. 2. That also means portability is a matter of copying a file.

2. No evaluation of any kind accompanies the product, and its claims are about surface area rather than capability, so none is strictly required. It would still be nice to know what the parallelism buys.

1. Putting heterogeneous components into one graph is the interesting decision, because it means the boundary between calling a tool, consulting a knowledge source and running generated code is expressed in the same notation. A reader can see the whole topology in one place, which is where most frameworks require three.

2. It also flattens a real distinction: those components have very different failure and security properties, and one notation does not make them equivalent. 3. No evaluation of the execution engine is published.

1. The central expression puts provider routing and per-role model assignment on one side and the behaviour pipeline on the other, which is a real separation rather than a naming convention.

2. Behaviours compose with an operator, so an agent's definition is an expression that can be read left to right, and two agents can be compared by comparing their expressions.

3. Runs produce reward-annotated records intended to feed fine-tuning, which is a coherent ambition. No result from that pipeline is published, so the loop is described rather than demonstrated.

1. Long-running agent work is a distributed systems problem wearing a new hat, and this is the first row on the board to name the primitives that literature settled on decades ago. A lease prevents two workers claiming the same goal, an attempt counter bounds retries, and a cooldown stops a tight failure loop.

2. Reconciliation after restart means durability was designed rather than discovered. 3. No evaluation is offered, and the claims here are structural, so none is owed.

1. Choosing plain files as the storage format is the decision that makes the rest defensible. A diagram held in application state is invisible to review; the same diagram written to the tree is diffable, bisectable and survives the tool that made it. 2. That also removes the migration problem, since abandoning the product leaves the artefacts behind rather than trapped.

2. No evaluation is published, and none is required: the claims are about representation and workflow, both of which are verified by reading the repository afterwards.

1. Independent recoverability is the part worth noticing. Checkpointing a whole session is common; checkpointing each workflow separately means a failure in one does not force the others to be replayed, which is a finer granularity than most orchestration layers attempt.

2. Pairing that with concurrent graph execution is coherent, since parallelism is what makes partial failure likely in the first place. 3. No throughput figure, no recovery latency and no evaluation are published, so the design is described but its cost is unknown.

1. The tool prints the file and the line range it is reading rather than a summary of it, which makes the retrieval step auditable in the only way that counts: the reader can check what was actually in the window. Most agents report their intentions and conceal their inputs. 2. That choice costs screen space and buys reproducibility, which is the correct side of that trade for a research-minded user.

2. No benchmark accompanies any of this. The design claim is about interaction, and interaction claims are the ones nobody knows how to measure.

1. Communication between roles is bounded and structured rather than a shared transcript, which keeps a delegation from inheriting everything the parent ever thought and keeps the exchange inspectable. 2. The completion signal is the important design decision: failed and partial work is reported as such, which is the rarest honesty in this category.

2. A system that can say it did not finish is measurable. One that reports success for everything cannot be evaluated at all. No benchmark is published, and after that admission, the absence is forgivable.

1. Generation is iterative by design, replacing the staged pipeline earlier builders used with a loop the model drives through tools. 2. Those tools read and write the project workspace, create restore points, deploy previews, and read browser console output, so the failure signal comes from the running artifact and returns to the same loop that produced it. 3. Underspecified requests trigger structured clarifying questions rather than assumptions.

2. Rollback replays a chosen commit forward as a new commit, leaving history intact. Verification against a deployed runtime is the strongest form available to a generator, and few of them do it.

1. The ownership inversion is the design: state belongs to a long-lived process backed by embedded storage, and clients attach to it, which is why a disconnection is not a loss. 2. External agent binaries connect over a standard client protocol and operate on that shared state rather than their own, so the durability applies to tools this project did not write. 3. Observability is available from a browser, a shell, a stream and a socket, so verification is not tied to one client.

No benchmark is published. The observation: the interesting property is transactional, not conversational.

1. There are two layers by design. A graph runtime supplies persistence and streaming for long-running stateful agents, and the agent framework's composition patterns sit on top, so an author can drop to the graph when the patterns do not fit. 2. The practices that usually live in undocumented prompt engineering are surfaced as hooks with names, covering compaction, context editing, retry on tool failure, limits on model and tool calls, planning and dynamic tool selection.

Turning tacit practice into declared configuration is the most reviewable form of this work, and almost nobody does it.

1. There is a single primitive. The Queen is a loop; workers are clones carrying identical tools and the identical model against separate tasks, spawned by calling run_worker while the system is already running, so no graph is compiled ahead of time. 2. Coordination happens through a shared tracker ledger and a persistent plan rather than a buffer handed between nodes. 3. Memory is scoped per agent and evolves through reflexion and learned skills.

The property purchased is uniformity: with exactly one kind of agent, every agent inherits the same recovery and the same observability. No benchmark is published.

1. This is the architecturally right answer. A process that never possesses a secret cannot leak it, regardless of what text arrives in its context, which converts a prompt-injection problem into an access-control problem where the existing literature is forty years deep.

2. The sandbox gives each agent its own filesystem and shell, so the boundary is per person rather than per organisation. 3. No evaluation is offered, and none would be easy to construct, so the claim rests on the structure. Here the structure is the argument.

1. Eliminating the gap between the development representation and the deployed one removes an entire family of defects, the ones that exist only because a prototype was transcribed. 2. Live attachment to a running instance means observation happens on the real execution, which is a stronger position than reconstructing behaviour from logs after the fact.

2. Two model vendors are supported and no evaluation is offered, which is consistent with a product that claims an authoring and debugging method rather than a capability.

1. Two design choices carry the system. Suspendable images make a run's state a first-class artefact, so resumption is restoration rather than replay, and the token cost of a recovered run is close to zero. 2. A single writer for state avoids the consensus problem entirely instead of solving it badly, which is the correct trade at this scale.

2. No evaluation is published: no recovery success rate, no overhead figure for suspension, and no comparison against restart-from-scratch.

1. Every agent action is recorded to SQLite or PostgreSQL, with optional Langfuse spans, which makes the execution trace a first-class artefact rather than a log line. For a delegating system this is the only way to answer which component produced a given output, and most frameworks in this category leave it to the user. 2. The deterministic mode declares inputs, outputs and dependencies per step, so a run is reproducible in the sense that matters: the same graph, not merely the same prompt.

2. No evaluation is published. Nothing here claims a capability that would require one.

1. Recording verification outcomes in the same trace as the changes that prompted them is the detail that separates an audit log from a transcript: the reader can check whether a claim was tested rather than whether it was asserted. 2. Assigning testing and review to agents that did not perform the implementation preserves the separation that makes any such check meaningful.

2. Nothing quantifies whether the separation catches more defects than a single agent asked to check itself. The design is principled and, as published, undemonstrated.

1. Most agents rebuild the prompt each turn and pay for it, because the ordering that makes a prompt readable is not the ordering that makes it cacheable. Treating prefix stability as a constraint on how context is assembled is a genuine architectural position and an uncommon one.

2. The saving is arithmetic rather than a benchmark, which is the honest way to make this kind of claim. 3. Nothing is published about how often the prefix survives a real editing session, and that number is the one a reader would want.