Agent S

Agent S clicks and types on the machine in front of it, across Linux, Windows and macOS, which is the only way to automate an application with no API. The trait that decides it daily is that it wants the computer to itself: a single monitor is a stated prerequisite, so the machine running it is not the machine you are working on.

Setup is more than a pip line, since the optical text step needs tesseract installed separately. Pick it when desktop applications are the target and you can dedicate a box. Pick Browser Use when everything you need lives behind a URL.

The risk is that there is no boundary. The prerequisites warn that the agent executes Python to control your computer and to use it with care, and no container isolation is listed, so a mistaken click or a generated command reaches your files, your sessions and anything already logged in. Desktop agents fail in ways that look like a user error rather than a stack trace.

Give it a dedicated machine with its own accounts. What it does right: every capability jump is tied to a dated package version, so you can install the exact release a given paper describes instead of guessing.

1. Two numbers are reported and only one is the system you install. Alone, in the hundred-step setting, it reaches 66%, above the previous best of 63.4%. 2. Adding Behavior Best-of-N raises the figure to 72.60%, which is what crosses the roughly 72% human reference. 3. Sampling several attempts and selecting among them is a legitimate method, and it is also a different cost profile from a single run.

The paper is accepted at TMLR, and generalisation is reported on two further environments rather than asserted. The distinction between the two figures is stated in the repository, which is the correct behaviour.

A research lab with a product attached. The repository invites you to skip the setup and use the company's cloud instead, which is where the business is, and the open framework serves as proof that the cloud is worth paying for. Three papers at three venues is a credible technical asset in a category where most claims are blog posts.

The exposure is that computer use is exactly the capability the frontier labs are building into their own models, and a scaffold advantage compresses each time one ships. Likely acquirer: a model lab wanting a desktop-control team. Position: watch the cloud's pricing for the real signal.

The demo is a computer operating itself, which does land in a room. Then the requirements: a dedicated single-monitor machine per user because it takes over the screen, plus a hosted endpoint for the visual grounding component, so sixty people means sixty workstations and an inference bill nobody forecast. There is no single sign-on, no audit of what it clicked, and no central policy.

It does not run in our pipelines and produces no reviewable output. Support is a Discord and a research group. Not yet. A single automation team with dedicated hardware and dedicated accounts is the only shape I would approve.

Apache-2.0 and the model layer is open at both ends. Supported inference includes vLLM alongside the hosted vendors, and the recommended grounding model is an open-weights release I can serve myself rather than a proprietary endpoint I rent. Keys are plain environment variables in my shell profile, not a settings database I have to reverse.

What is missing is the protocol layer: no client and no server, so it does not join the rest of my tooling and I would be writing the bridge. Everything else is a Python package I can read, patch and pin.