How close are AI agents that reliably complete ordinary office and web tasks, such as booking, filling in forms and working in spreadsheets, by operating a computer without close supervision?

The road so far

2 of 8 steps done, 3 in progress.

1. Done · 11 Apr 2024Benchmarks for computer use exist

   The OSWorld preprint, submitted on 11 April 2024, provides 369 computer tasks in real web and desktop applications. WebArena, a preprint from July 2023, provides functional websites from four common domains. Both give human baselines for comparison. [1][2]

2. Done · 22 Oct 2024Computer-use features released in beta and research preview

   Anthropic introduced computer use in public beta on 22 October 2024 and called it experimental, at times cumbersome and error-prone. In August 2025 it launched Claude for Chrome as a research preview with trusted users. [3][4]

3. In progress · 21 Nov 2025Simple interface tasks completed reliably

   In the UI-CUBE benchmark, submitted on 21 November 2025, simple UI interactions reached 67 to 85% success for five models, against 97.9% for humans. Simple interactions are still well below the human rate, and the benchmark covers only 136 such tasks. [7]

4. In progress · 28 Jul 2025Full benchmark score near the human baseline

   The OSWorld-Verified launch post of 28 July 2025 reports CoACT-1 at 60.76%, against a human baseline of about 72% taken from the original OSWorld study. Some later entries are reported higher, but those figures were not checked against a primary source here. [8][1]

5. Not started · 28 Jun 2026Long, multi-hour workflows completed reliably

   OSWorld 2.0, submitted on 28 June 2026, contains 108 long-horizon workflows that take a median human about 1.6 hours. The best reported system completed 20.6% of tasks under the binary completion measure, at a limit of 500 steps. [9]

6. Not started · 20 Apr 2026Consistent results across repeated runs

   A 2026 study runs OSWorld tasks repeatedly and reports variation between runs. It names three sources: stochasticity during execution, ambiguity in task specification and variability in agent behaviour. It gives no single headline success rate. [10]

7. In progress · 24 Nov 2025Protection against prompt injection that holds against adaptive attackers

   Anthropic reported that, for browser use, its safeguards cut attack success for Claude Opus 4.5 to 1.4% against an internal adaptive 'Best-of-N' attacker given 100 attempts per environment. The figure is shown in a chart. The company states that no browser agent is immune and that a 1% rate still represents meaningful risk. [5][6]

8. Not startedUnsupervised use in everyday work shown independently

   No independent field study of agents completing everyday office or web tasks without close supervision was found in the sources reviewed. Published evidence comes from benchmarks and developer reports, and it does not show what share of real tasks agents complete.

Why it matters

Much routine work happens in software that has no programming interface, so a person has to click through it. If agents could complete these tasks reliably, they could take on some of that work. The same ability also makes them a target, because a web page can try to give them instructions. Both the benefit and the risk depend on whether agents can finish tasks without someone checking every step.

What stands in the way

• UI-CUBE reports 9 to 19% success on complex workflows (its abstract gives 15 to 32%), against 61.2% for human evaluators with no prior experience of the applications.
• Long workflows are hard: the best system on OSWorld 2.0 completed 20.6% of tasks, measured at a 500-step limit.
• Results vary between runs of the same task, so a single success rate may not show how dependable an agent is.
• Indirect prompt injection from web content can make an agent act on instructions that the user did not give. Anthropic states that no browser agent is immune, and the sources reviewed show no defence that removes this risk.

Signals to watch

• Independent results on OSWorld 2.0 or OSWorld-Verified for a system in general release, not a research preview.
• Published repeated-run results showing success rates across many runs of the same tasks.
• A field study of agents doing office or web tasks with human checks removed for a set period.
• Prompt injection attack success rates reported on full computer use, not only browser use, with the test setup published.
• Agents that ask for approval before purchases, submissions or account changes being withdrawn from vendor products, or made the default.

Sources

1. preprint
   OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments
   Tianbao Xie, Danyang Zhang, Jixuan Chen et al.. arXiv, 11 Apr 2024.
2. preprint
   WebArena: A Realistic Web Environment for Building Autonomous Agents
   Shuyan Zhou, Frank F. Xu, Hao Zhu et al.. arXiv, 25 Jul 2023.
3. company announcement
   Introducing computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku
   Anthropic, 22 Oct 2024.
4. company announcement
   Claude for Chrome
   Anthropic, 25 Aug 2025.
5. company announcement
   Mitigating the risk of prompt injections in browser use
   Anthropic, 24 Nov 2025.
6. preprint
   Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection
   Kai Greshake, Sahar Abdelnabi, Shailesh Mishra et al.. arXiv, 23 Feb 2023.
7. preprint
   UI-CUBE: Enterprise-Grade Computer Use Agent Benchmarking Beyond Task Accuracy to Operational Reliability
   arXiv, 21 Nov 2025.
8. other
   OSWorld-Verified
   XLANG Lab, 28 Jul 2025.
9. preprint
   OSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks
   Mengqi Yuan, Zilong Zhou, Xinzhuang Xiong et al.. arXiv, 28 Jun 2026.
10. preprint
    On the Reliability of Computer Use Agents
    Gonzalo Gonzalez-Pumariega, Saaket Agashe, Jiachen Yang et al.. arXiv, 20 Apr 2026.

Last reviewed 22 Oct 2026.