open-operator-evals

by nottelabs · Agent Tool · ★ 47

About open-operator-evals

Opensource operators evals Key highlights You can investigate all replays/logs and reproduce the benchmark yourself 👇🏻 Notte leads the benchmark by achieving the highest performance with 86.2% self-reported success and 79% LLM-verified completion. It also has the fastest execution time at 47s per task and an impressive 96.6% task reliability—Percentage of tasks an agent successfully completes at least once when given multiple attempts Our evaluation of Browser-Use shows 77.3% self-reported agent performance and 60.2

ai-agentsai-toolsbrowser-automationbrowser-usecomputer-usecuallmnotteweb-agent

Quick Facts

Stars47
Forks7
LanguagePython
CategoryAgent Tool
Quality Score58.6242610917/100
Last Updated2025-04-11
Created2025-04-08
Platformsbrowser, python
Est. Tokens~9570k

Compatible Skills

These tools work well together with open-operator-evals for enhanced workflows:

  • TuriX-CUA — semantic(0.49)+complementary+rare_topics+same_lang+shared_platform (66%)
  • Web-Use — semantic(0.34)+complementary+rare_topics+same_lang+similar_pop+shared_platform (61%)
  • web-agent-protocol — semantic(0.49)+complementary+rare_topics+same_lang+shared_platform (57%)
  • clawreins — semantic(0.19)+complementary+rare_topics+same_lang+similar_pop+shared_platform (56%)

More Agent Tool Tools

Explore other popular agent tool tools:

View all Agent Tool tools →

Popular Python Agent Tools

Frequently Asked Questions

What is open-operator-evals?

open-operator-evals is Opensource benchmark evaluating web operators/agents performance. It is categorized as a Agent Tool with 47 GitHub stars.

What programming language is open-operator-evals written in?

open-operator-evals is primarily written in Python. It covers topics such as ai-agents, ai-tools, browser-automation.

How do I install or use open-operator-evals?

You can find installation instructions and usage details in the open-operator-evals GitHub repository at github.com/nottelabs/open-operator-evals. The project has 47 stars and 7 forks, indicating an active community.

View on GitHub → Browse Agent Tool tools