How we test
Arogo’s promise is small: tell us the job, and we find what your AI needs. An “add-on” here means the small program that lets your AI reach something, like a spreadsheet or your calendar (you may also see it called a connector). This page shows how we check we keep our promise, what we measured, and where we fall short. Last updated October 28, 2026.
The question
You can ask your AI to find add-ons for a job, or it can answer from memory. We test Arogo against both on the same jobs and check repository information on GitHub.
- Arogo: searches GitHub and the MCP registry for candidate connectors, ranks them, and reports when it finds no suitable match.
- An AI searching the web: Claude, asked to find the connectors for the same job, with web search switched on.
- An AI answering from memory: the same model, told not to search.
What counts
- Every link must exist. Invented links count against you.
- A connector counts as maintained if it is not archived and was updated in the last 180 days.
- For each app a job names, did the answer include a maintained connector for it? The scoring of app coverage is under review.
- Where an app’s own maker publishes a connector, was it picked?
What we measured
We tested ten selected jobs, three runs each, for thirty answers per approach. One example was “each morning, post our open Datadog incidents in a Slack channel”.
Results under review: We found errors in how the scoring script identifies some GitHub links and app names. We are withholding the comparison numbers until the results have been independently rechecked. The jobs were internal test cases, not customer studies.
Where it falls short
- Thirty answers per approach is a small test on selected jobs, not evidence of performance on every task.
- A search may find no suitable connector. That does not establish that no connector exists.
- A recent GitHub update is only a maintenance proxy. It does not establish that a connector works for a particular job or is secure.
- Our original app-coverage scoring needs correction and independent review.
What the safety label means
Connector cards use public facts on GitHub (maker, licence, contributors, published security advisories) and information from the connector’s README to help you assess it. This is not a code audit. We do not say a connector is safe. Grant it only the access the job needs.
What Arogo’s prompt does not do
We also compared more elaborate prompts with plain requests on selected tasks. In those internal tests, the more elaborate prompts did not consistently do better and sometimes did worse. This does not establish performance on every task. Your sheet starts with your own request. It can also include connector information and a request for a report from your AI.