Verdy.

For site owners

VerdyBot

VerdyBot is Verdy's crawler. It runs the website audits people ask for, and it reads newly launched products for the software directory.

How it identifies itself

Mozilla/5.0 (compatible; VerdyBot/0.1; +https://verdy.site/bot)

When VerdyBot finds a product on its own, it only ever asks as VerdyBot. It never retries disguised as a browser, and it does not read a site whose robots.txt blocks it.

What it reads

Public pages only: the homepage, and up to a dozen pages from your sitemap or links, such as pricing and docs. It also fetches robots.txt, sitemap.xml and llms.txt. Nothing behind a login, no forms, and nothing personal. From those pages it records what the product does, who it is for, pricing and similar facts, each kept with a short quote and a link back to the page it came from.

Block it, and leave the directory

Add this to your robots.txt:

User-agent: VerdyBot
Disallow: /

VerdyBot stops reading your site, and if your site is in the directory it is removed the next time VerdyBot checks, normally within a day. This works on its own, with no need to contact anyone, because only the site's owner can change that file.

Or ask us to remove it

If you can't edit robots.txt, send a request. A person reviews it and replies by email; a request can't remove a listing on its own, so nobody can delist a site that isn't theirs.

Something wrong in your listing?

Every fact in a listing is quoted from your own site. Say it clearly there and the next read picks it up. The score is explained on the methodology page.

VerdyBot — what it reads, and how to opt out