dev.workers.pathwren.www/robots-policy-lintweb-automation

Robots Policy Lint

Synced · awaiting check

Lint a robots.txt, test paths against RFC 9309, see which AI crawlers it really blocks.

Install

claude mcp add --transport http robots-policy-lint https://www.pathwren.workers.dev/c/mcp-registry-official/mcp/robots

Our take

Not yet reviewed. This entry is synced and verified automatically but hasn't had an editorial pass yet. The data above is real; the judgment is yours for now.

Something wrong or dead here? Report it

Verification record

last verified
today
in registry since
2026-09-01

More in web-automation

browse all →
browser-useweb-automation

browser-use

Verified · yesterday

Control a real Chrome browser to complete any task: fill forms, extract data, book flights.

hand-reviewed112k starschecked yesterday

Scraplingweb-automation

Scrapling MCP Server

Verified · 21 days ago

Web scraping with stealth HTTP, real browsers, and Cloudflare bypass. CSS selectors supported.

hand-reviewed74k starschecked 21 days ago

skyvernweb-automation

Skyvern

Verified · 20 days ago

AI-powered browser automation — navigate, click, fill forms, and extract data from any website.

hand-reviewed23k starschecked 20 days ago

apify-mcp-serverweb-automation

apify-mcp-server

Verified · 24 days ago

Extract data from any website with thousands of scrapers, crawlers, and automations on Apify Store ⚡

hand-reviewed3.0k starschecked 24 days ago