io.github.taylorsmithgg/crawl-censusweb-automation

Crawl Census

Verified · today

Ask before you fetch: will this domain serve your crawler, refuse it, or charge it?

Install

claude mcp add --transport http crawl-census https://crawlcensus.com/mcp

Our take

Crawl Census is aimed at answering a practical web-automation question: whether a domain is likely to allow, block, or monetize crawler access. It is best suited to teams doing large-scale scraping, indexing, or agent browsing who want a policy check before making requests. The use case is narrow but credible; with no visible adoption or packaging signals, it reads as a specialized, still-maturing utility.

reviewed by hand · 2026-08-24

Something wrong or dead here? Report it

Verification record

last verified
today
github stars
0
last commit
today
archived
no
license
NOASSERTION
in registry since
2026-08-23

github.com/taylorsmithgg/crawl-census-client

More in web-automation

browse all →
browser-useweb-automation

browser-use

Verified · 18 days ago

Control a real Chrome browser to complete any task: fill forms, extract data, book flights.

hand-reviewed108k starschecked 18 days ago

Scraplingweb-automation

Scrapling MCP Server

Verified · 12 days ago

Web scraping with stealth HTTP, real browsers, and Cloudflare bypass. CSS selectors supported.

hand-reviewed74k starschecked 12 days ago

skyvernweb-automation

Skyvern

Verified · 11 days ago

AI-powered browser automation — navigate, click, fill forms, and extract data from any website.

hand-reviewed23k starschecked 11 days ago

apify-mcp-serverweb-automation

apify-mcp-server

Verified · 15 days ago

Extract data from any website with thousands of scrapers, crawlers, and automations on Apify Store ⚡

hand-reviewed3.0k starschecked 15 days ago