io.github.taylorsmithgg/crawl-censusweb-automation

Crawl Census

Verified · 2 mo ago

Ask before you fetch: will this domain serve your crawler, refuse it, or charge it?

Install

claude mcp add --transport http crawl-census https://crawlcensus.com/mcp

Our take

Crawl Census is aimed at answering a practical web-automation question: whether a domain is likely to allow, block, or monetize crawler access. It is best suited to teams doing large-scale scraping, indexing, or agent browsing who want a policy check before making requests. The use case is narrow but credible; with no visible adoption or packaging signals, it reads as a specialized, still-maturing utility.

reviewed by hand · 2026-08-24

Something wrong or dead here? Report it

Verification record

last verified
2 mo ago
github stars
0
last commit
3 mo ago
archived
no
license
NOASSERTION
in registry since
2026-08-23

github.com/taylorsmithgg/crawl-census-client

browser-useweb-automation

browser-use

Verified · 44 days ago

Browser agents and stealth cloud browsers for AI: run web tasks, fill forms, extract data.

hand-reviewed117k starschecked 44 days ago

Scrapling MCP Server

Verified · 27 days ago

Web scraping with stealth HTTP, real browsers, and Cloudflare bypass. CSS selectors supported.

hand-reviewed90k starschecked 27 days ago

playwright-mcpweb-automation

playwright-mcp

Verified · 31 days ago

Playwright Tools for MCP

hand-reviewed38k stars3.5m dl/wkchecked 31 days ago

next-ai-drawioweb-automation

Next AI Draw.io

Verified · 36 days ago

draw.io diagrams your AI agent draws on a live browser canvas that you can also edit by hand

hand-reviewed36k stars16k dl/wkchecked 36 days ago

AIHawk

Verified · 15 days ago

AI browser agent: browses, clicks, types, and reads real web pages from plain-English instructions.

hand-reviewed32k starschecked 15 days ago