io.github.hamzawamiqhq/ai-crawler-checksearch

AI Crawler Access Check

Synced · awaiting check

Can GPTBot, ClaudeBot and PerplexityBot reach a site? Real fetch per crawler, firewall/CDN blocks.

Install

claude mcp add --transport http ai-crawler-check https://check.citant.ai/mcp?src=registry

Our take

Checks whether major AI crawlers such as GPTBot, ClaudeBot, and PerplexityBot can reach a site using real fetch behavior and block detection. It is useful for site owners, SEO teams, and infrastructure operators auditing crawler access through CDNs or firewalls. As a remote-only service with no package or repository signals listed, its operational transparency and reliability should be verified before depending on it.

reviewed by hand · 2026-10-07

Something wrong or dead here? Report it

Verification record

last verified
today
in registry since
2026-10-06
mcp-server-searchsearch

mcp-server-search

Verified · 24 days ago

MCP server for web search operations

hand-reviewed39k starschecked 24 days ago

repomixsearch

Repomix

Verified · 15 days ago

Pack local or remote codebases into AI-friendly files that LLMs and coding agents can read or search

hand-reviewed28k stars67k dl/wkchecked 15 days ago

screenpipe-mcpsearch

screenpipe

Verified · 16 days ago

Search your local screen recordings, audio transcripts, and computer activity from screenpipe.

hand-reviewed22k stars2.8k dl/wkchecked 16 days ago

sui-kapasearch

Sui Knowledge Base (kapa.ai)

Verified · 11 days ago

Semantic search across Sui documentation and knowledge sources, powered by kapa.ai

hand-reviewed7.8k starschecked 11 days ago

firecrawl-mcp-serversearch

Firecrawl MCP Server

Verified · 31 days ago

MCP server for Firecrawl — web search, scraping, and biomedical/arXiv paper search.

hand-reviewed7.4k starschecked 31 days ago

robinsearch

Robin: AI-Powered Dark Web OSINT

Verified · 11 days ago

Dark web OSINT over Tor: search onion engines, scrape pages, report with your own model.

hand-reviewed7.3k starschecked 11 days ago