browser-use
Verified · 41 days agoBrowser agents and stealth cloud browsers for AI: run web tasks, fill forms, extract data.
Analyze and generate robots.txt with AI crawler detection for 20+ bots like GPTBot and ClaudeBot.
claude mcp add robotstxt-ai -- npx -y [email protected]
This server analyzes and generates `robots.txt` rules with explicit attention to AI crawlers such as GPTBot and ClaudeBot. It is for site owners, SEO practitioners, and developers managing crawler policy as AI bot traffic becomes more relevant. The use case is timely and concrete, though users should still validate generated policies carefully because crawler behavior and preferred directives can change faster than a small utility may be maintained.
Browser agents and stealth cloud browsers for AI: run web tasks, fill forms, extract data.
Web scraping with stealth HTTP, real browsers, and Cloudflare bypass. CSS selectors supported.
MCP server for browser use access
Playwright Tools for MCP
draw.io diagrams your AI agent draws on a live browser canvas that you can also edit by hand
AI browser agent: browses, clicks, types, and reads real web pages from plain-English instructions.