Administration¶ Contents: Administration interface 🌐 Crawl a new URL ✔ Crawl queue 🕷 Crawlers 📊 Analytics ⚡ Crawl Policies Policy matching ⚡ Crawl 🌍 Browser 🔖 Archive 🕑 Recurrence 🔒 Authentication Actions Recursive crawling No limit recursion Limited recursion Partial limited recursion Atom and RSS feeds Caching for news aggregators 🔤 Documents Document detailed view Document list view ⭐ Tags 🕸 Domain Settings Browse mode Ignore robots.txt Robots.txt status Robots.txt allow/disallow rules 🍪 Cookies Cookies import 📡 Webhooks Webhook Configuration Example: Discord Notification Webhook 🔗 Excluded URLs 🔍 External Search Engines 👥 Permissions Crawl Permissions Search Permissions