Administration# Contents: Administration interface Adding URLs to the crawl queue Crawl status Crawl Policies Policy matching Indexing decision Browser Updates Authentication Recursive crawling No limit recursion Limited recursion Partial limited recursion Archive browsing Authentication handling Submitting forms Executing javascript Cookie Edition Documents Status Error Crawl DT Recursion remaining Rejected by robots.txt Too many redirects Show on homepage Domain Settings Browse mode Ignore robots Robots.txt status Robots.txt allow/disallow rules Cookies Excluded URLs External Search Engines User management