Generate robots.txt rules to block the crawl location to block but allow the crawling of location to crawl within the domain…