Robots.txt Text file that allows or not the access of the search engine robots (crawlers) to a website.