fbpx

8. What is robots.txt?

8. What is robots.txt?

This is a page that gives search engines information about the pages a company wants indexed or crawled. You can find this page by doing to YOURDOMAIN/robots.txt.