A “robots.txt” file is a text document located in the root directory of your website that specifies which pages should or should not be indexed by web browsers and search engines. Here are the basic steps to create a robots.txt file:
1. Create a text file using a text editor and save it as “robots.txt” ...