How can I use the Robots.txt File Generator?
Visit our website SEO Spider Tools and click on Free robots.txt file generator. Once you come on this page, enter the URL of your website and hit submit button.Once you get your robots.txt content, copy it and just it in your website.