What is robots.txt in WordPress?
A robots.txt file is a text file that notifies search engine crawlers not to explore certain areas of your website OR Robots.txt is a text file that allows a website to provide instructions to web crawling bots. It is located at the root of your website. It’s also known as the Robots Exclusion Protocol because …