Check Your Robots.txt File
1st Check That You Have The File Installed.
In The Address Bar For Your Favorite Web Browser Enter Your Website address the add /robots.txt to the end. Example http://www.seo-911.com/robots.txt
You should see:
What you don’t want to see is:
The above returns a 404 page error, which says the robots.txt file does not exit.
Confirm Your Robots.txt File Is Working
- Want to be found in the search results
- Want Your website to be indexed by the search engine
Use the following link to test your robots.txt file: