Google crawls your site just after reading the most essential files like robots.txt and sitemap.xml/sitemap.html from your server. Let me clear what these files do? The sitemaps contains the whole data of your site including links of posts, pages, tags, categories, authors, etc, can be said the whole structure of your site in only one file either in HTML or XML format. Robots.txt file tells the search engines which pages they have to crawl and also exclude those which don’t. This file usually contains sitemaps URLs, for WordPress it looks like:

More info on how to edit robots.txt file in WordPress, read this article!

Use Google Bot Bling to Know When Google Crawls your WordPress Site!!!

Well, Google Bot Bling is a WordPress plugin which describe complete information about when the Googlebot came on your WordPress site and what did the bot request from your site, and crawled. Until this plugin is in early development and it is designed well to help WordPress site owners when Google indexes their sites by sending emails from time to time.

How it Works?

There is a plugin settings page where you have to add your email and specify the time interval of sending emails about the Googlebot. To add the email and set the time interval follow these steps:

1. Download, install, and activate the plugin Google Bot Bling

2. Go to Settings > Google Bot Bling

3. What about secret file name? The secret file tells you about the requested data from your site by a bot on daily basis.

4. After setting up the plugin you’ll be receiving the emails from Google Bot Bling whenever it found the new crawled pages. See the email sent by this plugin:

Enjoy blogging with WordPress! 😀