Better Robots.txt creates a WordPress virtual robots.txt, helps you boost your website SEO (indexing capacities, Google ranking,etc.) and your loading performance –Compatible with Yoast SEO, Google Merchant, WooCommerce and Directory based network sites (MULTISITE).
With Better Robots.txt, you can identify which search engines are allowed to crawl your website (or not), specify clear instructions aboutwhat they are allowed to do (or not) and define a crawl-delay (to protect your hosting server against aggressive scrapers). Better Robots.txt also gives you full control over your WordPress robots.txt content via the custom setting box.
Reduce your site’s ecological footprint and the greenhouse gas (CO2) production inherent to its existence on the Web.