Why Google Indexes Blocked Web Pages Google's John Mueller explains why disallowed pages are sometimes indexed and that related Search Console reports can be dismissed Google’s John Mueller answered a question about why Google indexes pages that are disallowed from crawling by robots.txt and why the it’s safe to ignore the related Search Console reports about those crawls. Bot Traffic To Query Parameter URLs The person asking the question documented that bots were creating links to non-existent query parameter URLs (?q=xyz) to pages with noindex meta tags that are also blocked in robots.txt. What prompted the question is that Google is crawling the links to those pages, getting blocked by robots.txt (without seeing a noindex robots meta tag) then getting reported in Google Search Console as “Indexed, though blocked by robots.txt. Takeaways: 1. Confirmation Of Limitations Of Site: Search Mueller’s answer confirms the limitations in using the Site:search advanced search operator fo...