Is That Really Googlebot? How to Tell When a Crawler Is Lying to You
Open your server logs right now and you’ll probably see dozens of visits from “Googlebot.” Here’s the uncomfortable part: almost none of your site’s defenses actually check whether that’s true. A user agent string, the little bit of text a browser or bot sends to identify itself, is just a…







