Open your server logs right now and you’ll probably see dozens of visits from “Googlebot.” Here’s the uncomfortable part: almost none of your site’s defenses actually check whether that’s true.
A user agent string, the little bit of text a browser or bot sends to identify itself, is just a sentence. Anyone can write it. A scraper, a competitor’s bot, or an actual attacker can set their user agent to “Googlebot” and most WordPress sites will wave them straight through, because most plugins only ever check that one string and stop there.
Why This Actually Matters
Real search engine crawlers get special treatment almost everywhere. Rate limits get relaxed for them. Security plugins often exempt them entirely, on the reasonable logic that you don’t want to accidentally block Google and vanish from search results. Bot-blocking rules frequently carve out an exception the moment they see “Googlebot” or “Bingbot” in the request.
That exception is exactly what a fake crawler is counting on. If claiming to be Googlebot gets you past a plugin’s other checks, then every attacker with basic tooling already knows to claim it.
The Mistake Almost Every WordPress Site Makes
Ask most security plugins how they know a request is really from Google, and the honest answer is: they don’t. They see the word “Googlebot” in the user agent and take it at face value. It’s the digital equivalent of letting someone into a building because they said their name was on the guest list, without ever checking ID.
Google has actually addressed this directly, because the confusion is common enough that they wrote official documentation about it. Their own guidance states plainly that “the HTTP user-agent request header used by Googlebot is often spoofed by other crawlers,” and that if you’re relying on it to make decisions, you should verify the request is genuine first.
How Google Actually Says to Verify Its Own Crawler
Google’s real recommendation isn’t complicated, but it does require two steps instead of one:
- Reverse DNS lookup. Take the IP address that made the request and look up what hostname it resolves to.
- Confirm the domain. That hostname needs to end in
googlebot.com,google.com, orgoogleusercontent.com. Nothing else counts. - Forward DNS lookup. Take that hostname and look it up again, in the other direction. It needs to resolve back to the exact same IP address you started with.
If all three steps check out, you’re genuinely looking at Google. If a request fails any one of them, whatever sent it is not Googlebot, no matter what its user agent claims.
You can actually try this yourself on a real IP address in about ten seconds, no plugin required. Google publishes a real example in their own documentation: run host 66.249.66.1 and you’ll get back crawl-66-249-66-1.googlebot.com. Run host crawl-66-249-66-1.googlebot.com and it resolves right back to 66.249.66.1. That round trip is the entire trick.
(Source: Google’s official crawler verification documentation)
The Same Technique Works for Bing
This isn’t a Google-only idea. Bing recommends the identical approach for verifying Bingbot, just with a different expected domain: a genuine Bingbot request should reverse-resolve to a hostname ending in search.msn.com.
(Source: Bing’s own webmaster blog on verifying Bingbot)
Meta’s crawlers can be verified the same way too, tied to their own network ranges. The pattern holds across every major, legitimate crawler: the real ones can survive a network-level check, because they’re actually who they say they are. The fake ones can only ever fake the one thing that costs nothing to fake, the user agent string.
Why Most Plugins Don’t Actually Do This
Honestly, because it’s more work. Checking a user agent string is one line of code. Doing a reverse DNS lookup, confirming the domain, then doing a forward lookup to confirm the match, and caching the result so you’re not doing this on every single request, is a meaningfully bigger build. It’s the difference between checking someone’s name tag and actually calling their office to confirm they work there.
Most plugins take the cheap route because it’s invisible to a user unless something goes wrong, and by the time something goes wrong, it’s usually too late to notice why.
How to Check It Yourself, Right Now
You don’t need any special tooling to try this on your own logs today:
- Find an IP address in your logs claiming to be Googlebot or Bingbot
- Run a reverse DNS lookup on it (
host [IP address]on Mac/Linux, or an online reverse DNS tool if you’re on Windows) - Check the domain it resolves to against the list above
- Run a forward lookup on that domain and confirm it matches the original IP
If it doesn’t check out, whatever’s hitting your site isn’t who it claims to be, and it’s worth finding out what it’s actually doing there.
What This Means for Your Site
None of this is a reason to panic about every bot in your logs, most crawler traffic really is exactly what it claims to be. It’s a reason to be specific about which claims you’re trusting, and why.
This is one of the things SecondGate does automatically, on every request that claims to be a major search crawler, using exactly the network-verification method described above rather than trusting the name alone. It’s a small, unglamorous piece of a much bigger picture, but it’s the kind of detail that actually matters once you start looking for it.
