A Googlebot user-agent string does not prove that a request came from Google. Anyone can send that header. Check the source IP before allowlisting the request or treating it as verified Googlebot traffic.
Google documents two methods: checking its published crawler IP ranges, or checking reverse DNS and confirming the result with a forward lookup. Use the verification documentation for the correct ranges and hostname rules for each crawler class.
Start with the real client IP
Use the client IP recorded by your trusted CDN, proxy, or web server. An arbitrary X-Forwarded-For header can be supplied by the visitor. Configure the application to trust forwarding headers only from the proxies it actually uses.
Keep enough log detail to revisit the check:
| Detail | Why it helps |
|---|---|
| Client IP and proxy source | Lets you repeat the identity check |
| Timestamp and timezone | Connects the request to a release or blocking event |
| Method, host, and path | Shows which URL was requested |
| Original user agent | Identifies the claimed crawler family |
| Status, response size, and duration | Shows how the request was handled |
| Cache result and request ID | Helps find the corresponding CDN or server event |
Use published IP ranges for repeated checks
Download the applicable range file from Google's documentation and parse the addresses as IP networks. Match each observed client IP against those networks. A string-prefix comparison is not an IP-range check.
Google separates common crawlers, special-case crawlers, and user-triggered fetchers. Keep those classes separate. A verified Google address does not automatically mean the request was regular Googlebot Search crawling.
Cache the range file, record when it was retrieved, and define how it will refresh. If the file is unavailable or too old for your verification policy, mark the result as unverified instead of silently approving it.
Use DNS for an individual request
- Run a reverse DNS lookup on the observed source IP.
- Check the returned hostname against the documented Google hostname pattern for that crawler class.
- Resolve that hostname forward.
- Confirm that the original IP appears in the result.
Check the domain boundary as well as the suffix: a hostname merely containing googlebot.com is not enough. Record the lookup results and time so another person can review them.
Label the result clearly
| Result | Meaning |
|---|---|
| Claimed Googlebot | The header matches; the source has not been verified |
| Verified crawler | The source passed the appropriate IP or DNS check |
| Could not verify | A required field or verification source was unavailable |
| Failed verification | The source did not pass the selected check |
| Other Google fetcher | A verified request from a different crawler or tool |
Keep the original crawler claim alongside the verification result. A failed check needs investigation: the cause might be a spoofed request, incorrect proxy handling, or outdated verification data.
Check indexing separately
A verified crawl tells you that a particular crawler requested a URL at a particular time. The server log shows the recorded response status and timing. It does not show whether Google rendered the complete page or selected it for indexing.
Use URL Inspection for the indexed status and rendered-page checks when content is missing. Use Search Console performance data for impressions and clicks. Keep those observations alongside the crawl record when investigating a page, instead of treating a successful fetch as proof of search visibility.