Emergency server help: get in touch

bc.googleusercontent.com in Your Logs: Googlebot or a Scanner? How to Tell and What to Block

How to tell Googlebot from Google Cloud scanners in your access log, a tested classification script using Google's IP lists, and how to block single addresses without hurting search.

Published Updated 7 min read

Short answer: *.bc.googleusercontent.com addresses are Google Cloud virtual machines rented by anyone, not Googlebot. Google’s own crawlers resolve to googlebot.com or google.com, and its user-triggered fetchers to google.com or gae.googleusercontent.com. Check an address with a reverse and forward DNS lookup or against Google’s published IP lists, block abusive Cloud addresses one at a time (CSF or Imunify360), and never block whole Google ranges. (Reviewed October 2026.)

A DirectAdmin forum thread asks how to stop scanners from bc.googleusercontent.com without hurting Google Search. We ran a classification script over one day of our own web server log, checked sample addresses with DNS, and compared them with the IP lists Google publishes.

Who is behind googleusercontent.com

Reverse DNS looks likeWho it isBlock it?
crawl-66-249-67-34.googlebot.comGooglebot and other common Google crawlersNo
rate-limited-proxy-….google.comGoogle special-case crawlersNo
google-proxy-….google.com or ….gae.googleusercontent.comGoogle user-triggered fetchers (a person asked a Google product to fetch your page)Not by range
151.86.12.34.bc.googleusercontent.comA Google Cloud customer’s virtual machine: could be a monitor, a SaaS, a researcher or a scannerPer address, if it misbehaves

The pattern comes from Google’s verifying Google requests page. The important detail is that googleusercontent.com covers both: blocking the whole domain or all Google Cloud ranges can also block Google’s user-triggered fetchers and every legitimate service that runs on Google Cloud.

What one day of our log looked like

srvscripts.com, 20 October 2026, 35,400 requests, classified by the script below:

ClassRequests403/404 answersAddresses
Google crawlers (common and special lists)698124
Google user-triggered fetchers141012
Google Cloud customers (cloud.json)28124
Everything else34,6601342,066

Sample DNS checks from the same log (reverse lookup, then a forward lookup of the name):

66.249.67.34    -> crawl-66-249-67-34.googlebot.com.          -> 66.249.67.34
142.250.33.35   -> google-proxy-142-250-33-35.google.com.     -> 142.250.33.35
34.12.86.151    -> 151.86.12.34.bc.googleusercontent.com.    -> 34.12.86.151

Our site is small, so Google Cloud customers made only 28 requests that day. The addresses to act on are the ones with many 404s or probing requests, not the range.

Check one address

ip=34.12.86.151
name=$(dig +short -x "$ip"); echo "$name"
dig +short "$name"      # must return the same IP

A name ending in googlebot.com or google.com that resolves back to the same address is Google itself. A name ending in bc.googleusercontent.com is a Cloud customer. Anything that does not resolve back is not verified, whatever its User-Agent says. For a quick check in the browser, use our reverse DNS lookup.

Classify a whole log

This Python 3 script (standard library only) downloads Google’s current IP lists, sorts every client address in an access log into the four classes, and lists the busiest Google Cloud customer addresses with their 403/404 count. It only reads the log.

#!/usr/bin/env python3
"""google-ip-classify.py - split the clients in a web access log into Google crawlers,
Google user-triggered fetchers, Google Cloud customers and others. Read-only.
Usage: python3 google-ip-classify.py /var/log/httpd/domains/example.com.log [--top 15]"""
import sys, json, re, ipaddress, urllib.request, collections

FILES = {
    'google-crawler': ['https://developers.google.com/static/crawling/ipranges/common-crawlers.json',
                       'https://developers.google.com/static/crawling/ipranges/special-crawlers.json'],
    'google-fetcher': ['https://developers.google.com/static/crawling/ipranges/user-triggered-fetchers.json',
                       'https://developers.google.com/static/crawling/ipranges/user-triggered-fetchers-google.json',
                       'https://developers.google.com/static/crawling/ipranges/user-triggered-agents.json'],
    'google-cloud-customer': ['https://www.gstatic.com/ipranges/cloud.json'],
}

def load(url):
    with urllib.request.urlopen(url, timeout=20) as r:
        d = json.load(r)
    return [ipaddress.ip_network(p.get('ipv4Prefix') or p.get('ipv6Prefix')) for p in d['prefixes']]

def main():
    if len(sys.argv) < 2:
        print(__doc__); sys.exit(2)
    top = int(sys.argv[sys.argv.index('--top') + 1]) if '--top' in sys.argv else 15
    nets = {k: [n for u in v for n in load(u)] for k, v in FILES.items()}
    line_re = re.compile(r'^(\S+) \S+ \S+ \[[^\]]+\] "(?:\S+) (\S+)[^"]*" (\d{3})')
    hits = collections.Counter(); bad = collections.Counter(); cls = {}; paths = collections.defaultdict(collections.Counter)
    with open(sys.argv[1], errors='replace') as f:
        for line in f:
            m = line_re.match(line)
            if not m:
                continue
            ip, path, status = m.group(1), m.group(2), m.group(3)
            if ip not in cls:
                try:
                    a = ipaddress.ip_address(ip)
                except ValueError:
                    continue
                cls[ip] = next((k for k in ('google-crawler', 'google-fetcher', 'google-cloud-customer')
                                if any(a in n for n in nets[k] if n.version == a.version)), 'other')
            hits[ip] += 1
            if status in ('403', '404'):
                bad[ip] += 1
            paths[ip][path.split('?')[0]] += 1
    summary = collections.Counter(); summary_bad = collections.Counter()
    for ip, n in hits.items():
        summary[cls[ip]] += n; summary_bad[cls[ip]] += bad[ip]
    print('%-22s %8s %8s %6s' % ('CLASS', 'REQUESTS', '403/404', 'IPS'))
    for k in ('google-crawler', 'google-fetcher', 'google-cloud-customer', 'other'):
        print('%-22s %8d %8d %6d' % (k, summary[k], summary_bad[k], sum(1 for i in cls if cls[i] == k and i in hits)))
    print('\nTop Google Cloud customer IPs (candidates to rate-limit or block one by one):')
    print('%-40s %8s %8s  %s' % ('IP', 'REQUESTS', '403/404', 'MOST REQUESTED'))
    for ip, n in sorted(((i, hits[i]) for i in hits if cls[i] == 'google-cloud-customer'), key=lambda x: -x[1])[:top]:
        p, c = paths[ip].most_common(1)[0]
        print('%-40s %8d %8d  %s (%d)' % (ip, n, bad[ip], p[:60], c))

if __name__ == '__main__':
    main()

On DirectAdmin the current day’s log is /var/log/httpd/domains/example.com.log; earlier days are in the user’s domains/example.com/logs/ archives. Run it with nice on a busy server.

Block the addresses that misbehave

Block single addresses that show scanning behaviour (hundreds of 404s, requests for .env, wp-config backups or random .php files), preferably for a limited time. With CSF:

csf -td 34.12.86.151 86400 "GCP scanner: 404 flood"     # 24 hours
csf -d 34.12.86.151 "GCP scanner: 404 flood"          # permanent

With Imunify360, which manages blocks itself when it integrates with CSF:

imunify360-agent blacklist ip add 34.12.86.151 --comment "GCP scanner: 404 flood"

We took both command forms from the tools’ own help output on our server but did not run the blocks there. Blocking in the firewall also stops that address from reaching mail and other services, which is usually what you want for a scanner.

  • Do not block cloud.json as a whole. It held 1,128 prefixes when we checked. Uptime monitors, payment and webhook senders, feed readers and many SaaS tools run there.
  • Never block the crawler and fetcher lists. That removes your site from Google Search and breaks things like link previews.
  • Prefer rate limits to blocks for patterns. A rule that answers missing .php files before PHP starts, plus a fail2ban jail on repeated 404s, handles rotating addresses better than a growing deny list; see block PHP script scans without blocking visitors.
  • Report persistent abuse. Google Cloud accepts abuse reports for customer instances; include log lines with times in UTC.

How we tested

  • Log: one full day of srvscripts.com (20 October 2026, 35,400 lines, OpenLiteSpeed combined log format on DirectAdmin), classified with the script above on our production server, run with nice.
  • Google IP lists downloaded on 21 October 2026 (cloud.json 1,128 prefixes; common-crawlers 317; special-crawlers 272; user-triggered-fetchers 1,068; user-triggered-fetchers-google 504; user-triggered-agents 20).
  • DNS: reverse and forward lookups for two crawler, two fetcher and two Cloud customer addresses from the log.
  • Not tested: the CSF and Imunify360 block commands (we did not block anyone), and logs in other formats.

FAQ

Is bc.googleusercontent.com Googlebot?

No. It is the reverse DNS of Google Cloud customer virtual machines. Googlebot resolves to googlebot.com and back to the same address.

Can I block all of googleusercontent.com?

It is a bad idea. Google’s user-triggered fetchers can also resolve to gae.googleusercontent.com, and many legitimate services run on Google Cloud.

Where are Google’s official IP lists?

Crawlers and fetchers: the JSON files under developers.google.com/static/crawling/ipranges/. Google Cloud customer ranges: www.gstatic.com/ipranges/cloud.json.

Will blocking a Cloud scanner affect my SEO?

No, as long as you block single Cloud customer addresses and leave the crawler and fetcher ranges alone.

At a glance

bc.googleusercontent.com in Your Logs: Googlebot or a Scanner? How to Tell and What to Block summary card: *.bc.googleusercontent.com addresses are Google Cloud virtual machines rented by anyone, not Googlebot. Google’s own cra…
In short: *.bc.googleusercontent.com addresses are Google Cloud virtual machines rented by anyone, not Googlebot. Google’s own crawlers resolve to googlebot.com or google.com, and its user-triggered f…

Share this guide

Free website test

Is your website set up right?

Check SSL, security headers, redirects, robots.txt, sitemap, llms.txt and security.txt in one test. It takes about 30 seconds.