Scraping Google Shopping with sh-dgr__content Selector (2026 Guide)

Google Shopping’s sh-dgr__content selector is the anchor point for every product card in the results grid, and if you’re building a price monitor in 2026, it’s the first CSS class you need to understand. Google has shuffled its Shopping HTML structure several times over the past two years, but this class has remained stable enough to be a reliable extraction target — as long as you know which child selectors to reach for and when to rotate your IPs.

What sh-dgr__content Actually Is

Each product tile in a Google Shopping results page sits inside a div.sh-dgr__content wrapper. Within that wrapper, the child class you’ll spend most of your time with is a8pemb, which Google uses for the clickable product link and title anchor. The combination of sh-dgr__content and a8pemb gives you a reliable two-step selector that survives most minor DOM tweaks.

The HTML structure, simplified, looks like this:

<div class="sh-dgr__content">
  <a class="a8pemb" href="/shopping/product/...">
    <h4 class="translate-content">Blue Mechanical Keyboard</h4>
  </a>
  <div class="a8Pemb-price">$49.99</div>
  <span class="E5ocAb">4.3 ★ (212)</span>
</div>

Note the case sensitivity: a8pemb on the anchor and a8Pemb-price (capital P) on the price container. Mixing these up is the single most common reason scrapers return empty price fields.

Extracting Products with Python and BeautifulSoup

For a straightforward batch scrape, BeautifulSoup handles the parsing cleanly. Playwright or Puppeteer are better choices when Google serves a JS-rendered grid, but for cached SERP HTML fetched via a proxy API, this is enough:

from bs4 import BeautifulSoup

def parse_shopping_cards(html: str) -> list[dict]:
    soup = BeautifulSoup(html, "lxml")
    results = []
    for card in soup.select("div.sh-dgr__content"):
        title_el = card.select_one("a.a8pemb h4")
        price_el = card.select_one("div.a8Pemb-price")
        rating_el = card.select_one("span.E5ocAb")
        results.append({
            "title": title_el.get_text(strip=True) if title_el else None,
            "price": price_el.get_text(strip=True) if price_el else None,
            "rating": rating_el.get_text(strip=True) if rating_el else None,
            "link": card.select_one("a.a8pemb")["href"] if card.select_one("a.a8pemb") else None,
        })
    return results

Run this against a live fetch and you’ll typically get 20 to 30 product records per page. If len(results) == 0, you hit a CAPTCHA wall or a bot-detection interstitial — not a selector miss. Check the raw HTML first before blaming the parser.

For broader context on selector-based scraping across Google properties, the full breakdown in How to Scrape Google Shopping Results for Price Monitoring covers pagination, URL parameter control, and currency normalization in depth.

Handling Bot Detection and CAPTCHAs

Google Shopping is one of the harder Google surfaces to scrape at volume. It uses a layered detection stack: user-agent fingerprinting, TLS fingerprint checks, behavioral scoring, and IP reputation. Residential rotating proxies are non-negotiable above roughly 500 requests per day. Datacenter IPs get flagged within minutes on Shopping — Google appears to be more aggressive here than on web search.

The same infrastructure logic applies when scraping other Google surfaces. Best Proxy Types for Scraping Google Maps and Local Pack (2026) walks through the proxy tier tradeoffs in detail, and the conclusions carry over directly to Shopping.

Recommended proxy and rendering combinations by volume:

At the 20k+ tier, also add request delays with jitter (1.5 to 4 seconds between requests per proxy thread) and rotate Accept-Language headers to match your target geo.

Structuring a Price Monitoring Pipeline

For ongoing monitoring rather than a one-shot scrape, you need a schedule, a delta detector, and a storage layer. Here’s the minimal pipeline shape that holds up in production:

1. Fetch layer — Playwright headless with a residential proxy pool. Rotate IPs per request, not per session.
2. Parse layer — BeautifulSoup on the raw HTML using the sh-dgr__content / a8pemb selector pair above.
3. Storage layer — Postgres or BigQuery. Store raw HTML alongside parsed fields so you can re-parse when Google changes the DOM.
4. Delta detection — Compare current price to previous snapshot. Alert on changes over a configurable threshold (e.g., ±5%).
5. Retry layer — On CAPTCHA or empty parse, backoff and retry from a different IP. Log failure reason, not just failure count.

Storing raw HTML is the step most people skip and later regret. DOM changes are inevitable, and having the source lets you backfill without re-fetching.

The same pipeline logic — storing raw HTML, delta detection, retry handling — applies outside Shopping. Do Proxies Help Daily Housing Listing Monitoring? Real-World Test documents what breaks in production when you skip these layers on a high-frequency scrape, and the failure modes are nearly identical.

Common Errors and What They Mean

• sh-dgr__content returns 0 results: you have a CAPTCHA page, a “did you mean” redirect, or a consent interstitial. Print soup.title.text to confirm.
• a8pemb link exists but href is relative (starts with /shopping/): normal. Prepend https://www.google.com before storing.
• Price field is None for some cards: some listings are price-range or “check site” placements. These have a different price container class. Don’t error out — just log as null.
• Title returns garbled text: Google wraps titles in a translate-content class that can include hidden spans for translation fallback. Use .get_text(strip=True) and strip non-printable characters.

For comparison, structured data selectors on other review platforms behave differently but the error pattern taxonomy is similar — Scraping Airbnb Reviews with data-review-id Selector (2026 Guide) covers the same “selector returns empty, why?” debugging workflow applied to a different target.

If you’re scaling to review aggregation across multiple platforms alongside Shopping data, How Proxies Help Scrape Reviews at Scale: Yelp, Google, Trustpilot (2026) has the proxy pool sizing math worth reading before you provision infrastructure.

Bottom Line

Target div.sh-dgr__content as your container and a.a8pemb as your product link selector — that combination is the most stable extraction point on Google Shopping in 2026. Use residential rotating proxies from the start, store raw HTML alongside parsed fields, and build retry logic that distinguishes between a selector miss and a CAPTCHA wall. DRT will keep tracking selector stability as Google rolls out Shopping UI updates through the year.

Related guides on dataresearchtools.com

• Scraping Airbnb Reviews with data-review-id Selector (2026 Guide)
• Best Proxy Types for Scraping Google Maps and Local Pack (2026)
• Do Proxies Help Daily Housing Listing Monitoring? Real-World Test
• How Proxies Help Scrape Reviews at Scale: Yelp, Google, Trustpilot (2026)
• Pillar: How to Scrape Google Shopping Results for Price Monitoring