TLS handshakes and what they reveal about a client

Why the handshake matters before a single HTTP byte moves

Every conversation about bot detection eventually lands on headers and cookies, but a site can flag a client before it ever sees a User-Agent string. That happens during the TLS handshake, the negotiation that sets up encryption for the connection. It’s a layer most scraping setups never think about, because it’s handled by whatever library or runtime is making the request, not by the code the developer wrote. That’s exactly why it’s useful for detection: it’s hard to fake without deliberately engineering it.

If you run proxy infrastructure or scraping pipelines for any length of time, you eventually hit a wall where requests get blocked even though the headers look perfect, the IP is clean, and the request rate is reasonable. Nine times out of ten in those cases, the answer is sitting one layer down, in how the TLS handshake was constructed.

What’s actually inside a ClientHello

When a client opens a TLS connection, the first thing it sends is a ClientHello message. That message carries a specific, ordered set of fields:

• the TLS version it supports
• a list of cipher suites, in the order the client prefers them
• supported elliptic curve groups
• signature algorithms
• extensions, including ALPN (which advertises HTTP/2 or HTTP/1.1 support), supported versions, and key share data
• sometimes GREASE values, which are dummy entries browsers insert to force servers to handle unknown values gracefully

None of this is random. Each TLS library, whether it’s OpenSSL, BoringSSL, LibreSSL, or a language-specific stack like Python’s ssl module, builds this list in a fixed order determined by its own code. Chrome’s BoringSSL build orders and populates these fields differently from Firefox’s NSS, which is different again from what Python’s requests library produces through urllib3 and OpenSSL, which is different again from Go’s net/http.

The result is that the handshake alone, before any application data flows, tells a listening server what software actually opened the connection, or at least which TLS library and version it’s running.

JA3 and JA4: turning a handshake into a fingerprint

JA3 is the method that made this practical to use at scale. It takes the ordered fields from the ClientHello (TLS version, cipher list, extensions, elliptic curves, curve formats), concatenates them, and hashes the result into a single string. Two clients running the exact same TLS stack with the exact same configuration produce the same JA3 hash, regardless of IP, headers, or cookies.

JA3S is the server-side equivalent, built from the ServerHello, and less relevant to scraping since it’s the site’s own fingerprint, not the visitor’s.

JA4, the newer version, changes the hashing approach to be less brittle. JA3 could be thrown off by extension reordering or GREASE insertion in ways that didn’t reflect a real difference in client behavior. JA4 normalizes some of that and adds ALPN and other signals into the fingerprint. The result is a steadier identifier across TLS library versions, and one that’s harder to dodge by accident.

Neither JA3 nor JA4 identifies a specific person or device. They identify a software configuration. A JA3 hash tells a server “this connection came from a client using cipher suite order X with extension set Y,” which usually maps to “this is curl 8.x” or “this is Chrome 120 on BoringSSL” or “this is Python’s requests library with default urllib3 settings.”

Where this breaks scraping setups

Most HTTP client libraries used in scraping pipelines, requests, aiohttp, plain urllib, axios with the default Node TLS stack, produce a TLS fingerprint that doesn’t match any real browser. That’s fine for APIs that don’t check it. It becomes a problem the moment a target site compares the JA3/JA4 fingerprint against the User-Agent header a request claims to have.

If the User-Agent says “Chrome 120 on Windows” but the JA3 hash matches Python’s default OpenSSL cipher order, that’s an inconsistency a detection system can flag cheaply, without needing behavioral signals, IP reputation, or JavaScript challenges. It’s a single comparison against a known table of browser fingerprints. This is why operators sometimes see a proxy pool with clean residential IPs still get blocked at a much higher rate than expected: the IP was never the tell, the handshake was.

Running requests through a real browser engine, via a browser automation framework that drives actual Chrome or Firefox, doesn’t have this problem, because the TLS stack really is Chrome’s or Firefox’s. The handshake matches the User-Agent because both come from the same binary. That’s one reason browser automation tools remain heavily used in scraping despite being slower and more resource-hungry than raw HTTP clients: the TLS layer is authentic by construction, not by imitation.

TLS-impersonating clients, and their limits

There’s a category of tooling built specifically to make non-browser HTTP clients produce browser-matching TLS fingerprints. Examples include curl-impersonate and the various “tls-client” libraries. They work by patching or recompiling the underlying TLS library to replicate a specific browser’s cipher order, extension list, and GREASE behavior, so the resulting JA3/JA4 hash lines up with a real Chrome or Firefox release.

Worth being straight about what this does and doesn’t solve. It closes the specific TLS-layer/User-Agent mismatch described above. It does not make a client indistinguishable from a real browser. Detection systems that care about this layer typically don’t stop at TLS. They cross-check it against the HTTP/2 fingerprint (frame ordering, settings values, header compression behavior), the actual header order and casing, and whether the client executes JavaScript at all. A request with a perfect JA4 hash but no JS execution, or with an HTTP/2 SETTINGS frame that doesn’t match the claimed browser version, still stands out. And because these impersonation libraries target specific browser versions, they need active maintenance; a fingerprint frozen to a six-month-old Chrome release starts looking stale the moment that version stops being common in the wild.

There’s also a maintenance cost most people underestimate: browsers update their TLS behavior with new releases, sometimes multiple times a year, as cipher suites get deprecated or extension support changes. A JA3/JA4 hash that matched “current Chrome” in January can look anomalous by summer if the impersonation layer isn’t kept in sync with real browser release cycles.

What this means for pipeline design

None of this is about finding a trick that makes traffic invisible, because that’s not how the layer works. It’s about consistency. A request’s TLS fingerprint, HTTP/2 behavior, header set, and claimed User-Agent should all describe the same piece of software. Mismatches between those layers are cheap for a detection system to catch and expensive for an operator to diagnose after the fact, because the failure shows up as a block rate, not an error message pointing at the actual cause.

Practical implications for anyone running scraping infrastructure at any scale: know which TLS stack your HTTP client actually uses, don’t assume a proxy or IP reputation problem is the cause of an elevated block rate without checking the handshake layer first, and if using a browser-impersonating client, track whether its fingerprint set is still current against real browser releases. None of this guarantees a request goes through. It just removes one of the easiest, cheapest signals a site can use to tell you apart from what you claim to be.

If you want more breakdowns like this on the infrastructure layers behind scraping and bot detection, check out the rest of what we cover.

Get new guides and videos first — join the Telegram channel.