A scraper is not judged only by whether its fingerprint looks like Chrome, but by whether its IP, cookies and connection history continue to describe the same believable browser.
When a scraper starts getting blocked, the standard advice is predictable.
Change the User-Agent. Add the missing headers. Rotate the proxy. Generate new cookies. Try another Chrome profile. If none of that works, rotate everything more often.
But every one of those components can look valid on its own while becoming contradictory when combined. A Chrome User-Agent can be paired with the wrong TLS behavior. A valid cookie can appear from the wrong IP. One persistent cart token can jump between five supposed devices.
That problem becomes more visible at production scale. One request may look perfectly ordinary. A thousand sessions following the same sequence, timing and teardown can reveal the automation template behind them.
As Saksham Solanki, creator of the open-source HTTP client httpcloak, put it during our recent Reddit AMA:
Rotating the pieces separately manufactures a client that doesn't exist anywhere in the real world, and that's easier to spot than whatever you were trying to hide.
Saksham built httpcloak, a Go HTTP client designed to reproduce browser behavior across TLS, HTTP/2, HTTP/3 and the connection lifecycle, while working against a Cloudflare-protected target that was scoring on TLS and running HTTP/3. His experience comes from capturing browser traffic, comparing Chrome's networking behavior at the frame and byte level, and repeatedly correcting cases where httpcloak passed every public fingerprint test but still differed from Chrome underneath.
In our sixth r/WebScrapingInsider AMA, we asked Saksham why apparently browser-identical clients still get blocked, how identity breaks across proxies and sessions, where HTTP clients stop being sufficient, and what browser-impersonation product claims hide.
Here are the nine biggest insights from the discussion.
1. Rotating More Can Make Your Scraper Easier to Detect
When blocking increases, most scraping teams rotate more.
But a real browser does not randomly change its identity one attribute at a time. Its User-Agent, TLS behavior, headers, IP, cookies and session history belong together. Rotating those pieces independently can create a client that looks less natural with every attempted fix.
Asked about the most misleading advice given to developers trying to avoid blocks, Saksham pointed straight at this pattern:
Rotation only works if you rotate a whole identity at once, fingerprint, IP, cookies, headers, all of it together, and then let that identity live a while. Rotating the pieces separately manufactures a client that doesn't exist anywhere in the real world, and that's easier to spot than whatever you were trying to hide.
The failure mode is not rotation itself. It is rotating one dimension while the others stay put.
The problem is people rotate one dimension at a time. You rotate the UA but the TLS fingerprint stays put, so now you've got one JA4 claiming to be five different browsers, which is a much louder signal than just being one consistent wrong browser.
The same logic applies at every layer. A real browser does not randomize its core headers on every request, so per-request header rotation creates changes that match no actual browser session. Cookies and anti-bot tokens may be tied to the IP and session that established them, so moving them between rotating exits disconnects the application identity from its network history. And swapping browser profiles mid-run recreates the original problem:
If you're rotating profiles per request you're back to changing identity mid session, which is the thing that gets you flagged in the first place.
The stronger model that emerges from the AMA is to treat a scraping identity as a bundle: browser and OS profile, User-Agent and client hints, stable headers, IP and proxy session, cookie jar, challenge tokens, TLS tickets and connection state, and request history. Rotation happens between complete identities, at deliberate boundaries, with each identity allowed to persist for a believable period.
2. Matching Chrome Once Is Easy. Behaving Like Chrome Across 1,000 Sessions Is the Hard Part.
A scraper can be indistinguishable from Chrome for one request and conspicuously automated across a thousand sessions.
The difference is not necessarily the TLS signature. It is what every session does after the connection has been established.
Because matching chrome for one handshake is actually the easy part now, everyone can do that. What gets blocked at scale is looking identical across a thousand sessions, and that's a lifecycle problem, not a handshake problem.
It helps to separate three levels of apparent browser equivalence. Handshake equivalence: does the initial TLS exchange resemble the browser? Request equivalence: do the headers, ordering and initial frames resemble the browser? And lifecycle equivalence: does the client behave plausibly across the connection, the session and repeated identities?
Most tools and public tests concentrate on the first two. According to Saksham, large-scale reliability increasingly depends on the third: how many requests you send after connecting, the order in which you touch resources, pacing, concurrency, whether connections are reused or dropped, when cookies are generated and how quickly they are exercised, and whether every session performs the same warmup and the same teardown.
If you're doing the exact same thing every time after making the connection, same number of requests, same order, same pacing, same teardown, then the profile starts to become stale and you'd start getting blocks on the first request itself.
This matters more as you scale, because the target receives more examples of the underlying template. Adding proxies or nominally unique sessions does not add behavioral diversity if every identity executes the same state machine.
Saksham was candid that he learned this by getting it wrong:
I had thought if my connection matches 1:1 with Chrome on the wire then I cannot be blocked, but I learnt that the lifecycle of a connection matters just as much.
It also reframes what a "stale profile" means. Sometimes the profile is visibly outdated against current browser versions. But a current profile repeatedly attached to the same recognizable behavior goes stale too, and that version of the problem is not fixed by selecting a newer JA4.
3. A Perfect JA4 Result Can Still Hide the Bytes Giving You Away
At one point, httpcloak had a fingerprinting defect that every normal test said did not exist.
The decoded headers matched Chrome. Their order matched Chrome. The fingerprint result looked perfect. The bytes were still different.
My HPACK encoder was indexing headers differently to chrome, cookie and :authority in particular. Decoded headers came out identical, header order identical, so every fingerprint tool said perfect match. The bytes on the wire were different.
The reason this can happen is that the popular fingerprint metrics are summaries, not proofs.
JA4 sorts the TLS extension list before hashing. That gives it stability across Chrome's per-handshake extension shuffling, but it also means the hash cannot reveal the order actually transmitted. JA3 and JA4 hash which extensions are present, not their contents. An Akamai-style HTTP/2 fingerprint is a snapshot of the first few frames, settings, window updates, priorities and pseudo-header order. It says nothing about how the client behaves after that. And a testing endpoint only ever sees the isolated request you send it.
That leaves a long list of places where a client can still differ from Chrome: extension order and contents, HPACK representation, TLS record sizes, DATA frame sizing, flow-control cadence, stream resets, per-resource priorities and the whole connection lifecycle after the first request.
Nothing would have caught it except diffing raw frames against a real chrome capture.
None of this makes the metrics useless. It makes them a floor rather than a ceiling.
Matching JA3/JA4 gets you past the cheapest check they run, and you do need it, but it's one moment out of a whole session.
The buyer-level implication matters too: a tool can truthfully claim to match popular fingerprints without proving that it reproduces the complete browser implementation on the wire.
4. A Genuine Chrome Fingerprint Does Not Simply "Burn"
When a working scraper starts failing, developers often say its fingerprint has burned.
But a genuine Chrome fingerprint is not exclusive to scraping tools. It is shared by every real user running the same browser generation on the same OS. Blocking the signature indiscriminately would mean blocking part of the legitimate Chrome population.
Mimicking Chrome perfectly cannot be a signal, your Chrome and someone else's share the same signals only when you're on the same version and OS. What actually differs is behaviour, that's hard to implement over a lot of connections.
At its sharpest, Saksham's formulation was absolute:
Also, a correct TLS signature is never invalidated.
That claim needs immediate qualification, and the AMA itself supplies it. Saksham also recommends swapping to a Chrome profile with a different JA4 as a diagnostic: if chrome-151 is failing, try chrome-146 or chrome-133.
If one of those goes through then it's likely the JA4 that got flagged.
The way to reconcile the two statements is to separate what "burned" can actually mean. A signature can be rejected by a particular target or contextual rule without being globally invalid. A profile swap can help isolate that failure, but it also changes version-linked headers and other surrounding characteristics, so a temporary recovery does not prove the original TLS signature became universally malicious.
There is a related reason to stay inside the Chrome population rather than reaching for less-impersonated browsers. Being Chrome means being the majority. Moving to Firefox or Safari puts you in a much smaller pool where unusual behavior is more individually visible.
The practical conclusion: a genuine Chrome fingerprint is unlikely to become malicious in isolation. The context and behavior attached to it may be what has become recognizable.
5. QUIC Does Not Hide the Fingerprint. It Moves More of It Into Your Code.
HTTP/3 is encrypted, modern and designed differently from TCP-based HTTP.
That does not make the client harder to fingerprint. According to Saksham, it does the opposite.
QUIC didn't shrink the fingerprint surface, it moved it up into the application and made it bigger.
The architectural reason is where the transport lives. With TCP, the operating-system kernel owns much of the transport behavior, which limits what a user-space HTTP library can change and lets a proxy exit replace the TCP-side behavior entirely. With QUIC, the transport is implemented in the client library itself: which transport parameters are sent, their values and order, packet sizes and padding, packet coalescing, initial-packet construction, HTTP/3 SETTINGS, GREASE parameters and frames, QPACK behavior and per-request priorities. All of it is yours to control, and all of it is yours to get wrong.
QUIC can even expose history. Saksham gave the example of a transport parameter Chrome sends only on a resumed 0-RTT connection, carrying an actual measured round-trip time from a previous one.
That's a number on the wire making a claim about your past. Send it at the wrong time, or never send it, and you're saying something about yourself.
That creates a temporal consistency problem no static preset solves. Send historical state on a connection that should be new and the history is implausible. Never send resumption state and a supposedly long-lived browser identity looks unusual. Either way, the client's current claim has to match its own past.
The final trap is judging HTTP/3's importance by origin adoption statistics.
Most sites you'd scrape don't even offer it. But the protection layer sitting in front of them often does, and that's the connection getting graded, so it matters more than the raw adoption numbers suggest.
6. Millions of Proxy IPs Cannot Hide One Repeated Client
A proxy can change where the request appears to come from.
It cannot change the HTTP client making the request.
A proxy can't change your TLS or HTTP/2. What it does change is the TCP/IP fingerprint, which is per OS, and through a proxy that's the exit node's OS rather than yours.
It helps to hold two identities apart. The proxy changes the network identity: exit IP, reputation, ASN, geography, residential or datacenter classification, and the TCP/IP characteristics visible from the exit. Those differences are real. Saksham cited burned IPs, low ASN spread and datacenter classification as reasons one provider fails where another succeeds.
What the proxy does not touch is the client identity: TLS handshake behavior, HTTP/2 settings and frames, header encoding, HTTP/3 implementation details, cookies and tokens, request sequence, pacing and connection lifecycle.
That explains why two apparently conflicting industry claims can both be true. IP reputation may dominate on a particular target, and switching providers may still do nothing when the client or session behavior is the problem.
Also I had thought proxy would be a magical solution, if I have millions of IPs then they can't catch me, I couldn't have been more wrong haha.
The lesson is that network diversity does not create behavioral diversity. If every exit runs the same session template, you have distributed one client rather than created millions of believable users.
None of this makes the proxy layer unimportant. Pool size still affects concurrency, coverage and rate distribution, and Saksham was clear about the weight of the IP itself:
And underneath all of it your IP's reputation, which often matters more than the rest combined.
The insight is narrower and more useful: pool size cannot repair the other identity layers.
7. A Blocked Cart API May Think You Are Committing Fraud, Not Scraping
A cart API is not just another page to scrape.
It writes state, persists an identity and can affect transactions. The system defending it may care less about whether your TLS fingerprint is perfect than whether the cart behaves like it belongs to one real user.
The case came from a participant whose cart API requests kept failing despite rotating headers and cookies. Saksham's diagnosis started with a question: where does the cart identity live? A cart has to be tied to something to persist — a login, a device ID, a guest cart token or a cookie — and a HAR capture of a normal browse-and-add-to-cart flow shows what the cart request carries that a cold request would not.
From there, the failure is usually one of two things. Rotate the cookies that hold the cart identity, and you create a brand-new cart on every request: a flood of cart creations with no browsing in front of them. Or keep the token fixed and rotate everything around it:
Or if it's a token you're keeping fixed while rotating headers and ips around it, then it's one identity showing up from a bunch of devices and locations at once, and that hits fraud rules instead of bot rules, those are stricter.
Either way, the standard anti-bot reflex amplifies the anomaly.
Either way rotating harder makes it worse.
The category distinction matters beyond carts. Search, listing and product-detail endpoints read information. Cart APIs create and modify application state.
Also cart endpoints write, and write endpoints always get defended harder than read ones.
Since this participant wanted availability data, Saksham suggested checking whether the same information appears in the search, listing or product-detail responses in that same HAR, because the read path is far easier to work with. Worth noting: this was a diagnosis from the participant's description, not an inspection of the target's actual rules. But the framework generalizes: classify protection by business function, not by URL or status code.
8. The Cheapest Stack May Use a Browser Only to Establish the Session
TLS impersonation cannot execute a mandatory JavaScript challenge.
But that does not mean every request after the challenge needs a full browser.
Asked when developers should reach for a browser instead of an HTTP impersonation client, Saksham inverted the question:
It should be quite the opposite, try to use httpcloak (or any other tls lib) wherever possible, only go for a browser when absolutely necessary. When you want to do automation which has a lot of paths that its kinda hard to manage or an anti bot is present then browser is preferred, though even in that case you should try to have a hybrid system of doing solves with browsers and then use the cookies with a tls library to do further fetches, saves time and cost.
The boundary is real, and Saksham did not minimize it. An HTTP client can reproduce transport and request behavior, but it cannot provide the JavaScript runtime, DOM environment and interaction telemetry a forced client-side challenge requires. Some targets leave no choice but a real browser, a solver service, or a reverse-engineered challenge implementation.
It's a part of the bypass, not the whole bypass, you still need to solve the challenge if they present any.
The hybrid architecture works because the browser's expensive output — solved cookies and challenge state — can be transferred to a lighter client, provided the identity that established that state is preserved: sticky proxy or stable IP, same cookie jar, consistent browser profile and headers, and a plausible request sequence. Break the binding and the transferred state stops being believable.
The commercial logic follows directly:
What you can do to cut costs is generate fewer cookies through solving and get more re-fetches out of each one.
That points to a more useful production metric than solves per second: successful data-bearing requests per solved, coherent session. It connects browser cost, solver cost, proxy cost, cookie lifetime and successful data retrieval in one number.
9. "Supports the Latest Chrome" Hides a Continuous Reverse-Engineering Operation
To the user, supporting a new Chrome version looks like adding one item to a dropdown.
To the maintainer, it looks like this:
Hardest part is the forks. httpcloak sits on six of them, utls, quic-go, net, http, qpack and udpbara. One chrome change can mean patching utls, tagging it, patching quic-go, tagging that, bumping both in go.mod, rebuilding the cgo layer, then updating the python, node and dotnet bindings. None of that is visible from outside, it just looks like a version bump, but that chain is most of the actual work.
A browser profile spans TLS behavior, cipher and extension behavior, HTTP/2 settings and frames, header compression, HTTP/3 and QUIC, connection lifecycle, version headers and client hints, plus the language bindings that ship it all. When Chrome changes one layer, the update propagates through the chain. A version label on a product page does not reveal whether every layer moved together.
Verification is its own burden, and it resists automation:
Second hardest is you can't unit test "does this look like chrome". You need a real capture and a byte diff, so verification stays manual in a way most libraries never have to deal with.
Browser support is also not symmetric. Saksham is open that Chrome gets more attention than Firefox or Safari: each family behaves differently rather than merely presenting different headers, and maintaining all of them against monthly releases is expensive. So "supports Firefox" is a claim worth probing: is it a current, fully verified protocol implementation or a partial preset?
He was also fair about the competition:
On the alternatives, I want to be fair here, curl_cffi and curl-impersonate are good and actively maintained. They've got H3 fingerprints, extension permutation, custom ja3 and akamai strings, a lot of ground is covered.
That candor points at the market implication. As accurate static handshake impersonation becomes available in more tools, the differentiating work shifts to whatever cannot be copied from a README: speed and quality of browser-version maintenance, raw frame accuracy, HTTP/3 depth, session lifecycle and verification methodology. And because httpcloak is maintained mostly by one person, the same depth that creates the differentiation also creates concentration risk — a real consideration for anyone adopting any impersonation tool.
Conclusion: The Fingerprint Was Never the Whole Identity
The instinct, when a scraper gets blocked, is to hunt for the one bad signal. The burned fingerprint. The flagged IP. The missing header.
The picture that emerges from this AMA is different. Every component can pass its own test while the combination fails, because the combination is what gets graded: whether the fingerprint, IP, cookies, connection history and behavior keep describing the same believable browser, request after request, session after session.
The less useful questions are "Which fingerprint should I use?" and "How often should I rotate?"
The better ones are:
- Does my client match Chrome on the wire, not just on the fingerprint test?
- Does each identity keep its fingerprint, IP, cookies and headers together for a believable lifetime?
- Do my sessions vary the way a population of real users would?
- Is the endpoint I am hitting protected as a bot problem or a fraud problem?
- What does one usable record actually cost across browsers, solves, proxies and retries?
As Saksham summarized:
So yeah the handshake is a gate, the session is a score. You need to pass the gate to get scored at all, but passing it perfectly doesn't save a bad score if your overall activity is "too small" or detectable over multiple sessions.
The practical starting point is the one he kept returning to: capture a HAR of a real browser session, diff your scraper against it byte by byte, and then judge your system the way the anti-bot does — not one request at a time, but as a population of sessions.