An internet law professor explains how computer-access rules, contracts and copyright shape real scraping decisions about accounts, access methods, suppliers and data.
The page is public, so scraping it must be fine.
The scraper only collects prices, and prices are facts.
The scraping API vendor sends the requests, so the legal side is their problem.
Each assumption answers part of the question. None answers all of it, because a scraping operation involves how you reach a site, what you agreed to, what you copy and what you do with it.
As Santa Clara University law professor Eric Goldman put it during our recent Reddit AMA:
For my students, I summarize the law of scraping with the following tagline: “Scraping is ubiquitous but not clearly legal.” This breaks my students’ brains. If it’s so ubiquitous, it must be clearly legal…right?
That is not a claim that all scraping is unlawful. It describes a widespread activity governed by overlapping rules that courts have not settled.
Eric is Associate Dean for Research and Professor at Santa Clara University School of Law, where he co-directs the Datta Center for High Tech Law. He has practiced internet law since 1994, taught it since 1996, and fielded his first client question about scraping around 1997.
In our twelfth r/WebScrapingInsider AMA, we asked Eric how US law treats public data, logins, anti-bot systems, scraping vendors and AI agents.
Here are the nine biggest lessons, connected to the decisions you actually make. Like Eric's answers, this covers US law and is general education, not legal advice for your particular operation.
The Legal Concepts Behind the Conversation
Most of Eric's answers come back to four questions. Here is the vocabulary behind them.
1. Was access authorized? The Computer Fraud and Abuse Act (CFAA) is a federal law that prohibits accessing a computer "without authorization" or "exceeding authorized access." States have similar computer crime laws, such as California Penal Code 502. Separately, trespass to chattels is an old common-law claim for using someone else's physical property, here their servers, without permission in a way that causes harm.
Eric often uses "TTC" as shorthand for all three together: federal statutes, state statutes and the common-law claim. When you see TTC below, read it as "the computer-access family of claims."
2. Was an agreement formed? A website's terms of service (TOS) can only work as a contract if you actually agreed to them in a way the law recognizes. A clickwrap asks you to take an affirmative step, such as ticking a box or clicking "I agree" while creating an account. A browsewrap just posts terms somewhere on the site, often behind a footer link, and assumes that using the site means accepting them. Courts are generally much more willing to enforce the first than the second.
3. What material or access control is involved? Facts, such as a price, are not protected by copyright. Product photos and written descriptions usually are. That distinction governs copying.
DMCA Section 1201 is a different question. It prohibits circumventing a "technological measure" that controls access to a copyrighted work, even if you never copy that work. Picture a page with a price, three product photos and an anti-bot challenge in front of it. Copyright asks what you took. Section 1201 asks what you got past.
4. What does the legal document actually establish? Not every document carries the weight its tone suggests. A cease-and-desist letter (C&D) is a demand from one party, not a court order. An indemnity is a promise that one party will cover the other's costs. It does not stop anyone else from suing. A preliminary ruling, such as an early injunction or a decision on a motion to dismiss, is a step in a case, not the final word on it.
1. Public Data Does Not Mean Unrestricted Access
"It's publicly available" is the most common defense of a scraping project. It is also the most incomplete.
Public visibility does matter:
The law might presume that a server owner impliedly permits ordinary Internet traffic merely by making the choice to connect to the open Internet.
But that only speaks to access. Eric's framework runs every scraping campaign through computer-access laws, copyright, the DMCA and contracts. A good answer under one does not carry over to the others.
In order for a scraping campaign to be permissible, it must survive each and every one of the doctrines enumerated above.
That does not mean every doctrine applies to every scraper, or that unsettled law proves illegality. It means "the page is public" settles one question of several. We find it useful to collapse Eric's list into three:
- How does access occur? Anonymous requests, a logged-in account, a bypassed challenge.
- What obligations exist? Terms you accepted, notices you received.
- What material is collected or used? Facts only, or copyrighted text and images, and whether you republish them.
2. Your Scraper's Legal Position Can Change Without Changing a Line of Code
Engineers tend to treat a scraper's risk as a property of the code. Same requests, same risk.
Eric's answers suggest otherwise. Advising a website owner, he suggested anti-scraping statements around the site, and explained why:
They may not be legally effective, but the legal goal is to make it so that the scraper can’t argue that they didn’t know their scraping was unwanted.
That points to three questions that are easy to blur:
- Agreement. Did you accept an enforceable contract?
- Knowledge. Do you know the website objects?
- Authorization. Does that objection legally withdraw permission to access the servers?
A cease-and-desist letter sits on that fault line. It is not proof its demands are legally correct. But in the Power Ventures litigation, the Ninth Circuit held that continued access after a C&D was unauthorized under the CFAA.
Eric thinks the Supreme Court's 2021 Van Buren decision may have weakened that. A footnote left open whether authorization limits must be technical or can come from contracts and policies. If terms of service cannot withdraw permission, a one-sided letter arguably should not either:
So one possibility is that the power of C&Ds has gone down post-Van Buren.
Until courts confirm that reading, though:
Until then, a C&D recipient most likely "knows" they have been told not to come back.
Our read: the letter does not automatically make your scraping illegal. It does change the facts a court would see.
3. Adding a Login Can Add More Than Cookies: It Can Add Contractual Obligations
A team scrapes product listings anonymously. Logged-in users see stock levels and seller details, so the team creates an account and moves the scraper behind the login.
Technically, that is a session cookie. Legally, it may be a contract.
Eric called TOS formation "one of the great unknowns" of scraping law, but he laid out the clearest end of the spectrum:
If a scraper (1) creates an account, (2) runs the scraping through that account, (3) as part of the account formation process agrees to the TOS in a way the law recognizes, and (4) the TOS restricts scraping, then odds are high that scraping would be a breach of contract.
Every condition matters. At the other end, he said, terms presented only as a browsewrap may be disregarded entirely. So this is not "all logged-in scraping is unlawful." It is a conditional assessment, and the conditions are things you can check.
Asked about scraping behind a login the TOS prohibits, Eric put the business risk first:
The most important "risk" is that it becomes easier for the website to block the accounts, so any investments made in setting up the accounts or configuring the scraper could be mooted more quickly.
4. "We Only Collect Prices" May Not Resolve the Risk of Bypassing the Page's Defenses
Price scrapers lean on a comforting fact: prices are not copyrightable. For copying, that is largely sound. For Section 1201, Eric warned it may not help:
The short answer is that extracting only the uncopyrightable prices would not change the 1201 analysis at all because there are other copyrightable elements on the page that the anti-bot system is nominally "controlling access" to.
If an anti-bot system guards a page with photos and descriptions, the question is what was circumvented, not what was saved. In Reddit v. SerpApi, a federal court in New York let Section 1201 claims proceed on the theory that Google's bot-detection system controls access to copyrighted Reddit content in search results. In Google v. SerpApi, a California court dismissed Google's own claims because Google had not adequately shown a copyright interest in the scraped results, while giving it a chance to amend part of its case.
Eric is troubled by these interpretations and does not treat them as settled:
I don't put a lot of stock in the rulings yet. They are preliminary and haven't been reviewed by an appeals court. But they are chilling in the sense that they highlight how 1201 could be the anti-scraping doctrine to end all other anti-scraping doctrines.
Many of these rules are being written in AI litigation, and interpretations aimed at AI scrapers can end up applying to everyone. Still, none of this means every CAPTCHA or anti-bot system is a Section 1201 measure. Eric said plainly that he is not sure, and that the law "is going to keep developing."
5. A Scraping API Outsources Requests, Not Necessarily Legal Exposure
Many teams never scrape directly. They call a scraping API or buy a dataset. Eric drew a sharp line between the two. On live APIs:
Well, SerpApi is being put through the legal wringer right now, and its API client Perplexity is a co-defendant of the Reddit lawsuit. So "hiding" behind a third-party API may not change the analysis much.
Buying a finished data product can look different:
The data vendor may face greater legal risk, but the buyer doesn't necessarily share that risk equally. For example, a C&D to the vendor may not restrict the customer's data usage.
"May" is doing real work. Claims involving copyrighted material from the vendor may hit the customer just as hard, and a court could take note if a customer "induced" the vendor to break the law.
There are operational tradeoffs too: the data may be less fresh, the price includes the vendor's margin, it may need cleaning, and supply stops if the vendor gets blocked. Using data you already have and receiving new data are separate questions.
His bottom line:
FWIW, I would never assume the "supplier has covered the legal side."
6. Your Customer Can Authorize the Tool While the Website Still Disputes Its Access
Imagine a product that imports a customer's order history from a marketplace using the customer's own login. The customer clicked "connect." Does that settle it?
Not necessarily. Asked whether a user or their AI assistant breaks a site's "no bots" rule, Eric said:
The short answer is: we don't know. Maybe the user broke the rules. Maybe the assistant broke the rules. Maybe both broke the rules. Maybe neither.
The live example is Amazon v. Perplexity. A district court blocked Perplexity's Comet agent from shopping on Amazon for users. In August 2026, the Ninth Circuit vacated that injunction, holding on the record before it that the user, not Perplexity, was accessing Amazon's computers under the CFAA. Eric does not see it as the end of the story:
I don’t consider the Ninth Circuit’s ruling the final word in this matter. Amazon has other legal claims beyond the CFAA, and other courts might reach a different conclusion about who is really in the driver’s seat.
He listed "who is in control of the scraper from a legal standpoint" among the field's unresolved questions. None of this makes customer-authorized automation prohibited. It means consent from one side does not answer questions on the other.
7. hiQ Won the Argument Everyone Remembers, but Still Lost the Case
hiQ v. LinkedIn is often cited as proof that scraping public data is legal. There is a reason: hiQ won an early injunction stopping LinkedIn from blocking it, and the Ninth Circuit twice said hiQ's scraping of public profiles likely did not violate the CFAA. Those rulings genuinely shaped thinking on public data.
Then the case kept going.
Despite some intermediate hiQ-favorable rulings, hiQ lost in the end on the contract breach.
In late 2022, the district court found hiQ had breached LinkedIn's User Agreement, which it had accepted when creating an account. The parties settled with a consent judgment that required hiQ to stop scraping LinkedIn and delete the data it had collected.
So the case was not a winner for hiQ, which makes it hard to celebrate as a pro-scraping victory.
Eric also called the initial order requiring LinkedIn to permit access anomalous, and doubts many courts will follow it. A CFAA win at the injunction stage said nothing final about contracts, and contracts decided the case.
8. "The Customer Is Responsible" Is Not a Liability Shield for Providers
Most proxy providers and scraping APIs call themselves tools whose customers are responsible for how they are used. Eric pointed to Ticketmaster v. RMG, a 2007 case he still teaches:
Even way back then, the court didn't buy the argument that the legal blame should fall only on the customer.
Provider terms can allocate financial responsibility, for example through an indemnity. But the website being scraped never signed them:
The courts will be much more interested in who actually did what, not in how the contract describes their relationship.
That shifts attention from labels like "infrastructure" to conduct. Public claims count too. Eric warned that touting the ability to defeat any anti-scraping measure could be used as evidence of circumvention and of knowing that access was unwanted. He acknowledged the dilemma: customers often buy precisely for that capability.
All of that marketing is evidentiary gold to an adversary.
Rewording marketing does not change what a product does. What you say and what you do can both end up in front of a court.
9. The Best Scraping Optimization May Be a Different Way to Obtain the Data
When a startup asked what he would want to know first about a new e-commerce scraping project, Eric mostly raised business questions, not doctrinal ones:
Does the target have an API, and would it be better to use the API rather than scraping?
And, whether or not an API exists:
For example, if we had a data access deal, might the data be fresher or require less cleaning?
He also asked whether the target syndicates its data elsewhere, offering another source or a backup.
Our interpretation: this turns the decision into a comparison. A scraper carries ongoing maintenance, cleaning and interruption costs. An API or licensing deal may deliver fresher, structured data with fewer surprises. And even a favorable legal position does not keep data flowing:
Plus, Amazon can always make countermoves to technologically block or control Comet, so a legal win doesn’t necessarily translate into business success.
Alternatives are not automatically legally safe. APIs have their own terms. And a scraper that has not been noticed is not the same as one that is lawful.
Conclusion: The Useful Questions Are the Ones You Control
You cannot settle the open questions in scraping law. You can control most of the decisions they apply to: how you access a site, which terms you accept, how you respond when a site objects, which suppliers you rely on and what you do with the data.
Eric's answers rarely give clean verdicts. They show which distinctions matter and which questions need specific assessment.
As Eric summarized:
Notice how few of my questions dive deeply into the arcanity of the law of scraping. Once we start digging into that, we’re not going to get very comforting or satisfactory answers.
Write down how your scraper actually reaches each target, what you have agreed to, and where the data goes. That description is where any conversation with a lawyer should start.