Your cart is currently empty!
Questions to take to a lawyer before a collection project
Every scraping project I’ve run started with an engineering question: how do we get the data reliably. The projects that stayed out of trouble also started with a legal question: are we allowed to get this data this way. Those two conversations happen on different timelines. Engineering can start today. Legal needs to happen before you write a line of collection code, not after you’ve already got a pipeline pulling 50,000 pages a day.
This isn’t legal advice. I’m not a lawyer and this article isn’t a substitute for one. What I can offer, after years of running proxy infrastructure and scraping pipelines, is the list of questions that actually matter to a lawyer reviewing a collection project, and why each one maps to something real in how the scrape works. Bring this list to counsel. Don’t treat it as the answer.
Why “is scraping legal” is the wrong question
Lawyers get frustrated when a client asks “is web scraping legal” because the honest answer is “it depends on at least six variables you haven’t described yet.” Scraping itself is a technique. What matters is what you’re collecting, from where, how you’re accessing it, what you do with it after, and what agreements are in place between you and the site.
The useful version of “web scraping legal questions” is a list specific enough that a lawyer can actually give you a risk assessment instead of a shrug. That’s what this article is.
What data are you actually collecting
Before anything else, get specific about the data itself. “Product prices” and “user profile photos” carry completely different legal weight, even if the scraper code looks identical.
Ask your lawyer to help you draw the line between public business data (prices, stock counts, public reviews) and personal data (names tied to accounts, emails, phone numbers, anything that identifies a person). In most jurisdictions, personal data triggers a separate legal regime on top of whatever contract or computer-access law applies. In the EU that’s GDPR. In California it’s the CCPA/CPRA. Both regimes care about collection purpose, retention, and whether the person has any reasonable expectation the data stays where they put it.
If your project touches anything that could identify a real individual, that’s the first thing to flag, not the last.
What does the target site’s terms of service actually say
Pull the ToS and the robots.txt for every domain in scope before the meeting, not during it. Your lawyer needs to read the actual clauses, not a summary. Look specifically for:
- An explicit prohibition on automated access, scraping, or “harvesting”
- Rate limit or API-only access requirements
- Clauses about reselling or redistributing data pulled from the site
- Any account-creation terms you’d be agreeing to just by logging in to view the data
A site can have wide-open robots.txt and still have ToS language that bars scraping entirely. The two aren’t the same signal, and robots.txt has no legal force on its own, it’s a machine-readable request, not a contract. What matters legally is closer to whether you clicked through or otherwise agreed to terms, and whether courts in your jurisdiction currently treat ToS violations as breach of contract, trespass, or something with criminal exposure attached. That last one shifts by country and by year, which is exactly why this needs a lawyer reading current case law rather than a blog post.
Is there a computer-access law in play
In the US, this is the Computer Fraud and Abuse Act. Courts have gone back and forth for years on whether scraping public, logged-out pages counts as “unauthorized access” under the CFAA, and rulings on this have shifted meaningfully over time. That’s precisely the kind of question you need current counsel for rather than a cached answer, because the line moved multiple times in the last decade and could move again.
Ask directly: does accessing this site the way we plan to access it risk CFAA exposure, or the equivalent computer-misuse statute in the relevant country. Ask what changes the answer, like requiring a login, using credentials that aren’t yours, or continuing to access a site after receiving a cease and desist. A cease and desist letter is a meaningful legal event. If one ever arrives, stop and go back to counsel.
Does IP rotation or proxy use change anything
This comes up in almost every scraping legal conversation I’ve had, so raise it directly. Rotating through proxy infrastructure or distributing requests across many IPs doesn’t change what data you’re collecting or what agreement you may have violated to get it. It changes detection dynamics on the target’s side, not the underlying legal question. A lawyer assessing risk needs to know your actual access pattern (rate, volume, whether you’re circumventing an active block, whether you’re using residential IPs obtained through consent-based networks or otherwise) because some of those details matter to a court’s read on intent, but “we used proxies” is not itself a defense or an offense. Say what you’re actually doing and let counsel weigh it.
What happens to the data after collection
Collection is one legal event. Storage, use, and resale are separate ones. Walk your lawyer through the full pipeline: where the data lands, who has access, how long you keep it, whether it gets enriched with other datasets, and whether it’s going into a product you sell or license to third parties.
This matters a lot if personal data is anywhere in the pipeline. Purpose limitation is a real requirement under GDPR-style regimes, meaning data collected for one stated purpose can’t just get repurposed later without a fresh legal basis. If your business model involves reselling scraped datasets, that’s a different risk profile than internal competitive research, and your lawyer needs to know which one you’re building before they can size the exposure.
Has this target company litigated against scrapers before
Ask your lawyer to check. Some companies have a documented pattern of sending cease and desist letters or filing suit against scrapers, and that history is often public through court filings. Knowing a target has done this before doesn’t mean don’t scrape them, but it changes how carefully you want your contract terms, your data handling, and your response plan drawn up in advance.
What’s your actual authorization to build this
If you’re building this collection project for a client or employer rather than yourself, get the scope in writing before you start. What domains, what data fields, what volume, what’s out of bounds. This protects you if the legal exposure ends up landing on the person who commissioned the work rather than the person who wrote the scraper. It also gives your lawyer a document to review instead of a verbal description that might drift between the person who asked for the project and the person who explains it to counsel.
Bring a written description, not a demo
The single most useful thing you can do before the meeting is write down, in plain language, exactly what the pipeline does: target domains, data fields, access pattern, volume, storage, and what happens to the data downstream. Lawyers can give sharper answers to a page of specifics than to a live walkthrough of code, because the written version forces you to actually decide what you’re building instead of describing it in the moment.
None of this replaces the conversation itself. It just makes sure the conversation covers the things that actually determine your risk, instead of circling “is scraping legal” for an hour and walking out with nothing you can act on.
If you’re building out the infrastructure side of a collection project, we cover proxy setups, pipeline orchestration, and how bot detection actually works from the defensive side over at Data Research Tools.
Get new guides and videos first — join the Telegram channel.
Leave a Reply