Recommendations for hosting a web scraping service without getting sued

jimBlackHat

Newbie
Joined
Jan 14, 2015
Messages
3
Reaction score
1
Hi,

right now I am developing a web scraping service where people can scrape pages like yellow pages, ebay, amazon and all kind of shops for data they need. Everything runs on my server.

A bit more detailed: People register on my site an pay for the service. They now can scrape data from pages and download this data as excel, csv, xml, etc. I simply turn unscructured data into lists.

I have talked to a lawyer and he said that it would be a potential risk to start such a service in germany (this is where I live) because of "competition regulations" and "copy right".

Now I am looking for possibilities to avoid getting sued by shop owners that my potential customers will scrape.

Some ideas:
- What about setting up a limited (LTD) company?
or
- Use a web hoster thats not located in germany for registerting the domain name and running the server?

There are already several websites that are providing such kind of scraping service. I wonder how they manage this problem.

Any ideas are welcome!

EDIT: Another solution. Always use open proxy server and fake the user agent. Good or bad idea?

Thanks,
jim
 
Last edited:
There are many legit companies scraping content (search engines and sites like archive.org). If they can do it why can't you?
 
I don't think theres anything wrong with scrapping content. It's what you do with that content after it's scrapped
 
Scrapping contents or scrapping sales profile?

I don't fully understand what you're talking about, but if it's about scrapping content, a link back, even no follow is acceptable.

The truth is that, if you decide to scrap big companies like the ones you mentioned above, they can easily report your website to google and it will be deindexed faster than you can ever imagine
 
Sorry for the short description of my project. Here comes a bit more detailed description (edited first post too):

People register on my website and have to pay for the service. They now can scrape data from pages and download this data as excel, csv, xml, etc. My service simply turns unscructured website-data into lists. Only the person who have scraped that data can download the data.
There are already companies that do exactly that kind of service. Just search for "mozenda", "scrapinghub" or "connotatate" or in general "web scraping service".

My problem is that I talked to that german lawyer and he told me that there will be a risk that people sue me because of downloading their websites. "competition regulations" and "copy right".

This is why I am looking for ways to make it more "secure".

Maybe I am too paranoid!?
 
Back
Top