1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Recommendations for hosting a web scraping service without getting sued

Discussion in 'Business & Tax Advice' started by jimBlackHat, Jan 15, 2015.

  1. jimBlackHat

    jimBlackHat Newbie

    Joined:
    Jan 14, 2015
    Messages:
    3
    Likes Received:
    1
    Hi,

    right now I am developing a web scraping service where people can scrape pages like yellow pages, ebay, amazon and all kind of shops for data they need. Everything runs on my server.

    A bit more detailed: People register on my site an pay for the service. They now can scrape data from pages and download this data as excel, csv, xml, etc. I simply turn unscructured data into lists.

    I have talked to a lawyer and he said that it would be a potential risk to start such a service in germany (this is where I live) because of "competition regulations" and "copy right".

    Now I am looking for possibilities to avoid getting sued by shop owners that my potential customers will scrape.

    Some ideas:
    - What about setting up a limited (LTD) company?
    or
    - Use a web hoster thats not located in germany for registerting the domain name and running the server?

    There are already several websites that are providing such kind of scraping service. I wonder how they manage this problem.

    Any ideas are welcome!

    EDIT: Another solution. Always use open proxy server and fake the user agent. Good or bad idea?

    Thanks,
    jim
     
    Last edited: Jan 15, 2015
  2. jimBlackHat

    jimBlackHat Newbie

    Joined:
    Jan 14, 2015
    Messages:
    3
    Likes Received:
    1
    Wrong (sub) forum?
     
  3. ChrisX

    ChrisX Jr. VIP Jr. VIP

    Joined:
    Oct 8, 2011
    Messages:
    273
    Likes Received:
    139
    Gender:
    Male
    Home Page:
    There are many legit companies scraping content (search engines and sites like archive.org). If they can do it why can't you?
     
  4. TeKn1qu3z

    TeKn1qu3z Jr. VIP Jr. VIP Premium Member

    Joined:
    Jul 26, 2012
    Messages:
    904
    Likes Received:
    227
    Occupation:
    Amazon FBA Guru
    Location:
    The Office
    I don't think theres anything wrong with scrapping content. It's what you do with that content after it's scrapped
     
  5. wisdomkid

    wisdomkid Jr. VIP Jr. VIP

    Joined:
    Jun 20, 2011
    Messages:
    2,710
    Likes Received:
    790
    Scrapping contents or scrapping sales profile?

    I don't fully understand what you're talking about, but if it's about scrapping content, a link back, even no follow is acceptable.

    The truth is that, if you decide to scrap big companies like the ones you mentioned above, they can easily report your website to google and it will be deindexed faster than you can ever imagine
     
  6. jimBlackHat

    jimBlackHat Newbie

    Joined:
    Jan 14, 2015
    Messages:
    3
    Likes Received:
    1
    Sorry for the short description of my project. Here comes a bit more detailed description (edited first post too):

    People register on my website and have to pay for the service. They now can scrape data from pages and download this data as excel, csv, xml, etc. My service simply turns unscructured website-data into lists. Only the person who have scraped that data can download the data.
    There are already companies that do exactly that kind of service. Just search for "mozenda", "scrapinghub" or "connotatate" or in general "web scraping service".

    My problem is that I talked to that german lawyer and he told me that there will be a risk that people sue me because of downloading their websites. "competition regulations" and "copy right".

    This is why I am looking for ways to make it more "secure".

    Maybe I am too paranoid!?