Web Scraping member access website (tool?)

Snoopp

Newbie

Feb 19, 2012

31

2

• Jun 5, 2019

• #1

I have a member access website with valueable information where only people who have subscribed are allowed to enter. Is it possible to check somehow if some of them used scraping tool like Web Scraper for Chrome on my website to copy content from my page?

 

1

1guyhack

Junior Member

Jan 17, 2017

155

38

• Jun 5, 2019

• #2

You can't check if they are using web scrapper. Most of the professional scrappers don't use chorme scrapper, they are using professional applications which can't be traced by you.

 

Snoopp

Newbie

Feb 19, 2012

31

2

• Jun 5, 2019
• Thread Starter Thread Starter

• #3

I'm sure they are not professional scrappers. But I want to know if there are any leaks.

Am I able to detect someone using "Web Scraper" for Chrome or similar plugin?

 

thetrustedzone

Elite Member

Jun 15, 2010

3,994

3,463

• Jun 5, 2019

• #4

Snoopp said:

I'm sure they are not professional scrappers. But I want to know if there are any leaks.

Am I able to detect someone using "Web Scraper" for Chrome or similar plugin?

Click to expand...

you will be able to detect using user tracing mechanism , .e.g. blah username visits x pages in too short time y seconds ... or any similar concept could be applied , you could protect from basic and uniform scrapers using more browsing conditions like java script enabled , browsing time with captcha implementation for over time exceeds ....etc

 

Snoopp

Newbie

Feb 19, 2012

31

2

• Jun 5, 2019
• Thread Starter Thread Starter

• #5

the problem is that most of the information is available on a single page so user may see it for couple seconds or even minutes and it won't look suspicious...

Those protections you mentioned "JS enabled requirement" and "captcha over time" probably won't be enough in this case...

The question is if there are any analytical / spy tools that will allow me to see what plugins people use against my website?

 

dreadpixel

Banned - Selling outside marketplace

Jul 27, 2010

2,502

1,179

• Jun 5, 2019

• #6

You can track the user session visits and retention on each page.
Just add some logic to your coding to limit that possibility and make them more human.

 

Gogol

Administrator

Staff member

Moderator

Executive VIP

Jr. VIP

Sep 10, 2010

11,838

26,690

• Jun 5, 2019

• #7

Gogol said:

This calls for a custom website scraper. Try posting in the hire a freelancer section, for making a chrome driver based bot (the scraper will login, and scrape content as needed). I am sure you will find someone for cheap.

Click to expand...

Lol I look dumb. Sorry I totally misread op's question.

To answer the original question, nopes you can't do that easily. Luckily, there are services that will help you to make it as tough as you can. Recaptcha v3 is an example, that you can use to show the bots (or flag them, if detection is all you need); while not disturbing the real users.