Cookie Problem

tiagocardoso said:

So, I'm using a Google Chrome extension to create an auto data scrapper to extract jobs ads, but I'm facing some issues.

The website I'm currently extracting has a cookie or something (I don't even know) that shuts down accounts after X ads opened or X clicks (I still haven't deciphered it well).

My question is, if it's a cookie, for example, is it possible to block it? I'm a fuck*n newbie in these things.

If anyone can help me, I would appreciate it.

I know I am giving very vague data, but I don't know how to explain it.

Click to expand...

abutor said:

If you have access to the source code of the extension you should be able to modify it to delete cookies. This extension does it: https://github.com/SingleClickApps/disable-cookies/blob/master/core.js#L166

Click to expand...

DogNamedRottles said:

What about rate limits on the server that's serving the job ads? I would think after so many requests from a session or an ip that it would refuse requests. But then again, I don't have much information to go off of from your post.

Click to expand...

tiagocardoso said:

Might be the case, you know any way to rotate proxies on chrome?

Click to expand...

DogNamedRottles said:

I'm going to take an educated guess and say that you would need to switch out the proxies in a programmable way. For instances, using something like selenium or puppeteer and when connections fail due to rate limits, the headless browser will update the manifest to use a different proxy.

Again educated guess and I'm supporting it with a small bite of information here: https://botproxy.net/example/selenium-chromedriver-rotating-proxies/

You'll have to figure out when to switch out the proxies though if you have the programming knowledge

Click to expand...

tiagocardoso said:

Yep... I don't have programming knowledge, but I will check that guide. Thanks!
I guess I will need to pay someone to make a bot for me, I guess it's going to be the best way to go.

Click to expand...