Cookie Problem

[tiagocardoso]

So, I'm using a Google Chrome extension to create an auto data scrapper to extract jobs ads, but I'm facing some issues.

The website I'm currently extracting has a cookie or something (I don't even know) that shuts down accounts after X ads opened or X clicks (I still haven't deciphered it well).

My question is, if it's a cookie, for example, is it possible to block it? I'm a fuck*n newbie in these things.

If anyone can help me, I would appreciate it.

I know I am giving very vague data, but I don't know how to explain it.

 

[Deleted member 258825]

If you have access to the source code of the extension you should be able to modify it to delete cookies. This extension does it: https://github.com/SingleClickApps/disable-cookies/blob/master/core.js#L166

 

[DogNamedRottles]

So, I'm using a Google Chrome extension to create an auto data scrapper to extract jobs ads, but I'm facing some issues.

The website I'm currently extracting has a cookie or something (I don't even know) that shuts down accounts after X ads opened or X clicks (I still haven't deciphered it well).

My question is, if it's a cookie, for example, is it possible to block it? I'm a fuck*n newbie in these things.

If anyone can help me, I would appreciate it.

I know I am giving very vague data, but I don't know how to explain it.

What about rate limits on the server that's serving the job ads? I would think after so many requests from a session or an ip that it would refuse requests. But then again, I don't have much information to go off of from your post.

 

[tiagocardoso]

If you have access to the source code of the extension you should be able to modify it to delete cookies. This extension does it: https://github.com/SingleClickApps/disable-cookies/blob/master/core.js#L166

Thanks for the help, but I don't even know how to use that lmao

What about rate limits on the server that's serving the job ads? I would think after so many requests from a session or an ip that it would refuse requests. But then again, I don't have much information to go off of from your post.

Might be the case, you know any way to rotate proxies on chrome?

 

[DogNamedRottles]

Might be the case, you know any way to rotate proxies on chrome?

I'm going to take an educated guess and say that you would need to switch out the proxies in a programmable way. For instances, using something like selenium or puppeteer and when connections fail due to rate limits, the headless browser will update the manifest to use a different proxy.

Again educated guess and I'm supporting it with a small bite of information here: https://botproxy.net/example/selenium-chromedriver-rotating-proxies/

You'll have to figure out when to switch out the proxies though if you have the programming knowledge

 

[tiagocardoso]

I'm going to take an educated guess and say that you would need to switch out the proxies in a programmable way. For instances, using something like selenium or puppeteer and when connections fail due to rate limits, the headless browser will update the manifest to use a different proxy.

Again educated guess and I'm supporting it with a small bite of information here: https://botproxy.net/example/selenium-chromedriver-rotating-proxies/

You'll have to figure out when to switch out the proxies though if you have the programming knowledge

Yep... I don't have programming knowledge, but I will check that guide. Thanks!
I guess I will need to pay someone to make a bot for me, I guess it's going to be the best way to go.

 

[DogNamedRottles]

Yep... I don't have programming knowledge, but I will check that guide. Thanks!
I guess I will need to pay someone to make a bot for me, I guess it's going to be the best way to go.

If you have trouble finding someone to build the bot, I'll be happy to help out.