[Journey] 1 million UVs/month in 12 months using AI generated content. Let's do it!

[Alexion]

How do solve the below issue? i can only solve one recaptcha from one proxy. after the first try i get this message everytime the bot clicks the play button.
i tried both residential and 4g proxy.
View attachment 204062

Most probably, the issue sides with your proxies. They are probably used to death.

I have had no problem with 2 requests/second for a few months, running 24/7 on a cheap google cloud vm.

If you are going past first page, set delays between pages.

 

[Niffo]

Why use a proxy, when u can get a real 4 or 5g connection for cheaper compared to most providers. Just get urself a dongle or use a old mobile phone and connect it to ur PC.

 

[prvncqr]

How do solve the below issue? i can only solve one recaptcha from one proxy. after the first try i get this message everytime the bot clicks the play button.
i tried both residential and 4g proxy.
View attachment 204062

You can use your own script to scrape Google from your real residential IP. I'm scraping more than 1K queries daily without any issues. Around 100K results per day.

 

[Sartre]

You can use your own script to scrape Google from your real residential IP. I'm scraping more than 1K queries daily without any issues. Around 100K results per day.

I've scraped 700k autocompletes in 24hrs on my home IP using 2 computers

 

[prvncqr]

I've scraped 700k autocompletes in 24hrs on my home IP using 2 computers

Yes, we can. It depends on how we optimize the code. As long as the query looks natural, we don't face any issues.

 

[Sartre]

Yes, we can. It depends on how we optimize the code. As long as the query looks natural, we don't face any issues.

15 threads, 1 IP, firefox toolbar trick; very natural I guess I just have a good IP at home.

 

[prvncqr]

15 threads, 1 IP, firefox toolbar trick; very natural I guess I just have a good IP at home.

Ohh. I don't know about it. Would you mind sharing more information about that tool? It shall be helpful for other users as well.

 

[Alexion]

Ohh. I don't know about it. Would you mind sharing more information about that tool? It shall be helpful for other users as well.

If you want to do it programmatically, you can code a small script around this URL

http://suggestqueries.google.com/complete/search?client=firefox&q=unicorn+dildos

 

[prvncqr]

If you want to do it programmatically, you can code a small script around this URL

http://suggestqueries.google.com/complete/search?client=firefox&q=unicorn+dildos

Ok. Are there any limitations or just work like regular search?

 

[Alexion]

Ok. Are there any limitations or just work like regular search?

There's no apparent limit. I've tested it with 20 threads and no delays/problems. Clean ip

 

[prvncqr]

There's no apparent limit. I've tested it with 20 threads and no delays/problems. Clean ip

Ok thanks

 

[prvncqr]

What GPU are you using and how many articles are paraphrased daily approximately? I have a basic model GPU and it paraphrases at most 500 articles a day if the article length is 1k-1.5k words. What GPU or GPU server will you recommend to me?

 

[prvncqr]

why do you need to solve proxies? I've spent like $200 on SERP APIs in 2 months and have a shit ton of content up. I think I've got around 6 million serp entries in my database right now.

If you were ready to spend $200 for API, you could have bought 100% private proxies from a legit site and scraped millions of sites in a month easily by just doing a small correction in your script.

 

[prvncqr]

There's no apparent limit. I've tested it with 20 threads and no delays/problems. Clean ip

I just created a script to scrape the suggested queries. Google detected it as automated and blocked my access after scraping 3k queries. I used only one thread and the IP is clean.

 

[rampzeus]

I just created a script to scrape the suggested queries. Google detected it as automated and blocked my access after scraping 3k queries. I used only one thread and the IP is clean.

Just buy a bunch of proxies and have Dome delays

 

[neOtron]

If you want to do it programmatically, you can code a small script around this URL

http://suggestqueries.google.com/complete/search?client=firefox&q=unicorn+dildos

Thanks .dint know about this. Will create a script for this now. Is there any trick for scraping questions too ?
I guess my home ip is bad I get recaptcha every every 20 or so queries

 

[Alexion]

Thanks .dint know about this. Will create a script for this now. Is there any trick for scraping questions too ?
I guess my home ip is bad I get recaptcha every every 20 or so queries

For questions, there's a library for python called PAA something, do a quick google, you will find it.

However, it's easier to just do it yourself by getting the xPath from the PAA snippet. Just scroll into view first, otherwise, it won't load the question.
This is to get the answer as well. If you don't need that, just use that library, it works wonders.

L.E. Here it is https://pypi.org/project/people-also-ask/

 

[neOtron]

For questions, there's a library for python called PAA something, do a quick google, you will find it.

However, it's easier to just do it yourself by getting the xPath from the PAA snippet. Just scroll into view first, otherwise, it won't load the question.
This is to get the answer as well. If you don't need that, just use that library, it works wonders.

L.E. Here it is https://pypi.org/project/people-also-ask/

Ya I am already using it. Thanks

 

[GVald]

For questions, there's a library for python called PAA something, do a quick google, you will find it.

However, it's easier to just do it yourself by getting the xPath from the PAA snippet. Just scroll into view first, otherwise, it won't load the question.
This is to get the answer as well. If you don't need that, just use that library, it works wonders.

L.E. Here it is https://pypi.org/project/people-also-ask/

The big problem with that library, is that it's sending one request per question, so we're burning our IP/Proxy waaaay faster, where you could fetch all the questions you need with only one request, since everything is on the same page

 

[Alexion]

The big problem with that library, is that it's sending one request per question, so we're burning our IP/Proxy waaaay faster, where you could fetch all the questions you need with only one request, since everything is on the same page

Well, yes, but it's a great starting point for many.

Personally, I use Selenium without any proxies. One request can fetch even 300 questions.