My new PAA Journey

Nice one and Goodluck on your journey. Do you plan on selling your scrapper in the near future?
 
Advertise on BHW
Good Luck on this one. Just be watchful of the upcoming Google Update (that possibly is targetting PAAs)

I hope you are unaffected
 
SharonHe said:
Right now I have 17K posts in the WordPress database.
Every post is separate question with answer.
I generate tags for every post based on the title.
Then I use related posts plugin which I have modified to include the related posts (based on tags, title and main body) in the main body of the primary post as title H3 and answer. So the whole post now looks like it has 20 subheadings with content but actually those 20 subheadings are real posts in the wordpress and you can open every single one of them and get different related posts for that post too.
Also, my PHP script, before adding new post to the database, compares the content of all posts and checks if there is already post with the same content. If there is, then I do not add that as new post, I only take the question and append it to the post as meta data.
Then in my template I use function to extract the meatdata and show it in the top of the post as "Similar questions"

Then I use Table of content plugin in the top of the primary post to show all the subheadings and link them for quick access.

PS: Google adsense is still reviewing my page.
PSS: I use Google keyword planner to generate keywords, that I later feed to my Python scraper
Click to expand...
Thank you so much for your very clear answer. Hope the best with your journey.
 
TheDankChocolate said:
Good Luck on this one. Just be watchful of the upcoming Google Update (that possibly is targetting PAAs)

I hope you are unaffected
Click to expand...
To be honest I am really unlucky so I am sure I will get hit :D

thegcfrolamide said:
Nice one and Goodluck on your journey. Do you plan on selling your scrapper in the near future?
Click to expand...
It is more of a "system". More scripts and components working together to generate PAA sites. Right now I am also working on implementing GP2 AI generator in the "system". So it's not simply like complete script with interface. It's much more "coder" orienteadted. But why not, if the system proves it works I can sell it like that, with few improvements for easier working and tutorial.
 
SharonHe said:
To be honest I am really unlucky so I am sure I will get hit :D


It is more of a "system". More scripts and components working together to generate PAA sites. Right now I am also working on implementing GP2 AI generator in the "system". So it's not simply like complete script with interface. It's much more "coder" orienteadted. But why not, if the system proves it works I can sell it like that, with few improvements for easier working and tutorial.
Click to expand...
Congratulations once again on your journey and the very best while working on it
 
Current number of posts in wordpress 20.140

Indexed by google 80

Yesterday visitors 0

I installed YOAST SEO pro plugin, and google search console immediately found 20K urls in the sitemaps.
 
itz_styx said:
yea but with rotating proxies its no issue and you get much more data faster as you dont need delays like when you scrape google from 1 IP, but was just a suggestion.
if your setup works for you, then thats fine too. there are always multiple ways to do things :)
Click to expand...
how do you rotate proxies?
 
itz_styx said:
use a provider that has rotating proxies, so every request is sent through a different IP.
Click to expand...
Never thought of this. I am always relaying on my script. I use text file with ip\:port:user\:pass and parse this file so I can change it on every few requests. But your idea is much more elegant.
 
i used to do this too, but rotating proxies are so much better for scraping data and pricing is similar to normal proxies.
 
What kind of AI model do you use to answer them?
Good luck!
 
HustleTong said:
What kind of AI model do you use to answer them?
Good luck!
Click to expand...
Nothing, I just take the answer from google PAA. Now I am working to generate bigger content using neo AI, but that is still work in progress. I am limited with GPU power.
 
itz_styx said:
use a provider that has rotating proxies, so every request is sent through a different IP.
Click to expand...
Even if you use rotating proxies with new IP for every request, Google will still rate limit you. You need to fake your user agent and have some delay.
 
sagarpatil said:
Even if you use rotating proxies with new IP for every request, Google will still rate limit you. You need to fake your user agent and have some delay.
Click to expand...
no they wont if you get a new IP for every request and that IP hasnt been used for scraping. delays would be pointless.
faking the UA is obvious, but has not much effect on how much you can scrape. you might get a little more from the same IP, but its the number of requests they count until they show a captcha.
with rotating proxies you dont have the captcha problem and you dont need any delays so you can scrape tens of thousands in a short amount of time.
been there done that with many types of different data scraping from google.
 
itz_styx said:
no they wont if you get a new IP for every request and that IP hasnt been used for scraping. delays would be pointless.
faking the UA is obvious, but has not much effect on how much you can scrape. you might get a little more from the same IP, but its the number of requests they count until they show a captcha.
with rotating proxies you dont have the captcha problem and you dont need any delays so you can scrape tens of thousands in a short amount of time.
been there done that with many types of different data scraping from google.
Click to expand...

İ am using requests with rotating proxy with just 5 keywords and 10 multithread still getting captchas :D also using fake useragent.

its also giving errors i found some coders but they couldnt fix the problem. sometimes it gives pandas error sometimes other. i got really sick of it. i dont know progrramming.

i found the code on github. if you can help can you pm me ? i can pay you.
 
itz_styx said:
no they wont if you get a new IP for every request and that IP hasnt been used for scraping. delays would be pointless.
faking the UA is obvious, but has not much effect on how much you can scrape. you might get a little more from the same IP, but its the number of requests they count until they show a captcha.
with rotating proxies you dont have the captcha problem and you dont need any delays so you can scrape tens of thousands in a short amount of time.
been there done that with many types of different data scraping from google.
Click to expand...
Do you have any recommendations for proxies? I tried HydraProxy with new IP for every request but was still getting 429, it only worked if I put a delay of 7-10 seconds.
 
benman said:
Why do you store the q&a in database?
Click to expand...
Because I then use PHP script that connects with the wordpress framework to parse the data and post it as wordpress post together with tags and category.

also it is easier for my python scraper to skip the repeated data.
 
Advertise on BHW
You must log in or register to reply here.
Sign up now!

Main Menu

Home Main Forum List Black Hat SEO White Hat SEO BHW Newbie Guide Blogging Black Hat Tools Social Media Downloads

Marketplace

Account Selling Content / Copywriting EBooks / Methods Hosting Misc Proxies For Sale SEO - Link building SEO - Packages Social Media Social Media - Panels Web Design More

Making Money

Affiliate Programs Hire a Freelancer IM Journeys Making Money Pay Per Click Site Flipping Want to Buy

BlackHat World

Dispute Resolution Forum Suggestions & Feedback Introductions Lounge
Back
Top