Is anyone familiar with Paigham Bot?

[eNzim]

So I'm trying to scrape google for a keyword by using the advanced search operators in this format:

keyword site:website.com -unwantedword

What I did was that I uploaded this part: "site:website.com -unwantedword" as a footprint and insterted the keyword in the keyword section of the tool.

It was working fine but the tool only found about 200 URLs (no duplicates), while there are waaay more than that. For example Google shows 800k search results and when I check on website.com I can find a lot more than 200. But the tool says no more found and scraping is complete.
I'm using private proxies and death by captcha as well.

What am I doing wrong? Is there a setting I forgot to tweak or just the scraping method I'm using is bad?

I would appreciate some help from the pros, thank you in advance!

 

[miyuru2u]

I am using paigham bot full time now
Dont use paighambot for scraping with google
just use scrapebox and search get the urls
then scan urls with paighambot for contact pages
then blast away

 

[eNzim]

I am using paigham bot full time now
Dont use paighambot for scraping with google
just use scrapebox and search get the urls
then scan urls with paighambot for contact pages
then blast away

So what you are saying is that what I'm trying to do can't be done with Paigham bot?

Any other opinions?

 

[miyuru2u]

So what you are saying is that what I'm trying to do can't be done with Paigham bot?

Any other opinions?

Just do it with scrapebox to get more results

 

[DunDidIt2X]

So I'm trying to scrape google for a keyword by using the advanced search operators in this format:

keyword site:website.com -unwantedword

What I did was that I uploaded this part: "site:website.com -unwantedword" as a footprint and insterted the keyword in the keyword section of the tool.

It was working fine but the tool only found about 200 URLs (no duplicates), while there are waaay more than that. For example Google shows 800k search results and when I check on website.com I can find a lot more than 200. But the tool says no more found and scraping is complete.
I'm using private proxies and death by captcha as well.

What am I doing wrong? Is there a setting I forgot to tweak or just the scraping method I'm using is bad?

I would appreciate some help from the pros, thank you in advance!

You are probably getting blocked by google. Perhaps increase your throttle/delay or also possible you forgot to check the drop down menu to use custom footprints and your scraping with the default footprint . Be sure to follow proper sequence for the settings you are using.

And even though google says 80k results they really don't show this amount at all and in fact show substantially less than advertised. You can see how many results actually exist in Google by searching in your browser, change your result settings to show 100 results per page, then check and see how far it actually goes .I would wager that you'll find it doesn't go back as far as you are thinking it does.

At the end of the day, whether you use paigham bot, scrapebox, gsa or any other scraper its going to scrape the same results. Getting limited results will mean only one of the following:

Recaptcha is triggering and it's not being solved.
Your ip is being blocked, which can be caused for several reasons including accessing webpages to quickly or specific footprint requests ECT.

 

[SEO-Greg]

So what you are saying is that what I'm trying to do can't be done with Paigham bot?

Any other opinions?

use "beautiful soup' to scrape google ..pyp kick ass lol