So I'm playing around with this tool for the first time and I scraping for obvious keywords but it doesn't return many results. Maybe 20 or so then do nothing. If I hit stop and start and scrap again it will return 20 or so more reults then nothing. Is there some setting or something I need to change?
GSA Website Contact
- Thread starter hammer35
- Start date
with about 10 private and a bunch of public ones
So after going back and studying the logs, it look like its doing the search but something seem off. It seem to be returning the same results for every single page. I mean all the way up to page 65.
Including for big sites that you would think wouldn't be anywhere near page 65 in the search results. I will try to do a manual search and see what I see.
Ok looking at the logs it saying search/import delivered a known website which are just the websites on my global blacklist. However doing a manual search there are other sites that aren't getting added to the website list
Including for big sites that you would think wouldn't be anywhere near page 65 in the search results. I will try to do a manual search and see what I see.
Ok looking at the logs it saying search/import delivered a known website which are just the websites on my global blacklist. However doing a manual search there are other sites that aren't getting added to the website list
It's weird. Maybe some sites are getting skipped and not added to the list because they don't have contact forms? That would make sense except there plenty of sites on the list that don't have contact forms.
There go that theory. Manually I just found another site on the first three pages of Google that has a contact form that wasn't picked up by the tool.
There go that theory. Manually I just found another site on the first three pages of Google that has a contact form that wasn't picked up by the tool.
Last edited:
- Sep 15, 2012
- 5,705
- 2,774
How much delay you have set in per search request? If you are going too fast, then maybe your proxies got banned by Google.
And can you confirm if your proxies are passed for search engine scraping? Because most of the proxy providers offer solutions just for search engine scraping purpose. And also residential rotating proxies are used for this. Not sure which one you are using. If you are using static ones, then the results will be very slow. Because you have to set almost a minute delay per search query.
Last edited:
On a side note how does it determine what country a site is in? Is it just looking at the site ip? All does says is the host is in another country correct?
That doesn't really mean much. Or at least not what one might think on first thought. I could have a site hosted on a server in China but in fact the company could be in the USA.
Or does it check the DNS registration information? Which would make more sense. Of course many sites have DNS privacy turned on.
That doesn't really mean much. Or at least not what one might think on first thought. I could have a site hosted on a server in China but in fact the company could be in the USA.
Or does it check the DNS registration information? Which would make more sense. Of course many sites have DNS privacy turned on.
Last edited:
ciccrypto
BANNED
- Dec 12, 2022
- 64
- 22
For further results on the GSA website, get in touch with:
Expand the scope of the search.
Modify the search area.
Turn on proxies.
Change the connection's settings.
These adjustments can aid in obtaining more information and guard against scraping that is restricted or halted.
Expand the scope of the search.
Modify the search area.
Turn on proxies.
Change the connection's settings.
These adjustments can aid in obtaining more information and guard against scraping that is restricted or halted.
phil stewart
Regular Member
- Feb 20, 2019
- 327
- 88
Sven has made tons of improvements to the software since this thread, mainly correcting issues with the search engine functionality. It's better for sure but for speed scrapebox is still best.