lazlopanaflex
Junior Member
- Oct 15, 2020
- 176
- 52
it is failing. how can I save this raw HTML file you are asking for?
Scrapebox not scraping Bing
- Thread starter lazlopanaflex
- Start date
lazlopanaflex
Junior Member
- Oct 15, 2020
- 176
- 52
GoogleAlchemist
Regular Member
- Nov 25, 2009
- 320
- 44
Same issues for me. I tried downloading default engines again so they are the latest...and i also tried saving a new updated bing using the useragent mentioned. Still no go for bing or yahoo. Using private proxies, google works just fine.
- Jan 25, 2009
- 6,378
- 3,774
Go to settings >> harvester engine configuration >> bing
then click test engine. Does it pass? if it fails it will ask you if you want to save raw html.
Open it and have a look, is there urls there? or some tos splash page or ?
GoogleAlchemist
Regular Member
- Nov 25, 2009
- 320
- 44
When I test bing it gives me this pop up box:
Error -1
No links could be retrieved!
-1 Aborted
----
It does the same for the default (freshly downloaded from server so its current) and for the version I saved with the useragent u suggested.
I see no option to save a raw html file.Maybe because there are no urls because it didnt harvest anything? When I watch it in detailed harvester, it just quickly cycles thru all of my proxies (which are not blocked and are fast and working fine in google as we speak) and never harvests anything.
My footprints are good as Im harvesting them fine in google and I manually searched for a few of them in bing and they bring back plenty of results there.
Error -1
No links could be retrieved!
-1 Aborted
----
It does the same for the default (freshly downloaded from server so its current) and for the version I saved with the useragent u suggested.
I see no option to save a raw html file.Maybe because there are no urls because it didnt harvest anything? When I watch it in detailed harvester, it just quickly cycles thru all of my proxies (which are not blocked and are fast and working fine in google as we speak) and never harvests anything.
My footprints are good as Im harvesting them fine in google and I manually searched for a few of them in bing and they bring back plenty of results there.
GoogleAlchemist
Regular Member
- Nov 25, 2009
- 320
- 44
Same prob with no proxies enabled either and my home ip is fresh today and wouldnt be banned by bing
error log shows this
8/10/2021 9:28:05 AM: HTTP: -1 Aborted, URL: https://www.bing.com/search?q=test&sc=12-3&sp=-1&sk=&first=1&FORM=PERE&cnt=20 Proxy: :
8/10/2021 9:28:05 AM: HTTP: -1 Aborted, URL: https://www.bing.com/search?q=marketing&sc=12-3&sp=-1&sk=&first=1&FORM=PERE&cnt=20 Proxy: :
8/10/2021 9:28:05 AM: HTTP: -1 Aborted, URL: https://www.bing.com/search?q=gorilla&sc=12-3&sp=-1&sk=&first=1&FORM=PERE&cnt=20 Proxy: :
error log shows this
8/10/2021 9:28:05 AM: HTTP: -1 Aborted, URL: https://www.bing.com/search?q=test&sc=12-3&sp=-1&sk=&first=1&FORM=PERE&cnt=20 Proxy: :
8/10/2021 9:28:05 AM: HTTP: -1 Aborted, URL: https://www.bing.com/search?q=marketing&sc=12-3&sp=-1&sk=&first=1&FORM=PERE&cnt=20 Proxy: :
8/10/2021 9:28:05 AM: HTTP: -1 Aborted, URL: https://www.bing.com/search?q=gorilla&sc=12-3&sp=-1&sk=&first=1&FORM=PERE&cnt=20 Proxy: :
- Aug 9, 2017
- 5,594
- 6,886
This is what I updated to make it work for me:
1) changed Query String to:
2) changed Marker for "Next Page" to:
Remember to back stuff up before making changes.
1) changed Query String to:
Code:
https://www.bing.com/search?q={KEYWORD}&form=QBLH&sp=-1&pq={KEYWORD}&sc=1-18&qs=n&sk=&cvid=&first={PAGENUM}&cnt=20
Code:
">Next</a>|">Next</div></a>|class="sw_next">|title="Next page"Remember to back stuff up before making changes.
GoogleAlchemist
Regular Member
- Nov 25, 2009
- 320
- 44
Thanks, that made it test ok in settings but still same problem with or without proxies when trying to actively harvest.
update/edit: dunno what i did wrong but i cant replicate even the successful test now let alone any harvesting...i updated things, not sure what happened
update/edit: dunno what i did wrong but i cant replicate even the successful test now let alone any harvesting...i updated things, not sure what happened
- Jan 25, 2009
- 6,378
- 3,774
Start over. Download an entirely fresh copy of scrapebox to a new folder.
http://www.scrapebox.com/payment-received
Then update scrapebox to the latest version if not.
Try it.
If bing files go to settings >> harvester engine config > import >> download default engines.
then try it.
then you can try updating the user agent and making sure you have an ip from a country bing likes, like north america for example.
GoogleAlchemist
Regular Member
- Nov 25, 2009
- 320
- 44
tried a fresh install and all those steps, same issue
gonna send the raw html file thing to sbox support as i have no idea what im looking for
gonna send the raw html file thing to sbox support as i have no idea what im looking for
- Jan 25, 2009
- 6,378
- 3,774
Sounds like a good plan.
Also you could try some proxies from a different country and see if they work.
Pam Danny
Elite Member
- Feb 9, 2019
- 1,979
- 356
Having same issues.
But google scrapes perfectly, but as we know, Google needs good proxies, otherwise, the few you have will burn out.
I am getting proxies from GSA Proxy Scrapper. I always have about 900 auto saved and (updating every 30 minutes) used by Scrapebox
Tried everything I have seen here plus on your 2 videos, yet bing can't work. Sometimes, it will scrape 10 from one thread, then another 20 will be 0 and like that.
But google scrapes perfectly, but as we know, Google needs good proxies, otherwise, the few you have will burn out.
I am getting proxies from GSA Proxy Scrapper. I always have about 900 auto saved and (updating every 30 minutes) used by Scrapebox
- Jan 25, 2009
- 6,378
- 3,774
Try adding a character and deleting it, it may just be that it doesn't know you pasted something. Also try pasting it into notepad and then copy from notepad and paste into scrapebox, because that will strip any hidden data which could be causing an issue.
Else if you can give a screenshot that might give a clue.
Lastly its worth checking that you don't have a space before the user agent.
Pam Danny
Elite Member
- Feb 9, 2019
- 1,979
- 356
I was able to create another bing2 engine by just renaming.
Also I pasted the code on notepad before using.
All failed.
Just about 30 minutes ago, I decided to try again. This time it harvests 1 line and gives 10 line 0 harvesting.
I can give you more details as you need
Also I pasted the code on notepad before using.
All failed.
Just about 30 minutes ago, I decided to try again. This time it harvests 1 line and gives 10 line 0 harvesting.
I can give you more details as you need
- Aug 9, 2017
- 5,594
- 6,886
Here's the current engine I use for bing:
You can save that as bing.dat then import it via the harvester engine configuration. If that doesnt work then I'd look at proxies being an issue - I've seen bing start throwing out some ip bans where they just refuse to show any results, but still nowhere near as often as Google.
Code:
[Engine1]
Displayname=New Bing
QueryString=https://www.bing.com/search?q={KEYWORD}&sp=-1&pq={KEYWORD}&sc=1-49&qs=n&sk=&first={PAGENUM}&FORM=PORE
MustBeInLink=http
MustNotBeInLink=microsoft.com|overture.com|msn.com|bing.com|microsofttranslator.com|q=site:http
JustBeforeLink=href="
RightAfterLink="
PageStart=1
PageInc=10
NextPageMarker=">Next</a>|">Next</div></a>|class="sw_next">|title="Next page"
Translation=%3f=?|%3d==|%26=&|%3a=:|&=&|%2F=/
headerData=
Referer=0
SSLVersion=0
Favicon=/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAMCAgMCAgMDAwMEAwMEBQgFBQQEBQoHBwYIDAoMDAsKCwsNDhIQDQ4RDgsLEBYQERMUFRUVDA8XGBYUGBIUFRT/2wBDAQMEBAUEBQkFBQkUDQsNFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBT/wAARCAAQABADASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwD7E1T4Z2sfhnQdVi1G306K6thJcy38xwXIBARVUk9+KwNQ8CXdvpUuqWN3aaxp8JAmlsnJaL/fRgGH5Vqt4j0fxX4X0vStYuLjS7zS1ZILqKLzYnQ44ZQQQcKBken4Utt4g0Twf4f1ay0m5n1e/wBThNvJcSQ+TDFGQQcKTktyf89f5Tr0cpry9pFRjScE+ZT95TUFp7PfWWj93XfmSP26lUx1JcrblPmejj7vLzb83+HXfy5T/9k=
GracePeriod=0
UserAgent=Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:88.0) Gecko/20100101 Firefox/88.0
Selected=1
DetailedSelected=0
FollowRelocation=1
AddRelative=
AddFieldValue=
ClearCookies=0
ListMode=0
ReadOnly=0You can save that as bing.dat then import it via the harvester engine configuration. If that doesnt work then I'd look at proxies being an issue - I've seen bing start throwing out some ip bans where they just refuse to show any results, but still nowhere near as often as Google.
Last edited:
Pam Danny
Elite Member
- Feb 9, 2019
- 1,979
- 356
I will try that in a bit surely. Just that other engines also give similar error. So I think it's not a bing thing
- Jan 25, 2009
- 6,378
- 3,774
Sweet, thanks for the share!