Trying to build a custom scraper for images from user twitter accounts, but I can't really wrap my head around it despite having reviewed the custom scraper tool tutorial excessively. It seems it's still over my head, although I have had success on some other sites.
Any thoughts?
I also want to mention that SB if single-handedly the best investment I ever made in online endeavors.
Scrapebox already has an image scraper built in that can scrape from twitter, and the custom data grabber is only going to save off the image url, not the image.
here is a vid on the image scraper
https://www.youtube.com/watch?v=zicEH7ohseU
having the same issue with page authority add on like others (again)
contacting moz and this is their reply
On Tue, May 3, 2016 at 11:03 PM, Ian from Moz wrote:[TABLE="width: 771, align: center"]
[TR]
[TD]This sounds like it is an issue with your 3rd party tool. We have a lot of people write in with issues relating to ScrapeBox. Since SB uses proxies and allows for multiple API keys and rotates them (against our Terms of Use), it is very prone to being blocked by our backend service. If you are having troubles I recommend reaching out to them and having them contact us to get their tool in line with our TOUs. [/TD]
[/TR]
[TR]
[TD]
[/TD]
[/TR]
[/TABLE]
Having exactly the same issue here with Page Authority addon , setup new Moz account etc. and still not working??
@Sweetfunny
So I opened a support ticket with Moz about getting my account and IP whitelisted (because of above explained issue), and today Ian from Moz got back to me, saying:
So for everyone here the issue is definitely moz and not scrapebox. The reason is you can simulate the exact same request, byte for byte in a browser, in programs like netpeak, in scrapebox etc... and all of them get randomly blocked. Ive also talked with other people that are using other tools and having the same issue.
Further MOZ has no way of knowing if a request is from scrapebox or a browser, there is no footprint, so they can't block scrapebox specifically. Scrapebox does offer proxies as noted, but other tools that don't that are 100% inline with moz TOU are also being blocked, moz is just trying to weed out free traffic most likely.
Also some users have hit me up and I told them to contact moz and they say they talked with someone at moz that whitelisted their IP and then all was well, while other users get a response from moz that moz doesn't whitelist ips. There seems to be a "grab an excuse from the bucket" game going at moz support where they pick and excuse and hand it to you hoping you will go away. Thats my personal opinion anyway.
I would just keep at them and let them know many tools aren't working, tools in line with their TOU etc... you don't even need to tell them its scrapebox. Hopefully they will dial back the firewall like before.
At the end of the day moz is running a business and while they are giving away free access to their stuff, Im sure if they did not give away free access but could make just as much money, they they would happily do that. I can't fault them for wanting to minimize free traffic and setting low limits. However they are indeed blocking all sorts of legitimate requests and its nothing to do with their TOU its to do with saving cash flow in resources. Again my opinion.
Lately (as in, the past few days) I've been getting nothing but 503 errors when trying to scrape from Google. Anyone happen to know what would cause this and what I can do to fix it? I thought it might be something to do with the proxies, but here are a few things:
- All of my proxies (private ones of course) test out just fine in the proxy tester
- I can use any of my proxies within a browser to do a Google search no problem (no captchas pop up or anything)
- Scraping with an old version of SB (1.16.6) works just fine with my proxies as well
I'm not sure what to make of it.
Update: When I try to scrape without proxies I get a 200 (it connects) but then the connection times out.
Update #2: Doing a Google search in a browser with my proxies IS giving me a captcha now. So I guess that's the problem. Also I can scrape without proxies no problem when I tether my phone, so the connection timing out must be a network adapter/router issue.
Update #3: I reset my router to factory settings and now I can scrape without proxies, so that's good. Proxies are still cucked though. Am I curious now though - is 503 the error everyone gets whenever they're getting captchas for Google searches? I've done a lot of searching on this subject but no one else seems to have made that connection.
When you reset your router your ISP probably assigned you a new IP, which is why it works (Im guessing) but 503 from google and 302 is proxies blocked. If you wait 24-48 hours your proxies will probably be unblocked, if not then you probably need new proxies.
I have a helpful video on this here:
https://www.youtube.com/watch?v=GadX5AXiW34
I have tested craigslist links as well but still the speed is a problem and sometimes it does not scrape emails from craigslist pages at all.
Craigslist isn't cut and dry you need to generate the email pages. I have a video
https://www.youtube.com/watch?v=3gjewPDf4rM
I'm getting this issue with vanity name checker where it adds the web 2.0 domain name onto the end of the urls I import. Not sure what I'm doing wrong or if its the software? I can't remember it doing this last time I used the feature which was a few months ago now.
View attachment 79618
Those aren't urls you entered. a url needs a protocol such as http:// in front. So it needs to be
http://domain.com not domain.com
When you enter this in a browser it assumes you want http and does that for you, but the vanity checker needs you to have that so it can auto distinguish between a keyword and a url.
