scrapebox is so close to being a domain miner. can it scrape? yes can it extract external domains? yes can it check PA DA? I am not sure, but that's not hard to implement. can it check domain availability? at this point no, but that's not hard to implement I know of the 2 programs made for domain mining but I dont want to pay a monthly fee. I would pay like $20 for a SB add on. why dont they make it? this should be so easy for someone with programming knowledge.
I actually have people routinely ask me how to do this, and there are even videos on it. It does scrape It does extract external domains It does check PA/DA - Page Authority Addon - added maybe a year ago, or actually I think late 2013 can it domain availability - yes - they added that in early 2014 I believe. A lot of people actually use SB to get expired domains
Actually, you are right, it does have domain availability checker. But last time I used it, it sucked. a lot of false positives and false negatives.
As loop line said, it can do all of the things you mentioned. However, I think what you're missing is that scrapebox is a web scraper - the Expired Doman Miner and DomRipper are web crawlers. While scrapers are very simple, web crawlers are significantly more difficult to implement, especially with the memory constraints of your typical VPS. If you you ask anyone building PBNs at any scale, they'll tell you that crawling is absolutely necessary to stay competitive.
I think you have to manually remove the https:// sites from the URL structure before you check whether the domain is registered or not in Scrapebox and you'll also want to remove the weird TLDs that you can't register since they'll often pop up as available to register.
I don't think SB can check domain availability. I just checked all the addons and there is no such thing. for example if you were to check the following domains, how would you check them in SB? abc.com hty.org fhere.net
Scraping refers to processing a list of web pages to extract some data from In this case it would be individual domains for processing. Scrapers, like scrapebox, only process data from the Urls you input Crawling processes data from the web page, similar to a scraper, but then it crawls to the links on the page also, and then processes them. for example, if you wanted to process all the pages on bbc.com, a scraper would need you to have a list of all Urls on bbc.com. A crawled can simply take the input of the root domain, bbc.com and then will crawl all the pages for you. Does that make sense?
It would be great if they somehoe built a Add On to get expired domains from Scrapebox.. I know there are Videos and great help to get from them, however, it's very complicated I find. DomRipper is OK, but so far I don't had luck to find a single domain with it.. on 900 different domains... maybe the quality is to low.. will keep trieing for a while.
So first you say it has it and it sucks, and then you say it doesn't have it? I think you are not really doing much research before you make statements. Go to Grab/check >> check unregistered domains.
It does have some flaws. Copy and import this list into Scrapebox and then use the domain checker to see if they're available: https://apple.com https://google.com https://yahoo.com So long as it's https it'll always show up as available when you do a domain check. This is what is most likely causing the problems.
actually, the problem is not with https. I just tried a list of domains and 5 says they are available. I tried all 5 and none of them are actually available.
You need to start giving examples. So far, https domains are the only ones I know are bugged. If you post which ones gave you false positives they might be able to fix it.
The price of Domripper is the guarantee that it will fail in the long run. I can think of 3 software combo, one being scrapebox, that will smash any "pro" expired domain crawler. 350 $ a month, what a joke. Regarding scrapebox, I still think that is the best seo software out there. Well designed, affordable and very useful. If they will integrate a profi domain crawler or make it as a premium addon I will buy it.
In 1.x I believe it does not have the dlls needed and/or support https in the domain checker and actually in multiple functions. Your examples show as not available in 2.x, which supports https just fine in every function I have tried. I agree with myownhero. We aren't posting in this thread to say that your wrong or disagree with you, I don't care if you are right or wrong and I don't have time to go around trying to prove a point of someone being right or wrong etc... I have a business to run. We are posting here because we genuinely want to help. If you want help then you need to post a LOT more specific detail. If you just want a place to chat or ramble about Scrapeboxes failures without actually giving an example of something that could be fixed (Assuming something was actually wrong, because the dev will fix issues. I mail them bugs and they always fix them) then just say so and we will all leave and you can speak with your self. If you genuinely want help then you are going to have to be the first person to help your self by helping us. Please give explicit detail, screenshots would be good as well. Agreed. Thx.
I am not so sure, that you are completely right with the definitions of scraping and crawling. I think ?scraping" and "crawling? means the same in regard to domains, it only depends on the depth you adjust? I mean, at our domain scraper I can tell the tool to scrape just the links of the seed URL (depth=1) or the links of the links of the seed URL (depth=2) or the links of the links of the links of the links of the seed URL (depth=3) and so on... Correct me if I am wrong? Cheers Dan
