Hi, I already sent you a PM for the coupon to try the pro version
thanks
 
Hello,

I started to test it yesterday, i have some problems.

1) I did all settings but i cant use domain hunting from a website feature. It says crawling started but not starting to crawling. What settings do i need if i want to scrape all wikipedia or all cnn ?

2) Is it possible to scrape just .com, .net, .org, .info, .co TLD's while scraping ?

3) Its adding some domains and different TLD's to my list but when i check them via domain registration company: ''domain is taken.'' For example: ***** added to my list after then scraping.

Thanks,
 
Last edited:
I am also interested in purchasing the Pro version, please send me a coupon as well.
Thanks.

is there any trial version to try out your software before I buy?

can i have coupon for trial?

I would like to try out trial version.Seriously Interested in pro version.

Can you send me the coupon for the trial version? Would love to try this out.

All should have your coupon codes by now. If any are still waiting just let me know and I will send one over to you.

Hello,

I started to test it yesterday, i have some problems.

1) I did all settings but i cant use domain hunting from a website feature. It says crawling started but not starting to crawling. What settings do i need if i want to scrape all wikipedia or all cnn ?

2) Is it possible to scrape just .com, .net, .org, .info, .co TLD's while scraping ?

3) Its adding some domains and different TLD's to my list but when i check them via domain registration company: ''domain is taken.'' For example: ******** added to my list after then scraping.

Thanks,

1) What exactly are you entering for something like Wikipedia? I have tested on a few machines and there does not appear to be an issue, nor has anyone else mentioned an issue. It could just be that the website address is not being entered correctly or that the proxies are not working. Also make sure that you increase the number of crawl levels if you wish to crawl the site as a crawl level of 1 (default level) will only crawl the page that you enter for domains.

2) You can set a filter using the controls in the Filter Domains tab. You cannot specify to just scrape these types of domains but you can filter for them automatically if you set the filters before hand.

3) If I were you I would remove that domain from your post (edit it asap), it is dropping very soon and is a very valuable domain. From what I have seen .sg domains are reasonably priced and it could be worth a lot of money. So it is not available right now but it could be in the next few days, you would have to check the full details on the drop period for a .sg domain, it is already in the deletion process so definitely one to keep an eye on.
 
All should have your coupon codes by now. If any are still waiting just let me know and I will send one over to you.



1) What exactly are you entering for something like Wikipedia? I have tested on a few machines and there does not appear to be an issue, nor has anyone else mentioned an issue. It could just be that the website address is not being entered correctly or that the proxies are not working. Also make sure that you increase the number of crawl levels if you wish to crawl the site as a crawl level of 1 (default level) will only crawl the page that you enter for domains.

2) You can set a filter using the controls in the Filter Domains tab. You cannot specify to just scrape these types of domains but you can filter for them automatically if you set the filters before hand.

3) If I were you I would remove that domain from your post (edit it asap), it is dropping very soon and is a very valuable domain. From what I have seen .sg domains are reasonably priced and it could be worth a lot of money. So it is not available right now but it could be in the next few days, you would have to check the full details on the drop period for a .sg domain, it is already in the deletion process so definitely one to keep an eye on.

Hello Jimbo,

Thank you.
1) What crawl level i need for all wikipedia ?
2) Thanks.
3) I edited.
 
1) What crawl level i need for all wikipedia ?

A lot, it is an absolutely huge website and you won't have enough memory to crawl the entire site. I am currently working on a system that will be able to crawl far more levels than is currently possible as opposed to how it is now where all crawled pages are stored in memory. The new system should be available tomorrow or the day after.

The current way is by far the fastest way but limiting to a lot of users based on the amount of memory it uses for sites where there are many millions of pages. The new method will not have this limitation.
 
A lot, it is an absolutely huge website and you won't have enough memory to crawl the entire site. I am currently working on a system that will be able to crawl far more levels than is currently possible as opposed to how it is now where all crawled pages are stored in memory. The new system should be available tomorrow or the day after.

The current way is by far the fastest way but limiting to a lot of users based on the amount of memory it uses for sites where there are many millions of pages. The new method will not have this limitation.

WOW cannot wait for that update. This thing is going to be a beast by the time you're done with it; already is!

Hey, another suggestion that would be of great benefit I think:

You currently have options to crawl a site and all its internal pages and also t load a list of pages, but that only crawls the level of the page, and no further;

Would it not be possible to load a list of sites and have the soft crawl them one after the other and save all domains from all of them.

This way if you are targetting a small specific niche and only looking for domains within that niche you could come up with a list of 20-30 small-mid authority sites in that niche, and have the soft crawl each one at a time accoriding to the specified depth level in the settings, and output all th expired domians from all the sites. This prob would not be good for something like Wikipedia or HuffPost, but is ideal for smaller specific niche based sites.

What do you think?
 
Hi, could you please send me the coupon for the trial version? Thanks.

Sent.

thank you.........

Just signed up... looking forward to giving it a spin.

Thanks guys.

WOW cannot wait for that update. This thing is going to be a beast by the time you're done with it; already is!

Hey, another suggestion that would be of great benefit I think:

You currently have options to crawl a site and all its internal pages and also t load a list of pages, but that only crawls the level of the page, and no further;

Would it not be possible to load a list of sites and have the soft crawl them one after the other and save all domains from all of them.

This way if you are targetting a small specific niche and only looking for domains within that niche you could come up with a list of 20-30 small-mid authority sites in that niche, and have the soft crawl each one at a time accoriding to the specified depth level in the settings, and output all th expired domians from all the sites. This prob would not be good for something like Wikipedia or HuffPost, but is ideal for smaller specific niche based sites.

What do you think?

What you are suggesting is currently how things are done. When crawling a list of "seed sites" using the "Crawl Page List" DHG will group all pages in the list from the same domain into a website profile and then crawl that as a single site and not deviate from that site when crawling until it gets to the next site profile.

I can go into more depth on this if you would like, I know I am not the best at describing things at times.
 
Back
Top
AdBlock Detected

We get it, advertisements are annoying!

Sure, ad-blocking software does a great job at blocking ads, but it also blocks useful features and essential functions on BlackHatWorld and other forums. These functions are unrelated to ads, such as internal links and images. For the best site experience please disable your AdBlocker.

I've Disabled AdBlock