What website to scrape?
- Thread starter pindarots
- Start date
razharov
Senior Member
- Oct 19, 2022
- 898
- 961
oooor, simply login to expired domains, and filter by wikipedia backlink
That sounds cool
- Sep 10, 2010
- 11,805
- 26,657
Oh well, that’s an option for sure. But, there are lots of wolves waiting there to register one before you do so..oooor, simply login to expired domains, and filter by wikipedia backlink
Depends how up to date the database isoooor, simply login to expired domains, and filter by wikipedia backlink
razharov
Senior Member
- Oct 19, 2022
- 898
- 961
Well yes, but actually no, or not in the way it would worth to bother.Oh well, that’s an option for sure. But, there are lots of wolves waiting there to register one before you do so..
Registers, hosts, etc take 90% of the good domains, after that the "wolfs" and after it goes to the market for open registration.
I checked like 10-100k already from expired domains, and the success rate to find that isn't spammed to death with adult history is almost 0.
I remember once I found a business consulting domain with DA40, and 0 spam backlinks, that was my biggest found
Can even build a bot that buys them automatically based on a list or based on amount of backlinks or something like thatWell yes, but actually no, or not in the way it would worth to bother.
Registers, hosts, etc take 90% of the good domains, after that the "wolfs" and after it goes to the market for open registration.
I checked like 10-100k already from expired domains, and the success rate to find that isn't spammed to death with adult history is almost 0.
I remember once I found a business consulting domain with DA40, and 0 spam backlinks, that was my biggest found
razharov
Senior Member
- Oct 19, 2022
- 898
- 961
That would be the worst idea ever.
The more the backlink the more likely its spammy, if there would be a good domain with many backlinks, it would never go on public registration.
Ye i mean whatever makes a domain valuable
- Sep 10, 2010
- 11,805
- 26,657
You could choose to procrastinate about everything or be productive. Always up to you.Well yes, but actually no, or not in the way it would worth to bother.
Registers, hosts, etc take 90% of the good domains, after that the "wolfs" and after it goes to the market for open registration.
I checked like 10-100k already from expired domains, and the success rate to find that isn't spammed to death with adult history is almost 0.
I remember once I found a business consulting domain with DA40, and 0 spam backlinks, that was my biggest found
Why I suggested in my OP was to give him a cool idea to build a project on. A practical project always challenges you in new ways you might not think about. There is no harm to try. Even if no one else uses the tool, he at least has a private tool for such work.
Secondly, you might still get awesome domains if you are lucky. So it is win-win.
Ye totally agree.
I'm currently working on it and still doing research how to make this work. I will probably automate everything send it to a database and send an alert out to Telegram when a domain is expired.
First have to find some logic how to scrape wikipediia than scrape the url do a whois lookup and save it. Also it might help focus on a specific category first.
Thanks for the idea. I have it up and running right now and found already some expired domains but not available unfortunately. Not sure how this works after some googling a came across this: https://www.icann.org/resources/pages/epp-status-codes-2014-06-16-en#redemptionPeriod
nurulafsar17
BANNED
- Dec 12, 2020
- 66
- 14
The type of website you should scrape will depend on the specific data you are looking for. If you are looking for general information about a topic, a news website or an encyclopedia may be a good option. If you are looking for more specific data, you may want to consider scraping a forum, a review website, or even a business directory. Ultimately, the type of website you decide to scrape should depend on the types of data you are hoping to obtain.
- Apr 26, 2020
- 1,711
- 995
Scrape something of high value.
For example, there are 1 million plus people on indie hackers most of them have their email openly available.
Scrape them with their mail.
Some guy did that and made a slack group of highly successful entrepreneurs and now makes really good money through podcasts and blog sponsorships.
Sounds interesting. Will take a look into it for sure
- Sep 10, 2010
- 11,805
- 26,657
You can register them after the redemption period if I got it right.