mainceaft
Senior Member
- Apr 10, 2013
- 881
- 184
Hi as title said I want to develop Crawler that search the web and hunt good old domains that have BL's from big Authority sites , the crawler will give you warning whenever any of these domains is about to expire son to hunt it ,
This will be my first to do something on this scale , as you can see this project can become expensive specially on servers side (e.g 3 Dedi servers or more )
The tools I'll develop will have three parts first the crawler that will scab the internet searching for links ,
The DB engine which will pickup the crawler data and sore in the DB
And last the user interface where you can config the server to skip certain type of website(e.g pron ) , + it's will alert you (by any means possible) to get ready to hunt new expired domain .
At the beginning I'll develop the crawler in php(with a lot of performance tweaking) to save time and easy modification/upgrade , if the system were succeed I'll port it to C/C++ .
This is not easy task as you know and wont complete at one day/night I'll need at least two week before start testing it , so please be patient .
Last but not least , why would I needs partner !
First to share ideas and improve the design and usability , honesty I don't have that experience using any SEO tools at all , Second funding the project since I'm current can' effort paying to any Dedicated hosting plan , the first testing server will cost around 30$ , third to have motivations and not get myself distracted with something else .
One more thing you my ask how good my skills are and what's my previous working experience , well I design couple tools in the past for private client (one of them related to twitter automation) , I never worked with these big PHP frameworks like Laravel Symfony beaus it's will gain me nothing I don't develop E-Commerce sites or Schools learning platform Or the next Facebook , i love creating automation tools simply because it's what computers are made for , and I love work with big data specially crawling the web .
This will be my first to do something on this scale , as you can see this project can become expensive specially on servers side (e.g 3 Dedi servers or more )
The tools I'll develop will have three parts first the crawler that will scab the internet searching for links ,
The DB engine which will pickup the crawler data and sore in the DB
And last the user interface where you can config the server to skip certain type of website(e.g pron ) , + it's will alert you (by any means possible) to get ready to hunt new expired domain .
At the beginning I'll develop the crawler in php(with a lot of performance tweaking) to save time and easy modification/upgrade , if the system were succeed I'll port it to C/C++ .
This is not easy task as you know and wont complete at one day/night I'll need at least two week before start testing it , so please be patient .
Last but not least , why would I needs partner !
First to share ideas and improve the design and usability , honesty I don't have that experience using any SEO tools at all , Second funding the project since I'm current can' effort paying to any Dedicated hosting plan , the first testing server will cost around 30$ , third to have motivations and not get myself distracted with something else .
One more thing you my ask how good my skills are and what's my previous working experience , well I design couple tools in the past for private client (one of them related to twitter automation) , I never worked with these big PHP frameworks like Laravel Symfony beaus it's will gain me nothing I don't develop E-Commerce sites or Schools learning platform Or the next Facebook , i love creating automation tools simply because it's what computers are made for , and I love work with big data specially crawling the web .