Search results

  1. L

    Website bot crawler, any ideas?

    Yesterday I wrote a bot that can crawls everything on a website (all internal links like google bot) with only 120 lines of code. Now i planning will take all the data of all websites in the same niche and then create a collection website (eg movie sites), do you have any ideas with this bot?
Back
Top