1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Linux spider for seo

Discussion in 'Black Hat SEO Tools' started by interkul, Aug 4, 2017.

  1. interkul

    interkul Jr. VIP Jr. VIP

    Joined:
    May 21, 2017
    Messages:
    125
    Likes Received:
    31
    Gender:
    Male
    Hi guys, what I'm looking for is web spider, working on linux (Windows is also acceptable), that works on many threads.
    What he should do:
    analyze a list of different domains, build sitemap for all of them, search for specific regex on every page and save urls that contain that regex.
    If there is no any, he can just make sitemaps for list of my domains with many threads and save them to file.