1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Xenu/ScreamingFrog/ScrapPrograms Max Threads

Discussion in 'Black Hat SEO' started by paulolpd, Jan 20, 2017.

  1. paulolpd

    paulolpd Newbie

    Joined:
    Apr 1, 2013
    Messages:
    26
    Likes Received:
    1
    Hello seo partners,
    Any of you guys use Xenu link sleuth or Screamingfrog to gather links in order to find expired domains?
    Currently, i am using screamingfrog and i have a doubt setting the speed of the program. How many simultaneously threads do you think its ok to put in the crawl speeding set up, considering that increasing a lot of speed can make the server block the crawl process or return with more timeout/failed responses? (Xenu has the same setting adjust)
    PS: My internet conection has a download speed of 60mb/s
    Thanks for the help!