1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Problem: Xenu won't finish spidering, any help?

Discussion in 'Black Hat SEO Tools' started by Traste, Feb 24, 2014.

  1. Traste

    Traste Regular Member

    Joined:
    Dec 18, 2011
    Messages:
    304
    Likes Received:
    22
    Hi,

    This is mi setup for Xenu: 100 threads and 200 levels. It is running on a virtual machine with 6 GB of RAM and a quad core processor. According to task manager, the computer has 40% free RAM and 50% CPU. But Xenu won't finish:

    View attachment 46488

    Sometimes I get an error: "Out of memory", but there's still free RAM. Any ideas?

    Thanks
     
  2. tb303

    tb303 Power Member

    Joined:
    Dec 18, 2011
    Messages:
    601
    Likes Received:
    280
    I dont think the standard xenu can use that much ram as its 32bit application? or am i wrong on how that works?
    I have seen a 64bit version of xenu somewhere though but cant find it now...

    Also 200 levels is a lot! I wouldnt expect that to ever finish!
     
  3. Traste

    Traste Regular Member

    Joined:
    Dec 18, 2011
    Messages:
    304
    Likes Received:
    22
    Yes, but still... Xenu stops suddenly, no info, no nothing :-/
     
  4. tb303

    tb303 Power Member

    Joined:
    Dec 18, 2011
    Messages:
    601
    Likes Received:
    280
    • Thanks Thanks x 1
  5. Traste

    Traste Regular Member

    Joined:
    Dec 18, 2011
    Messages:
    304
    Likes Received:
    22
  6. allaboutdomains

    allaboutdomains Junior Member

    Joined:
    Mar 3, 2014
    Messages:
    135
    Likes Received:
    29
    Xenu and scrapers like them are really CPU and memory intensive. I finally built one which was optimized for finding expired domains and now I find hundreds of quality domains each week.
     
  7. Dicbo

    Dicbo Newbie

    Joined:
    Nov 30, 2013
    Messages:
    9
    Likes Received:
    0
    Occupation:
    Internet Marketer
    Home Page:
    ive been running xenu on different VPS and the speed is so slow. Anybody knows a good vps that can handle Xenu multi-threading?