1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

How to sort through 10,000,000 links?

Discussion in 'Black Hat SEO' started by Robby54, Jul 17, 2012.

  1. Robby54

    Robby54 Regular Member

    Joined:
    Nov 6, 2008
    Messages:
    446
    Likes Received:
    122
    Location:
    BHW
    I harvested about 10 millions links but I do not have know what would be the easiest way to sort through them.

    What I mean for example is, some of the sites harvested are: Plig, PHPDug, SMF, Article Directory's. I would like to import them into senuke but it seems senuke dose not sort through your list and you need to do it before hand.

    I have used UD which can do this automatically and post to the sites once sorted but to be honest it kinda sucks, some sites it said that I got banned from, when checked manually was working perfectly fine. Some sites which it said was not possible to post on (like some wiki sites) I was able to post with them using extreme wiki poster, etc... and I don't want to pay $40 a month for a program just to use it to sort through lists.

    So can anyone recommend me a better alternative for sorting through list based on the platform of the site?
     
  2. Falian

    Falian Junior Member

    Joined:
    Feb 1, 2010
    Messages:
    127
    Likes Received:
    91
    I'd run them through scrapebox and sort by PageRank
     
  3. Robby54

    Robby54 Regular Member

    Joined:
    Nov 6, 2008
    Messages:
    446
    Likes Received:
    122
    Location:
    BHW
    That's what I was thinking but the thing is I thought scrapebox would only be able to detect the platform during harvesting, can it also do it with pre existing list?
     
  4. Nitros

    Nitros Power Member

    Joined:
    Jan 30, 2009
    Messages:
    573
    Likes Received:
    295
    Yes it can, with "Blog Analyzer" addon.
     
  5. tezman

    tezman Newbie

    Joined:
    Feb 11, 2012
    Messages:
    2
    Likes Received:
    0
    Why not split the file in few hundred chunks, and then if you know some PHP coder or yourself then make an array of desired catches / platform...

    Software yada :loco:
     
  6. Junkfood00

    Junkfood00 Elite Member

    Joined:
    Sep 13, 2011
    Messages:
    1,949
    Likes Received:
    1,336
    1. Harvest a lot of proxies
    2. Test and divide your proxies in lists, more than 50 in each
    3. Divide your link list into 10, each list containing 1 million
    4. Create 10 instances of Scrapebox
    5. Add to each SB instance a list of proxies and links
    6. Use the Blog Analyzer addon
    7. See if your system crashes :D

    Though it detects only WordPress/Blog Engine/Moveable Types.
     
  7. Dang3r81

    Dang3r81 Jr. VIP Jr. VIP Premium Member

    Joined:
    Jan 18, 2011
    Messages:
    301
    Likes Received:
    235
    Location:
    Germany
    Home Page:
    Hi Robby54,

    you should try the sick platform reader. You find it in the sick submitter forum and its free.
    Its not the best, but it sorts you the list, that you will get a overview ;)
    And you can add your own footprints, to filter it better as the original filter.

    In the past i used this often. But you must should split your list in smaller parts :) I started 50 times the platform reader at one time as i filtered 10 millions urls. lol

    Regards,
    Manuel