1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

[Method] Find And Sort Millions Of Forums The Easy Way

Discussion in 'Black Hat SEO' started by HelloInsomnia, Aug 9, 2011.

  1. HelloInsomnia

    HelloInsomnia Jr. Executive VIP Jr. VIP Premium Member

    Joined:
    Mar 1, 2009
    Messages:
    1,816
    Likes Received:
    2,912
    I haven't given back to the community in a long time so I wanted to share a super easy method to find millions of forums that you can get backlinks from.

    No fluff here let's jump right to it.

    What you need:

    Scrapebox
    Crusher Forum Finder (see below to download)
    Sick Platform Reader (Google it)


    The method:

    Instead of searching for profiles of spammers on forums we are going to be searching for forum userlists. Some of these userlists will display the URL of a website right on the userlist page. So instead of getting one URL at a time we can grab 5,10, or even 30.

    So let's do a quick example using an SMF forum.

    I'm actually at home right now so I'm just going to go through this using Google but it will translate easily to Scrapebox.

    First we need to find the userlists of some SMF forums, I'm using this footprint:

    Code:
    inurl:mlist "powered by smf"
    
    We can do this one of two ways, we can either find a forum which displays the URLs on the userlist page, and then scrape all of the userlist pages. Or we can input thuosands of keywords and just scrape for any userlist pages.

    The second way is pretty self explanatory so lets focus more on the first.

    I put that above footprint into Google and the first result was this:

    Code:
    http://www.thelunaticfam.com/forum/index.php?action=mlist
    
    That forum displays the URLs on the userlist page so we are in luck! We can now try to scrape more userlist pages the lazy way, or we can look at the URL and manipulate it to get more pages.

    To do that we take the URL of the second page which is:

    Code:
    http://www.thelunaticfam.com/forum/index.php?action=mlist;sort=realName;start=30
    
    And we put it into a text file and then add 30 to the end of the URL so the next page would be start=60 and then start=90 and so on.

    So our text file would look like:

    Code:
    http://www.thelunaticfam.com/forum/index.php?action=mlist;sort=realName;start=30
    http://www.thelunaticfam.com/forum/index.php?action=mlist;sort=realName;start=60
    http://www.thelunaticfam.com/forum/index.php?action=mlist;sort=realName;start=90
    http://www.thelunaticfam.com/forum/index.php?action=mlist;sort=realName;start=120
    http://www.thelunaticfam.com/forum/index.php?action=mlist;sort=realName;start=150
    And so on
    
    And that forum only has 6 pages but many will have hundreds or thousands.

    The reason we do this is to get the userlist pages which contain the URLs.

    Moving on...

    We then take those pages and use the link extractor plugin in Scrapebox to extract all the outgoing links. These are the links of forum spammers, and some links of course will be legit but thats okay.

    Depending on the amount of links you should export the list and split the list up to about 1000-2000 links at a time.

    Then simply check the backlinks of the extracted links. I like to do 1000-2000 at a time because I am running Scrapebox on a normal PC others may be able to handle a lot more links, just do what works best for your setup.

    If you did this right you will have hundreds of thousands (or even millions) of links, but that's not good enough because we only want the forums and depending on your program you need to know which URLs are on each forum platform.

    That's where the Crusher forum finder comes into play. Take all of the URLs (you may want to split them actually into 100k at a time for this) and run them through the Crusher forum finder. This will sort the forums from the rest of the links.

    At this point you should have tens of thousands of forums (or millions if you do it big enough!).

    The last part is optional depending on the program you use to make forums profiles. But you can then split the list down into 10k at a time and run them through the Sick platform reader. This should tell you with pretty good accuracy which ones are SMF or vBulletin, etc...

    Link to the Crusher Forum Finder:
    Code:
    http://www.mediafire.com/?5lazq0e7e56ekk1
    
    Virus Total for Crusher Forum Finder

    1/42
    Code:
    http://www.virustotal.com/file-scan/report.html?id=a7a9dd1984226c17a788924805ed8f218a30797ee45768e186c50e7a6eef4a0b-1312922120
    
    Link to the Crusher Forum Finder (64 bit):
    Code:
    http://www.mediafire.com/?3n9n0m9922vy9nf
    
    Virus Total for Crusher Forum Finder (64 bit):

    Code:
    http://www.virustotal.com/file-scan/report.html?id=0cae63f2afb68dfcc2fda78a8753484a5e3d77dfaef1a7861c5e89d2bd69fc15-1312922793
    
    I'm pretty sure it is a false positive but just in case run Crusher Forum Finder in a sandbox just in case.

    If Crusher is here on BHW show him some love, he made the forum finder - not me.
     
    • Thanks Thanks x 2
    Last edited: Aug 9, 2011
  2. hgear

    hgear Power Member

    Joined:
    Feb 8, 2008
    Messages:
    546
    Likes Received:
    193
    Occupation:
    Stacks On Stacks Manager
    Location:
    At Your Mom's House
    Holly crap...epic!
     
  3. fr33sh

    fr33sh Registered Member

    Joined:
    Jan 29, 2012
    Messages:
    98
    Likes Received:
    4
    This is really great share. i wish i had xrumer. Thanks allot.