1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Another Bot for PHPBuilt

Discussion in 'Black Hat SEO Tools' started by meatro, Nov 15, 2011.

  1. meatro

    meatro BANNED BANNED

    Joined:
    Nov 21, 2009
    Messages:
    568
    Likes Received:
    997
    This is another bot that I built originating from PHPBuilt's ideas.

    This thing scrapes Squidoo top pages for URLs.. Here's how it works:

    1.) Go to Google or Scrapebox and search for:
    site:squidoo.com/topics/top {keyword}

    This returns a bunch of Squidoo's top 100 pages. Get those URLs (hope you have SB)

    2.) Save those URLs into a text file. Show this bot where to find that text file, show it where to save the URLs it finds.. Give it some proxies and run.

    3.) Take those URLs, run them through SB for PR check and there you have it. High PR Squidoo lenses that are always most likely always going to be highly ranked, therefore near the homepage and right on the homepage for whatever category they're in.

    (PR check your "topic/top" URLs first, then PR check the URLs this thing scrapes from those topics for the best Squidoo lenses to get your lens good PR.)
    [​IMG]
    Uploaded with http://imageshack.us
    Code:
    http://www.virustotal.com/file-scan/report.html?id=cba931fcb78921450da5719013e8979876501a003837911335fb1aeee5544833-1321323897
    Code:
    http://www.mediafire.com/?jgn1hpfhyie0yy7
     
    • Thanks Thanks x 5
  2. meatro

    meatro BANNED BANNED

    Joined:
    Nov 21, 2009
    Messages:
    568
    Likes Received:
    997
    I let this run overnight and checked PR this morning...

    I ended up with over 100,000 Squidoo lenses, and over 2,100 of those are PR4 or higher (88 PR5, 7 PR6, 1 PR7, all else PR4). Nearly 12,000 are PR3. :)
     
  3. dowser

    dowser Power Member

    Joined:
    Jun 5, 2011
    Messages:
    685
    Likes Received:
    122
    Location:
    canada
    Thanks, will give it a try as the idea is sound!
     
  4. maxok

    maxok Jr. VIP Jr. VIP Premium Member

    Joined:
    Oct 11, 2011
    Messages:
    696
    Likes Received:
    207
    Occupation:
    Internet
    Location:
    mmm
    thanks ;)
     
  5. dowser

    dowser Power Member

    Joined:
    Jun 5, 2011
    Messages:
    685
    Likes Received:
    122
    Location:
    canada
    How many urls have you started with?
     
  6. meatro

    meatro BANNED BANNED

    Joined:
    Nov 21, 2009
    Messages:
    568
    Likes Received:
    997
    I started with about 2,000 URLs. I just did a simple Scrapebox search for:

    F: site:squidoo.com/topics/top

    Then generic keywords like "home, auto, tech, internet, ipod, etc." That returned me 2,000+ Top 100 pages, which I ended up with just over 100,000 lens URLs. (Not all Top 100 pages actually HAVE 100 lenses.)
     
  7. dowser

    dowser Power Member

    Joined:
    Jun 5, 2011
    Messages:
    685
    Likes Received:
    122
    Location:
    canada
    Sweet, I'm scraping right now!
     
    • Thanks Thanks x 1
  8. dowser

    dowser Power Member

    Joined:
    Jun 5, 2011
    Messages:
    685
    Likes Received:
    122
    Location:
    canada
    I have a problem - when I start scraping - the "Found Top Lenses" doesn't change from zero and the output file goes from zero to 100 and back to zero. What am I doing wrong?
    I'm running it on win2003 vps.

    P.S. I've tried it on win7 and it's the same...
     
    Last edited: Nov 16, 2011
  9. meatro

    meatro BANNED BANNED

    Joined:
    Nov 21, 2009
    Messages:
    568
    Likes Received:
    997
    First you need to create a text file containing your Top 100 pages. (These are your squidoo.com/topics/top/category links from Scrapebox. 1 per line, I name mine "TopLensURL" to avoid any confusion.)

    Next, you need to create a blank text file for the found lenses.

    Set the bot's "Top 100 URL List/Lens URL List" to your "TopLensURL" file.
    Set the bot's "Save Lenses Found" to your blank text file.

    The reason it 'seems' to stay at 0, is because it moves very quickly, you will see the number begin to increase once it gets to higher figures. This is what it will do.. Scrape a Top 100 list. The number will go to 100, then back to 0. Scrape the next Top 100 list. The number will go to 200, then back to 0. Scrape the next.. 300, 0. Etc, etc.

    I did this originally because it would help to keep the bot from using too much memory when the list gets extremely long, however it actually made it very slow once it gets very long (20,000+ lenses).

    I have updated it not to do this and to keep a constant count visible, so instead of going 100... 0... 200... 0... 300... 0, it now just goes 100.... 200... 300. This makes it much faster at higher numbers.

    Please let me know if you have any other issues, I hope this helps. :)

    Code:
    http://www.virustotal.com/file-scan/report.html?id=d40d94c7536ad84b0c964e6138498f5a19edad1cc60520763e3027dac9530812-1321456750
    Code:
    http://www.mediafire.com/?hqwe16s7vge9rxs
     
    Last edited: Nov 16, 2011
  10. dowser

    dowser Power Member

    Joined:
    Jun 5, 2011
    Messages:
    685
    Likes Received:
    122
    Location:
    canada
    OK, thank you! The confusing part was that I would import the resulting "found" to the SB and sometimes it would be 100 and sometimes zero :)

    I will use your suggestions now, thank you!
     
  11. WizIMS

    WizIMS Power Member

    Joined:
    Sep 24, 2011
    Messages:
    684
    Likes Received:
    870
    Location:
    Skype - Wiz.IMS
    Home Page:
    Great idea

    PM me man or add my Skype.. we might have some work :)
     
  12. dowser

    dowser Power Member

    Joined:
    Jun 5, 2011
    Messages:
    685
    Likes Received:
    122
    Location:
    canada
    Wow, now it's working like a charm! It takes under 10 secs to get 100 top lenses! Can't wait to put it on my vps with fast connection!