Tool ...

Discussion in 'Black Hat SEO Tools' started by crisnoop, Jul 23, 2012.

  1. crisnoop

    crisnoop Newbie

    Joined:
    Jul 10, 2012
    Messages:
    8
    Likes Received:
    1
    Seeking a tool to extract all urls from a domain. I extract all urls from a forum or blog to know what is most pr!
     
    • Thanks Thanks x 1
  2. rostonix

    rostonix Senior Member

    Joined:
    Dec 20, 2009
    Messages:
    897
    Likes Received:
    1,446
    Occupation:
    Developer
    Location:
    Russia
    It's doable in Zennoposter if you have skills :)
     
  3. jkwilson78

    jkwilson78 Regular Member Premium Member

    Joined:
    Jun 24, 2010
    Messages:
    224
    Likes Received:
    312
    You could try Xenu Link Sleuth for the link extraction part. You can set it to dig X number of levels deep in a site to extract all urls from a site, ignore external links, and it is multithreaded. I usually run 50 threads. It's very fast and best of all free :)

    To check the PR in bulk you could use something like scrapebox or use Netpeak Checker...it's free too!
     
  4. VIC SEO

    VIC SEO Elite Member

    Joined:
    Feb 19, 2010
    Messages:
    2,165
    Likes Received:
    365
    Gender:
    Male
    Occupation:
    SEO Specialist
    you can extract urls from same domain using scrapebox ,just put site: in the volumn ,select proxies and what type of blog it is and hit it.it will harvest all urls from single domain
     
    • Thanks Thanks x 1
  5. crisnoop

    crisnoop Newbie

    Joined:
    Jul 10, 2012
    Messages:
    8
    Likes Received:
    1
    iSynergy, and forums?
     
  6. extremeboy

    extremeboy Jr. VIP Jr. VIP

    Joined:
    Jul 8, 2010
    Messages:
    3,437
    Likes Received:
    700
    Occupation:
    World Best RANK Tracker SERPCloud.com
    Home Page:
    Scrapebox can do it and for it you don't need proxy as well you are scarping its url from 1 domain instead that sites allow that much client request on server too fast ;)
     
  7. crisnoop

    crisnoop Newbie

    Joined:
    Jul 10, 2012
    Messages:
    8
    Likes Received:
    1
    I just bought a box scrape, someone could teach me to do what his friend iSynergy said:
    site: in the volumn
    put it where?
    THX.
     
  8. paulwilliams972

    paulwilliams972 Regular Member

    Joined:
    Apr 24, 2012
    Messages:
    370
    Likes Received:
    37
    Location:
    Tester World
    scrapebox is always good. i like it and highly recommend.
     
  9. jkwilson78

    jkwilson78 Regular Member Premium Member

    Joined:
    Jun 24, 2010
    Messages:
    224
    Likes Received:
    312
    crisoop,

    In the "harvester" box, just put in "site:domaintolookup.com" without the quotes. Also make sure the "custom footprint" option is selected. Then in the "Search Engines & Proxies" section put "1000" in the results box. This is the max number of urls you can grab at once.
     
  10. tb303

    tb303 Senior Member

    Joined:
    Dec 18, 2011
    Messages:
    986
    Likes Received:
    677
    If the site has a sitemap you can get more than 1000 results by using the "Scrapebox sitemap scraper" addon i think
     
  11. csguy

    csguy BANNED BANNED

    Joined:
    Jul 13, 2012
    Messages:
    396
    Likes Received:
    42
    I would use "site:domain" in Google (or scrapebox) no need to check a footprint, it'll just give you all the pages Google knows about. Then you can run them through PR checker, OBL, etc.
     
  12. Bestbuyfoam

    Bestbuyfoam Elite Member

    Joined:
    Nov 14, 2009
    Messages:
    1,637
    Likes Received:
    539
    Thanks guys for sharing this going to have to break out my SB again.
     
  13. crisnoop

    crisnoop Newbie

    Joined:
    Jul 10, 2012
    Messages:
    8
    Likes Received:
    1
    Friends, thank you for help was exactly what I needed, now that SB is very powerful!
    Another question: - If you get the self, posting comments on large lists you run the risk of being banned by google?
    How does this story have banned the domain?