1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Tool ...

Discussion in 'Black Hat SEO Tools' started by crisnoop, Jul 23, 2012.

  1. crisnoop

    crisnoop Newbie

    Joined:
    Jul 10, 2012
    Messages:
    8
    Likes Received:
    1
    Seeking a tool to extract all urls from a domain. I extract all urls from a forum or blog to know what is most pr!
     
    • Thanks Thanks x 1
  2. rostonix

    rostonix Senior Member

    Joined:
    Dec 20, 2009
    Messages:
    897
    Likes Received:
    1,446
    Occupation:
    Developer
    Location:
    Russia
    Home Page:
    It's doable in Zennoposter if you have skills :)
     
  3. jkwilson78

    jkwilson78 Regular Member Premium Member

    Joined:
    Jun 24, 2010
    Messages:
    224
    Likes Received:
    311
    You could try Xenu Link Sleuth for the link extraction part. You can set it to dig X number of levels deep in a site to extract all urls from a site, ignore external links, and it is multithreaded. I usually run 50 threads. It's very fast and best of all free :)

    To check the PR in bulk you could use something like scrapebox or use Netpeak Checker...it's free too!
     
  4. VIC SEO

    VIC SEO Elite Member

    Joined:
    Feb 19, 2010
    Messages:
    2,156
    Likes Received:
    363
    Gender:
    Male
    Occupation:
    SEO Specialist
    Location:
    iSynergyMedia
    Home Page:
    you can extract urls from same domain using scrapebox ,just put site: in the volumn ,select proxies and what type of blog it is and hit it.it will harvest all urls from single domain
     
    • Thanks Thanks x 1
  5. crisnoop

    crisnoop Newbie

    Joined:
    Jul 10, 2012
    Messages:
    8
    Likes Received:
    1
    iSynergy, and forums?
     
  6. extremeboy

    extremeboy Jr. VIP Jr. VIP

    Joined:
    Jul 8, 2010
    Messages:
    2,992
    Likes Received:
    646
    Occupation:
    World Best RANK Tracker SERPCloud.com
    Home Page:
    Scrapebox can do it and for it you don't need proxy as well you are scarping its url from 1 domain instead that sites allow that much client request on server too fast ;)
     
  7. crisnoop

    crisnoop Newbie

    Joined:
    Jul 10, 2012
    Messages:
    8
    Likes Received:
    1
    I just bought a box scrape, someone could teach me to do what his friend iSynergy said:
    site: in the volumn
    put it where?
    THX.
     
  8. paulwilliams972

    paulwilliams972 Regular Member

    Joined:
    Apr 24, 2012
    Messages:
    370
    Likes Received:
    37
    Location:
    Tester World
    scrapebox is always good. i like it and highly recommend.
     
  9. jkwilson78

    jkwilson78 Regular Member Premium Member

    Joined:
    Jun 24, 2010
    Messages:
    224
    Likes Received:
    311
    crisoop,

    In the "harvester" box, just put in "site:domaintolookup.com" without the quotes. Also make sure the "custom footprint" option is selected. Then in the "Search Engines & Proxies" section put "1000" in the results box. This is the max number of urls you can grab at once.
     
  10. tb303

    tb303 Power Member

    Joined:
    Dec 18, 2011
    Messages:
    601
    Likes Received:
    280
    If the site has a sitemap you can get more than 1000 results by using the "Scrapebox sitemap scraper" addon i think
     
  11. csguy

    csguy BANNED BANNED

    Joined:
    Jul 13, 2012
    Messages:
    396
    Likes Received:
    42
    I would use "site:domain" in Google (or scrapebox) no need to check a footprint, it'll just give you all the pages Google knows about. Then you can run them through PR checker, OBL, etc.
     
  12. Bestbuyfoam

    Bestbuyfoam Jr. VIP Jr. VIP Premium Member

    Joined:
    Nov 14, 2009
    Messages:
    1,637
    Likes Received:
    536
    Thanks guys for sharing this going to have to break out my SB again.
     
  13. crisnoop

    crisnoop Newbie

    Joined:
    Jul 10, 2012
    Messages:
    8
    Likes Received:
    1
    Friends, thank you for help was exactly what I needed, now that SB is very powerful!
    Another question: - If you get the self, posting comments on large lists you run the risk of being banned by google?
    How does this story have banned the domain?