1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Website Link Extractor ???

Discussion in 'Black Hat SEO' started by WTennisPL, Feb 25, 2011.

  1. WTennisPL

    WTennisPL Registered Member

    Joined:
    Jan 26, 2011
    Messages:
    66
    Likes Received:
    0
    I am looking for a tool/software can get all link from a website. Anybody know ?
     
  2. chris456

    chris456 Regular Member

    Joined:
    May 17, 2010
    Messages:
    281
    Likes Received:
    567
    win web crawler , web data extractor , visual web spider , last one is the best one , web data extractor 8.1 you can find here on BHW
     
  3. haridada

    haridada Senior Member

    Joined:
    Oct 9, 2008
    Messages:
    996
    Likes Received:
    382
    Location:
    Chennai
    one question i want to know is, i have a site full of .html files and .php files too. Does any of the above tool just extract the .html files only?

    Just want to know so i could download the one which does what i need? :confused:
     
  4. chris456

    chris456 Regular Member

    Joined:
    May 17, 2010
    Messages:
    281
    Likes Received:
    567
    All of them has exclusion extensions there , so if you are trying to avoid to extract .php files , you can try to add it to other extensions like .jpg , .swf and the others , I have never try it but I think it should avoid visiting it (all of them has this option I think , Visual Web Spider is best because you can also setup H1 , H2 , H3 , <title> , <body> , <p> , <br> , <html> almost all tags , so it is much more powerful than web data extractor or win web crawler , I like win web crawler because it is quick and for link extraction is the best , I wanted to upload it here too , but when I wanted to test it for viruses , I have found 13 -:( , so I didn't post it here , I am using it for 2 years , never had a problem but the program is not clean .
     
  5. WTennisPL

    WTennisPL Registered Member

    Joined:
    Jan 26, 2011
    Messages:
    66
    Likes Received:
    0
    visual web spider is the best software ? The soft can index all link from a website as well ?
     
  6. chris456

    chris456 Regular Member

    Joined:
    May 17, 2010
    Messages:
    281
    Likes Received:
    567
    yes , it can , all of them support external links extraction, you can also setup following those external links to another sites , so it can grab the links infinitely
     
  7. WTennisPL

    WTennisPL Registered Member

    Joined:
    Jan 26, 2011
    Messages:
    66
    Likes Received:
    0
    Are you owner of that soft ?
    I just used trial version but it seem not work well.
    I got error MySQL Error.... in Page Tittle and Page Text
     
  8. roknimam

    roknimam Regular Member

    Joined:
    May 13, 2010
    Messages:
    202
    Likes Received:
    217
    Occupation:
    I enjoy good company and good times however I'm no
    Location:
    {Mind|Desk,Bead}
    If you own scrapebox you can use addon Link Extractor, if you don't own scrapebox you should buy it. If you don't want spending money you can do it with SeoQuake toolbar it's not perfect but it's free, also you can analyze bunch of other things.
     
  9. Derek Foreal

    Derek Foreal Junior Member

    Joined:
    Apr 10, 2010
    Messages:
    190
    Likes Received:
    767
    Gender:
    Male
    Location:
    East Coast USA
    Well you could always use a simple pc of java script code that you just simply create a bookmark in firefox and Name the bookmark something descriptive such as "Scrape All Links" then for the location copy & paste the below pc of java script.

    Every-time you hit that bookmark it'll scrape all links on the page you have open and it'll open a new page with all the links it scraped so you can easily save it, copy & paste it somewhere, etc.

    I used to use this alot and it's quite handy even though I haven't used it in awhile 8)

    Code:
    javascript:loc=location.href;x=document.getElementsByTagName(%22A%22);y=window.open();y.document.write(%22<html><head><title>Ripped%20Links<title></head><body><h3>Links%20from%20%22+loc+%22</h3>%22);for(n=0;n<x.length;++n){if(x[n].href!=%22%22)y.document.write(%22<p>%22+x[n].href+%22%20</p>%22);}y.document.write(%22</body></html>%22);y.document.close();void%200;
     
  10. WTennisPL

    WTennisPL Registered Member

    Joined:
    Jan 26, 2011
    Messages:
    66
    Likes Received:
    0
    I have scrape as well but the soft do only link added, not do all website :(
     
  11. chris456

    chris456 Regular Member

    Joined:
    May 17, 2010
    Messages:
    281
    Likes Received:
    567
    I am not an owner of the soft - Visual web spider was best so I 've purchased it , Web data extractor I have uploaded here one month ago (search for it ), it works , win web crawler I can upload here too - works really very smoothly and quickly , but as I said , I use it 2 years but when I was willing to upload it here too, I made a virustotal which reported it is 13/43 virus positive .

    I never got any error with visual web spider ... strange u did.
     
  12. WTennisPL

    WTennisPL Registered Member

    Joined:
    Jan 26, 2011
    Messages:
    66
    Likes Received:
    0
    Ok, please send me one via pm :)
    Thanks
     
  13. Aty

    Aty Jr. VIP Jr. VIP

    Joined:
    Jan 27, 2011
    Messages:
    5,400
    Likes Received:
    3,691
    Home Page:
  14. SuperLinks

    SuperLinks Elite Member

    Joined:
    Jul 14, 2008
    Messages:
    2,903
    Likes Received:
    847
    Location:
    New York
    Scrapebox can do it.

    Just Scrape all of the sites URLs, or use its XML/Sitemap crawler functionality.

    Then load those into the link extractor plugin, opt in for either internal or external, or both and let it loose.

    You'll get the links you need :)
     
  15. Typlo

    Typlo Jr. VIP Jr. VIP Premium Member

    Joined:
    Sep 22, 2010
    Messages:
    1,285
    Likes Received:
    499
    Location:
    New York
    Home Page:
    Download "link gopher" plugin for firefox. Simply click "extract all links" and it gives you a list of the urls
     
  16. SuperLinks

    SuperLinks Elite Member

    Joined:
    Jul 14, 2008
    Messages:
    2,903
    Likes Received:
    847
    Location:
    New York
    Is that for the page, or the whole site?
     
  17. Checkmate

    Checkmate Elite Member

    Joined:
    Aug 9, 2010
    Messages:
    1,534
    Likes Received:
    637
    ScrapeBox can do this will the addon called "link extractor"
     
  18. macdonjo3

    macdonjo3 Jr. VIP Jr. VIP Premium Member

    Joined:
    Nov 8, 2009
    Messages:
    5,560
    Likes Received:
    4,317
    Location:
    Toronto
    Home Page:
    ScrapeBox has an addon like this?
     
  19. hotleatherdreams

    hotleatherdreams Registered Member

    Joined:
    Mar 29, 2010
    Messages:
    78
    Likes Received:
    18
    Occupation:
    ecommerce website CODER - SUCK at design and SEO
    Location:
    third rock from hell
    yes SB does. check the addons menu for link extractor. im bummed i cant get into my computer right now to use it
     
  20. moonlighsunligh

    moonlighsunligh Jr. VIP Jr. VIP Premium Member

    Joined:
    May 1, 2010
    Messages:
    1,623
    Likes Received:
    218
    scrapebox can do next to anything