1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

How to find and collect invalid links from webpages?

Discussion in 'Black Hat SEO Tools' started by wpbacklinks, Dec 15, 2012.

Thread Status:
Not open for further replies.
  1. wpbacklinks

    wpbacklinks Jr. VIP Jr. VIP Premium Member

    Joined:
    Mar 27, 2010
    Messages:
    3,399
    Likes Received:
    1,339
    Gender:
    Male
    Occupation:
    Affiliate Marketer
    Location:
    Everywhere
    Hi,

    Need someone to help me.

    I want to Check and Collect invalid links from webpages automatically, how can i do that? without the copy-paste thing.

    Say i have many urls inside a .txt file.

    http://www.pressreleasesite1.com/tech/0023432421
    http://www.pressreleasesite1.com/tech/0023432422
    http://www.pressreleasesite1.com/tech/0023432423
    http://www.pressreleasesite1.com/tech/0023432424
    http://www.pressreleasesite1.com/tech/0023432425
    http://www.pressreleasesite1.com/tech/0023432426
    etc.
    arround 10.000 urls or more inside 1 .txt file

    The script/tool must check all links inside those webpages (each webpage may have 5 OBL or more), and collect all invalid links from those webpages (only the invalid ones) Then the script will automatically generate a new .txt file contains all invalid links.

    Any help is appreciated. i even want to pay if someone can get this done.

    or anyone know a good tool which can do this task? i do search on google with no luck.

    a big THANK YOU!
     
    Last edited: Dec 15, 2012
  2. iboga

    iboga Junior Member Premium Member

    Joined:
    Apr 2, 2009
    Messages:
    106
    Likes Received:
    12
    Occupation:
    project manager / company owner
    Location:
    France
    Well easy way ils to write a script to do this...
     
    • Thanks Thanks x 1
  3. mhatter

    mhatter Regular Member

    Joined:
    Feb 8, 2009
    Messages:
    224
    Likes Received:
    93
    • Thanks Thanks x 1
  4. wpbacklinks

    wpbacklinks Jr. VIP Jr. VIP Premium Member

    Joined:
    Mar 27, 2010
    Messages:
    3,399
    Likes Received:
    1,339
    Gender:
    Male
    Occupation:
    Affiliate Marketer
    Location:
    Everywhere
    Thank you! I'd appreciate it if you can help.

    Thank you for The response, i have tried both but these tools will do the same old way. i don't want to check all url inside the .txt message 1 by 1. and also they are desktop software. any script which can do this easy?
     
    Last edited: Dec 16, 2012
  5. bizarre

    bizarre Newbie

    Joined:
    Sep 28, 2012
    Messages:
    39
    Likes Received:
    23
    I use a web page crawler and just craw sites looking for dead links. There are lots of tools like Xenu or Screaming frog that can do this. I don't know if that helps given you want to do it from a text folder. one other thing that occurs to me is if its only a free hundred in a txt and you just want to see if the page loads use a online tool to open urls. Google open multiple urls and there will be one you can just copy and paste a list into in order to open them all at once.
     
    • Thanks Thanks x 1
  6. SmartMan

    SmartMan BANNED BANNED

    Joined:
    Jul 25, 2012
    Messages:
    673
    Likes Received:
    1,244
    Hi mate. You can do this with SB.

    1. Import your list into SB's Link Extractor Addon and harvest external links.
    2. After saving extracted external links, run it through Alive Check Addon to filter out dead links.(Run this process 3-4 times to make sure all links are thoroughly checked)
    3. Export Dead links to your Hard Drive.

    Let me know if you want any help from me.
     
    • Thanks Thanks x 2
  7. wpbacklinks

    wpbacklinks Jr. VIP Jr. VIP Premium Member

    Joined:
    Mar 27, 2010
    Messages:
    3,399
    Likes Received:
    1,339
    Gender:
    Male
    Occupation:
    Affiliate Marketer
    Location:
    Everywhere
    Thanks. I think some of desktop softwares like Xenu, etc. can't help that much.

    Currently i am using firefox mozilla addon called "LinkChecker" but you know that wasting time so much. to copy and paste the invalid links manually.
     
Thread Status:
Not open for further replies.