1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

[help] duplicate domains remover

Discussion in 'Black Hat SEO Tools' started by banel, Nov 10, 2010.

  1. banel

    banel Regular Member

    Joined:
    Mar 30, 2010
    Messages:
    287
    Likes Received:
    16
    Can anyone suggest me a good software that can remove duplicate domains from a list of urls ?
     
  2. tempnews

    tempnews Newbie

    Joined:
    Mar 8, 2009
    Messages:
    13
    Likes Received:
    6
    If u are with linux type in the shell "sort filename.txt | uniq > newfilename.txt" ..
     
  3. dunhill

    dunhill Power Member

    Joined:
    Nov 15, 2009
    Messages:
    532
    Likes Received:
    117
    Occupation:
    Fulltime IM
    Location:
    Bermuda Triangle
    just go to fiverr .. ask them to code for you.. :D
     
  4. Tighearna

    Tighearna Registered Member

    Joined:
    Apr 21, 2010
    Messages:
    53
    Likes Received:
    19
    Occupation:
    Bokor
    Location:
    739 Bourbon St, New Orleans, LA 70116
    Notepad++ can also do it.... but my money is with scrapebox
     
  5. banel

    banel Regular Member

    Joined:
    Mar 30, 2010
    Messages:
    287
    Likes Received:
    16
    I don't have unix neither scrapebox.
    How can I use notepad to remove duplicate domains? I can't find nothing in the menu.
     
  6. extremephp

    extremephp BANNED BANNED

    Joined:
    Oct 19, 2010
    Messages:
    1,293
    Likes Received:
    1,272
    This Would Help :D

    http://office.microsoft.com/en-us/excel-help/delete-duplicate-rows-from-a-list-in-excel-HA001034626.aspx

    ~ExP~
     
  7. banel

    banel Regular Member

    Joined:
    Mar 30, 2010
    Messages:
    287
    Likes Received:
    16
  8. galus

    galus Registered Member

    Joined:
    Mar 28, 2009
    Messages:
    98
    Likes Received:
    19
    Location:
    Taipe
    Try urlwasher, simple software.
     
  9. banel

    banel Regular Member

    Joined:
    Mar 30, 2010
    Messages:
    287
    Likes Received:
    16
    Hi, Galus, i have urlwasher. It's not good, it can only wash by full domain. The option with "by url" it's not good, i tested with a smaller list.
     
  10. tipstricks

    tipstricks Newbie

    Joined:
    Aug 28, 2010
    Messages:
    1
    Likes Received:
    1
    Remove Duplicate URL ? Free Software
    tipstrickstutorials.com/remove-duplicate-url-free-software/
     
    • Thanks Thanks x 1
  11. thomansfel

    thomansfel Jr. VIP Jr. VIP

    Joined:
    Dec 11, 2009
    Messages:
    535
    Likes Received:
    162
    Occupation:
    working for myself
    Location:
    in a cave with WiFi
    paste your list here: textmechanic.com/Remove-Duplicate-Lines.html and you are done
     
    Last edited: Nov 11, 2010
  12. en.smartis

    en.smartis Jr. VIP Jr. VIP Premium Member

    Joined:
    May 4, 2009
    Messages:
    292
    Likes Received:
    230
    Location:
    127.0.0.1
    pm me a txt and i can remove dups with scrapebox
     
  13. banel

    banel Regular Member

    Joined:
    Mar 30, 2010
    Messages:
    287
    Likes Received:
    16
    Thanks, but as I said in the first post I want to remove the duplicate domains, not duplicate lines.
    Thanks man, this is what i'm looking for.
    Cheers!
     
  14. jaikanth123

    jaikanth123 Newbie

    Joined:
    Jan 25, 2010
    Messages:
    5
    Likes Received:
    0
    Occupation:
    seo analyst
    Location:
    bangalore
    Home Page:
    Just go with fiverr they will surely help you how to remove duplicate domain
     
  15. popzzz

    popzzz Supreme Member

    Joined:
    Apr 12, 2009
    Messages:
    1,337
    Likes Received:
    13,699
    Location:
    Don't touch the REP!
    My choice for this task! :thumb:

    Virus Scan:
    File name: Remove Duplicate URL.exe
    Submission date: 2010-11-12 14:32:15 (UTC)
    Current status: finished
    Result: 0/ 43 (0.0%)
    Code:
    http://www.virustotal.com/file-scan/report.html?id=b5fed102a0d65dd2a0efb3fad0815758f9e30ed2b7425f277267bfa3435327b7-1289572335
    ------------------------------------------------------------
    DISCLAIMER:
    Although the author has courteously provided virus scan information for this post, it does NOT alleviate
    any users from using appropriate virus software on their own computers and using such software prudently.

    Two things are infinite: The universe and stupidity. I'm still not convinced of the universe.
    ------------------------------------------------------------

    Enjoy .... :cool2:

     
  16. groggi42

    groggi42 Newbie

    Joined:
    Nov 13, 2010
    Messages:
    22
    Likes Received:
    4

    Nice tool for newbies like me. Simple and effective. Thank you.