1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

[HELP]How to remove duplicate text with comma seperated

Discussion in 'BlackHat Lounge' started by imguy, Aug 18, 2012.

  1. imguy

    imguy BANNED BANNED

    Joined:
    Jun 25, 2011
    Messages:
    182
    Likes Received:
    191
    Hi,

    How to remove duplicate text seperated by comma? anybody know any site or software for this purpose?

    Example:
    Before - AE,AE,DE,BE,CE,CE,FE,GE,GE

    After - AE,DE,BE,CE,FE,GE

    P.S: I am sorry if I am posting this thread on wrong section, please MOD move this thread in appropriate section.

    Thanks.
     
  2. HoNeYBiRD

    HoNeYBiRD Jr. VIP Jr. VIP

    Joined:
    May 1, 2009
    Messages:
    5,912
    Likes Received:
    7,150
    Gender:
    Male
    Occupation:
    Geographer, Tourism Manager
    Location:
    Ghosted
    Use Notepad++

    Navigate to Search/Replace or Hit Ctrl+H
    Set "Search Mode" to "Extended"
    Find what:, (comma without space)
    Replace with:\n
    Hit "Replace All"

    This will remove the commas and put the content before each comma into a new line.

    Navigate to TextFX/TextFX Tools
    Make sure "+Sort outputs only UNIQUE (at column) lines" is checked
    Then "Sort lines case sensitive at column"

    This will remove all duplicate lines.

    If you want the result to be comma separated, then do the reverse what you did at first:
    Navigate to Search/Replace or Hit Ctrl+H
    "Search Mode" was set to "Extended"
    Find what:\n
    Replace with:, (comma without space)
    Hit "Replace All"
     
    • Thanks Thanks x 1
  3. imguy

    imguy BANNED BANNED

    Joined:
    Jun 25, 2011
    Messages:
    182
    Likes Received:
    191
    WOW! Rep+:)