1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Remove Duplicates Tool

Discussion in 'Black Hat SEO' started by facebookdude, Jan 29, 2011.

  1. facebookdude

    facebookdude Elite Member

    Joined:
    Feb 28, 2010
    Messages:
    1,506
    Likes Received:
    2,490
    Hey guys. I have 2 email lists and I want to remove the duplicates. Is there any tool I can use to do this?
     
  2. cyberzilla

    cyberzilla Elite Member Premium Member

    Joined:
    Nov 15, 2009
    Messages:
    2,204
    Likes Received:
    3,364
    Location:
    zeta reticuli
    Try this
    Code:
    http://www.brothersoft.com/remove-%28delete%29-duplicate-email-addresse-126183.html
    or else you can do that manually using excel. Here is the step-by-step instruction

    Code:
    http://www.digdb.com/excel_add_ins/duplicates_find_remove_dedupe/
     
    Last edited: Jan 29, 2011
  3. kkvsam

    kkvsam Senior Member

    Joined:
    Oct 11, 2009
    Messages:
    936
    Likes Received:
    569
    Occupation:
    SYS ADMIN
    Home Page:
    Try this . works perfectly...
    Code:
    http://www.mediafire.com/?q9ui54ebabsw1uz
    VT
    Code:
    http://www.virustotal.com/file-scan/reanalysis.html?id=6932267da6141f3fd306c7f695652589828fba9882628d188c3dd298943b4e04-1296312178
     
    Last edited: Jan 29, 2011
  4. cyberzilla

    cyberzilla Elite Member Premium Member

    Joined:
    Nov 15, 2009
    Messages:
    2,204
    Likes Received:
    3,364
    Location:
    zeta reticuli
    @kkvsam Please add VT scan if it is an exe file:)
     
    • Thanks Thanks x 1
  5. kkvsam

    kkvsam Senior Member

    Joined:
    Oct 11, 2009
    Messages:
    936
    Likes Received:
    569
    Occupation:
    SYS ADMIN
    Home Page:
    Thanks uploading!!!!!:D
    done...
     
    • Thanks Thanks x 1
    Last edited: Jan 29, 2011
  6. f0rked

    f0rked Registered Member

    Joined:
    Jan 25, 2009
    Messages:
    71
    Likes Received:
    64
    Code:
    #!/usr/bin/perl
    
    use strict;
    use warnings;
    
    my $file = 'email.txt';
    my %seen = ();
    {
       local @ARGV = ($file);
       local $^I = '.bac';
       while(<>){
          $seen{$_}++;
          next if $seen{$_} > 1;
          print;
       }
    }
    print "finished processing file.";
    Simple, clean, and fast.
     
    • Thanks Thanks x 1
  7. facebookdude

    facebookdude Elite Member

    Joined:
    Feb 28, 2010
    Messages:
    1,506
    Likes Received:
    2,490
    How do I use this man? :D I'm baked and cant figure it haha
     
  8. mr285

    mr285 Newbie

    Joined:
    Jul 21, 2010
    Messages:
    16
    Likes Received:
    4
    This is what i've been using. So far so good:
    textop[dot]us/Lines-tools/Delete-Duplicate-Lines

    Obviously replace [dot] with "."
     
  9. f0rked

    f0rked Registered Member

    Joined:
    Jan 25, 2009
    Messages:
    71
    Likes Received:
    64
    Make a new file and name it whatever you'd like, say removedupes.pl, download ActivePerl(google it), and place your list(name it emails.txt) in the same location as the removedupes.pl file, and run removedupes.pl. That simple. Your file with dupes removed will still be labeled emails.txt, and you'll have a new file named emails.bac, which contains the old list before the dupes were removed.

     
    • Thanks Thanks x 1
    Last edited: Jan 29, 2011
  10. kkvsam

    kkvsam Senior Member

    Joined:
    Oct 11, 2009
    Messages:
    936
    Likes Received:
    569
    Occupation:
    SYS ADMIN
    Home Page:
    extract rar file and run the exe file. then click "email list" tab and import emails.then click the "clean list" button. If you want to compare another email list, click the tab of "Remove list" and import that list to it. Then click "email list" tab and then click "clean list" button .

    Very easy...:D
     
  11. BH_Manic

    BH_Manic Junior Member

    Joined:
    Aug 28, 2009
    Messages:
    158
    Likes Received:
    711
    Occupation:
    Student, IMer
    Location:
    India
    I guess SB can do. Just copy-paste that list to keyword scrapper and remove duplicates.
     
  12. ericsson

    ericsson Elite Member Premium Member

    Joined:
    Apr 25, 2009
    Messages:
    2,642
    Likes Received:
    8,132
    Occupation:
    www
    Location:
    Swe
    Home Page:
    Use this man.


    Send-Safe List Manager ;)
     
  13. soulless1

    soulless1 Junior Member

    Joined:
    Aug 20, 2009
    Messages:
    106
    Likes Received:
    21
    Occupation:
    Marketing
    Location:
    USA
    I have used this skymem(Dot)com