1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Whats the best free tool to remove duplicate images?

Discussion in 'Instagram' started by nickcruz, Jun 21, 2016.

  1. nickcruz

    nickcruz Junior Member

    Joined:
    Mar 10, 2016
    Messages:
    113
    Likes Received:
    23
    What is the best tool to remove duplicate images from a folder? I have a folder with 10k images I've downloaded but I can see there are duplicates, any easy way to delete them without having to do it one by one?
     
    • Thanks Thanks x 1
  2. HoNeYBiRD

    HoNeYBiRD Jr. VIP Jr. VIP

    Joined:
    May 1, 2009
    Messages:
    7,315
    Likes Received:
    8,281
    Gender:
    Male
    Occupation:
    Geographer, Tourism Manager
    Location:
    Ghosted
  3. KenTZ

    KenTZ Regular Member

    Joined:
    Jan 28, 2015
    Messages:
    418
    Likes Received:
    223
    • Thanks Thanks x 1
  4. nickcruz

    nickcruz Junior Member

    Joined:
    Mar 10, 2016
    Messages:
    113
    Likes Received:
    23
    Thats great thanks guys
     
  5. KenTZ

    KenTZ Regular Member

    Joined:
    Jan 28, 2015
    Messages:
    418
    Likes Received:
    223
    You're welcome.

    Not sure if you're interested in a method for scraping images for IG, but if you are, take a look at this.
     
  6. nickcruz

    nickcruz Junior Member

    Joined:
    Mar 10, 2016
    Messages:
    113
    Likes Received:
    23
    I read that thread before. Thats how I knew there was a tool for removing duplicate content. I looked for it in the instagram section but I couldn't find it lol , now I see it was in the making money section. Thanks again
     
  7. dzyothwrld

    dzyothwrld Registered Member

    Joined:
    Apr 14, 2016
    Messages:
    86
    Likes Received:
    3
    With AVG Tune-Up you can search for duplicate files and easily delete them.
     
  8. HoNeYBiRD

    HoNeYBiRD Jr. VIP Jr. VIP

    Joined:
    May 1, 2009
    Messages:
    7,315
    Likes Received:
    8,281
    Gender:
    Male
    Occupation:
    Geographer, Tourism Manager
    Location:
    Ghosted
    I use a duplicate finder which is specialized for pics, because a general duplicate remover doesn't remove all duplicate pics (i think those identify by MD5 hash).
    Oftentimes the same pics can be found in several different resolutions, VisiPics can identify all the different resolutions of the same pic, not just the 1:1 copies and effectively removes them.
     
  9. nickcruz

    nickcruz Junior Member

    Joined:
    Mar 10, 2016
    Messages:
    113
    Likes Received:
    23
    That visipics worked great and was really fast too 10k images in 5-7 mins. Does anyone know of a tool that removes watermarks or text from images?
     
  10. BenChevy

    BenChevy Junior Member

    Joined:
    Jul 24, 2013
    Messages:
    183
    Likes Received:
    8
    Any know how to remove duplicate videos? Would be rather difficult wouldn't it?
     
  11. Baros

    Baros Junior Member

    Joined:
    Jan 30, 2011
    Messages:
    198
    Likes Received:
    36
    Image watermarks are hard. Been there done that. instead of trying, depending on where the mark is, you might just cut it out and place a solid color with a brand or something. Or blur each. On a mass scale, sounds rough tho.
     
  12. HoNeYBiRD

    HoNeYBiRD Jr. VIP Jr. VIP

    Joined:
    May 1, 2009
    Messages:
    7,315
    Likes Received:
    8,281
    Gender:
    Male
    Occupation:
    Geographer, Tourism Manager
    Location:
    Ghosted
    I'd say any general duplicate remover could work, which uses MD5 hash and/or exact file size to identify duplicates, but those only work for 1:1 exact copies. If any of the dupes are longer or shorter just even by a millisecond for instance, the MD5 hash will be different, so these tools won't be able to find those duplicates.

    I'm not sure if anything like VisiPics exists for videos, it would require huge processing power to analyze videos that way.