1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Need a "duplicate finder" tool

Discussion in 'BlackHat Lounge' started by Crystical, Aug 28, 2012.

  1. Crystical

    Crystical Registered Member

    Joined:
    Oct 28, 2011
    Messages:
    64
    Likes Received:
    30
    Does anyone know of a tool that can see if there are any words that are on two different web pages?

    For example, let's say an eBay auction has a lot of different items and you know a website that buys items of that sort and has a list of what they're currently looking for. Instead of having to copy an item from eBay and searching the other website for that word/item manually, I'd like some sort of tool that I can use that does that work for me.

    Is there anything out there like this? I was thinking of using Word for this and then searching for duplicated words, but not sure if that's the best way and how to do that.

    Thanks guys
     
    Last edited: Aug 28, 2012
  2. Tensegrity

    Tensegrity Jr. VIP Jr. VIP Premium Member

    Joined:
    Apr 22, 2009
    Messages:
    1,823
    Likes Received:
    968
    Look into Auto Hotkey. It basically will record anything you do as a macro, so you could automate just about anything.

    Also, there is imacros for firefox and chrome, records anything you do in the browser. It has limitations, but it's great for copy and paste stuff.
     
  3. Crystical

    Crystical Registered Member

    Joined:
    Oct 28, 2011
    Messages:
    64
    Likes Received:
    30
    Unfortunately, that won't do. The items I want to compare are cards, so I would need to record myself comparing tens of thousands of different cards before I can automate it :p Comparing web pages or documents for duplicate words would be way easier, but I can't seem to find a tool like that unfortunately.

    Thanks though.
     
  4. Tensegrity

    Tensegrity Jr. VIP Jr. VIP Premium Member

    Joined:
    Apr 22, 2009
    Messages:
    1,823
    Likes Received:
    968
    Can't you just do google search like:
    Code:
    site:ebay.com keyword
    
    Then grab the results and store to a file? You could use PHP cURL for that.

    If you go the route of building a tool for it, then no matter what you use (imacros, auto hotkey, PHP, etc) really just requires a list of what is being compared to be looped through. You could even automate the building of that list. Be creative!
     
  5. evilclown

    evilclown Senior Member

    Joined:
    Jul 31, 2012
    Messages:
    805
    Likes Received:
    575
    Occupation:
    Party Clown
    Location:
    Clownville
    What about copyscape? It can find if there is any other website using the same context.
     
  6. kvmcable

    kvmcable Supreme Member

    Joined:
    Dec 28, 2010
    Messages:
    1,355
    Likes Received:
    2,815
    Occupation:
    24 year business owner - old school dude
    Location:
    KFC - BW3
    Dupefree Pro will get the job done.
     
  7. Crystical

    Crystical Registered Member

    Joined:
    Oct 28, 2011
    Messages:
    64
    Likes Received:
    30
    Thanks guys, but unfortunately those things are not what I'm looking for. Either way thanks for your help. I'll keep searching! Surely there must be some guy who's made a tool like the one I'm searching for.
     
  8. SEOnetic

    SEOnetic Senior Member

    Joined:
    May 21, 2012
    Messages:
    995
    Likes Received:
    120
    Best one imo. Copyscape is way too costly.