1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Duplicate pages (not urls) in craigslist scrape

Discussion in 'Black Hat SEO' started by drifterzz, Aug 14, 2015.

  1. drifterzz

    drifterzz Newbie

    Joined:
    Feb 21, 2014
    Messages:
    1
    Likes Received:
    0
    I've scraped craigslist to email potential clients to my directory, but I find people have several pages with the same content in different cities. How can I take my list of craigslist urls and delete the duplicate content urls so I'm not sending these people multiple emails?