1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Need a tool to clean large url list. Compare on domain level.

Discussion in 'Black Hat SEO' started by blackhatnewer, Jun 19, 2014.

  1. blackhatnewer

    blackhatnewer Newbie

    Joined:
    May 1, 2014
    Messages:
    7
    Likes Received:
    2
    I want to clean large amount of list.
    Since scrapebox dupremove addon can remove dup on large list.
    But dupremove can't compare on url or compare on domain level.
    Is there a tool can compare on very large list?
     
  2. mindmaster

    mindmaster Jr. VIP Jr. VIP Premium Member

    Joined:
    Sep 16, 2010
    Messages:
    2,501
    Likes Received:
    1,135
    Location:
    at my new office
    Depends on what you call large lists.
    Scrapebox can compare lists with 1 millions against other list with more than 1 million. It just up to your PC power.

    Import lists of 1 millions urls, then use 'Import URL List - Select the URL List to Compare on Domain Level'.
     
  3. blackhatnewer

    blackhatnewer Newbie

    Joined:
    May 1, 2014
    Messages:
    7
    Likes Received:
    2
    yeah thanks your reply. I asked on scrapeboxforum loopline said scrapebox 2.0 is coming :)
    But now, i just can separate whole list to 1 million each before compare.