Need a tool to clean large url list. Compare on domain level.

I want to clean large amount of list.
Since scrapebox dupremove addon can remove dup on large list.
But dupremove can't compare on url or compare on domain level.
Is there a tool can compare on very large list?

Depends on what you call large lists.
Scrapebox can compare lists with 1 millions against other list with more than 1 million. It just up to your PC power.

Import lists of 1 millions urls, then use 'Import URL List - Select the URL List to Compare on Domain Level'.

yeah thanks your reply. I asked on scrapeboxforum loopline said scrapebox 2.0 is coming
But now, i just can separate whole list to 1 million each before compare.