Remove Duplicates 2.5M+ LINES MORE! (HELP)! NOTEBOOK ++

[code API]

Yes, Scrapebox can do this I think.

In fact in super large quantities than OP's expectations.

 

[loopline]

In fact in super large quantities than OP's expectations.

Yes the dupe remove addon in scrapebox caps at around 180 million lines, and thats a windows grid limit not a scrapebox limit. 2.5 million is easy, you can do that right in the main grid in scrapebox in the upper right hand quadrant of the main scrapebox window. Probably wouldn't do more then 20 million in that grid, if you need more then 20 million Id probably use the addon, its faster at that point.

 

[DailyVlogerz]

Use the swiss army knife aka Scrapebox. Install dedupe addon! job done!