Remove Duplicates 2.5M+ LINES MORE! (HELP)! NOTEBOOK ++

Permat said:

Yes, Scrapebox can do this I think.

Click to expand...

In fact in super large quantities than OP's expectations.

speedie said:

In fact in super large quantities than OP's expectations.

Click to expand...

Yes the dupe remove addon in scrapebox caps at around 180 million lines, and thats a windows grid limit not a scrapebox limit. 2.5 million is easy, you can do that right in the main grid in scrapebox in the upper right hand quadrant of the main scrapebox window. Probably wouldn't do more then 20 million in that grid, if you need more then 20 million Id probably use the addon, its faster at that point.

Use the swiss army knife aka Scrapebox. Install dedupe addon! job done!