help needed: combine csv's + delete duplicates

Danman123

Regular Member

Jul 23, 2016

210

37

• Mar 27, 2017

• #1

Does anyone know a tool that can do as described in the title? I have a few csv's with over 1.5 million email adresses. But i want to clean them up a bit.

 

sysco32

Power Member

Feb 5, 2014

708

287

• Mar 27, 2017

• #2

Danman123 said:

Does anyone know a tool that can do as described in the title? I have a few csv's with over 1.5 million email adresses. But i want to clean them up a bit.

Click to expand...

Try the program csvsplitter.I used it before and it is good.

 

rafark

Elite Member

Jan 15, 2013

3,249

1,979

• Mar 27, 2017

• #3

If you're a programmer you can export it to a local sql database (using LOAD DATA INFILE) do your stuff in SQL and then export it. That's what I did just a couple a weeks ago for a client. Very handy, if you know SQL.

 

H

hackedd

Junior Member

Aug 11, 2010

147

178

• Mar 27, 2017

• #4

I can do it for you for a couple of bucks. Let me know if interested...

 

D

deal with it

Senior Member

Sep 30, 2015

1,086

707

• Mar 27, 2017

• #5

use ABLEBITS

 

Danman123

Regular Member

Jul 23, 2016

210

37

• Mar 27, 2017
• Thread Starter Thread Starter

• #6

deal with it said:

use ABLEBITS

Click to expand...

thanks!

 

uncutu

Elite Member

Aug 6, 2010

3,506

3,376

• Mar 27, 2017

• #7

hire va to do it for you or try manually with notepad++ with textFX plugin to remove dupes and Merge plugin to merge multi files into 1 file.

 

mainceaft

Senior Member

Apr 10, 2013

881

184

• Mar 27, 2017

• #8

1 million Is huge amount of lines it's very hard to doing it manually . I can develop PHP secript do the Job , id you hire me I will give your the source code of the script so you can us it lather .