Tool to split huge text file into X number of lines?

ScreamingMidget said:

I have Scrapebox, but it seems to take too long and requires too much memory.

The text file has over 45 million lines.

I want to split this file into 45 separate files (1 million each).

Are there any other tools that can do this? I'd also like a tool that can randomize the lines in the text file.

Click to expand...

dragonguy4 said:

so what you can do is rename your txt file to csv first, then open in excel, create new column, fill in with RANDBETWEEN formula, drag the value all the way to bottom, then sort this randomized value

then save. after that, you can split using csv splitter

this is what i am using: https://sourceforge.net/projects/splitcsv/files/latest/download

Click to expand...

ScreamingMidget said:

Unfortunately, when I tried to open the CSV file, Excel just give up - probably a memory issue.

I guess I'll just stick with Scrapebox and wait patiently.

Click to expand...

ScreamingMidget said:

Unfortunately, when I tried to open the CSV file, Excel just give up - probably a memory issue.

I guess I'll just stick with Scrapebox and wait patiently.

Click to expand...

Sweetfunny said:

Have you tried using the Text File Tool in ScrapeBox rather than the Export & Split on the search engine harvester? It should work better with large files, you can access it under Tools > Text File Tool (dups, sort, split etc)

Click to expand...

ScreamingMidget said:

Hi, yeah I've tried that too, which is a bit quicker. It's the randomisation bit that literally takes around 12 hours. I've started doing it overnight, so not too bad.

Click to expand...