HELP! Need to split 3GB CSV file into smaller files.

Discussion in 'BlackHat Lounge' started by CoyoteAssassin, Oct 30, 2012.

  1. CoyoteAssassin

    CoyoteAssassin Elite Member

    Joined:
    Jan 3, 2010
    Messages:
    1,862
    Likes Received:
    3,936
    Occupation:
    Full Time IMer
    Location:
    USA
    I have a 3GB CSV file that I need to split into 200MB files. I've tried CSV splitter and a few others but the 3GB file is too large.

    Does anyone have a solution?

    I ran a scrape job and the results ended up being much higher than I anticipated and no I can't open the file. :p

    Thanks.

    -CA
     
  2. qrazy

    qrazy Senior Member

    Joined:
    Mar 19, 2012
    Messages:
    1,115
    Likes Received:
    1,726
    Location:
    Banana Republic
  3. CoyoteAssassin

    CoyoteAssassin Elite Member

    Joined:
    Jan 3, 2010
    Messages:
    1,862
    Likes Received:
    3,936
    Occupation:
    Full Time IMer
    Location:
    USA
    That is what I usually use but it will not work this time. It says that the split was complete but it only generates a 73KB file and does nothing else.

    Thanks though.
     
  4. me100rabh

    me100rabh Regular Member

    Joined:
    Jun 27, 2009
    Messages:
    374
    Likes Received:
    881
    Tried Hjsplit ? I guess it can do this job easily.....
     
    • Thanks Thanks x 1
  5. qrazy

    qrazy Senior Member

    Joined:
    Mar 19, 2012
    Messages:
    1,115
    Likes Received:
    1,726
    Location:
    Banana Republic
    I usually use this to split large files of above 2GB. I suspect the program expects the format which isn't suited for this particular file.
     
    • Thanks Thanks x 1
  6. CoyoteAssassin

    CoyoteAssassin Elite Member

    Joined:
    Jan 3, 2010
    Messages:
    1,862
    Likes Received:
    3,936
    Occupation:
    Full Time IMer
    Location:
    USA
    While looking for this I found GSplit so I'm running it. I'll post results.

    From what I can tell Hjsplit will not split CSV into readable files. I do not need to upload or send these. I need each file to be in a format I can open and edit.

    ---

    The CSV file is in the same format as always.


    UPDATE: GSplit worked like a charm. If you every run into this problem, that is the program to go with. Thanks all.