1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

HELP! Need to split 3GB CSV file into smaller files.

Discussion in 'BlackHat Lounge' started by CoyoteAssassin, Oct 30, 2012.

  1. CoyoteAssassin

    CoyoteAssassin Elite Member

    Joined:
    Jan 3, 2010
    Messages:
    1,862
    Likes Received:
    3,906
    Occupation:
    Full Time IMer
    Location:
    USA
    I have a 3GB CSV file that I need to split into 200MB files. I've tried CSV splitter and a few others but the 3GB file is too large.

    Does anyone have a solution?

    I ran a scrape job and the results ended up being much higher than I anticipated and no I can't open the file. :p

    Thanks.

    -CA
     
  2. qrazy

    qrazy Senior Member

    Joined:
    Mar 19, 2012
    Messages:
    1,113
    Likes Received:
    1,712
    Location:
    Banana Republic
  3. CoyoteAssassin

    CoyoteAssassin Elite Member

    Joined:
    Jan 3, 2010
    Messages:
    1,862
    Likes Received:
    3,906
    Occupation:
    Full Time IMer
    Location:
    USA
    That is what I usually use but it will not work this time. It says that the split was complete but it only generates a 73KB file and does nothing else.

    Thanks though.
     
  4. me100rabh

    me100rabh Jr. VIP Jr. VIP Premium Member

    Joined:
    Jun 27, 2009
    Messages:
    372
    Likes Received:
    878
    Tried Hjsplit ? I guess it can do this job easily.....
     
    • Thanks Thanks x 1
  5. qrazy

    qrazy Senior Member

    Joined:
    Mar 19, 2012
    Messages:
    1,113
    Likes Received:
    1,712
    Location:
    Banana Republic
    I usually use this to split large files of above 2GB. I suspect the program expects the format which isn't suited for this particular file.
     
    • Thanks Thanks x 1
  6. CoyoteAssassin

    CoyoteAssassin Elite Member

    Joined:
    Jan 3, 2010
    Messages:
    1,862
    Likes Received:
    3,906
    Occupation:
    Full Time IMer
    Location:
    USA
    While looking for this I found GSplit so I'm running it. I'll post results.

    From what I can tell Hjsplit will not split CSV into readable files. I do not need to upload or send these. I need each file to be in a format I can open and edit.

    ---

    The CSV file is in the same format as always.


    UPDATE: GSplit worked like a charm. If you every run into this problem, that is the program to go with. Thanks all.