1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

--> [Help!] How to Handle Huge CSV files?

Discussion in 'BlackHat Lounge' started by ElectroMan, Aug 3, 2010.

  1. ElectroMan

    ElectroMan Regular Member

    Joined:
    Dec 13, 2009
    Messages:
    254
    Likes Received:
    764
    Hey guys, I've got some huge csv files with about 250k rows. I need to simply take one column and copy/paste it into a text (notepad) file so I can load it for a bot.

    Problem is that in excel my numbers in that column become 1Exxx because they are so large.

    I tried open office but there is a limit of 65k rows. I split the files to be 50k rows each but it was taking forever to copy/paste each 50k file.

    I even tried CSVed but I couldn't figure out how to export one column in the format that I need.

    If I open the CSV file in notepad the column I need has "" quotations and I have a few million entries.

    Help!
     
  2. ElectroMan

    ElectroMan Regular Member

    Joined:
    Dec 13, 2009
    Messages:
    254
    Likes Received:
    764
    I'm not using it for this.

     
  3. corbin2

    corbin2 Registered Member

    Joined:
    Jan 16, 2010
    Messages:
    66
    Likes Received:
    41
    What you need is a CSV Splitter, a tool that divides large csv files into smaller ones.

    I use this one: hxxp://www.fxfisherman.com/forums/forex-metatrader/tools-utilities/75-csv-splitter-divide-large-csv-files.html#post727
     
    • Thanks Thanks x 1
  4. jaerehan

    jaerehan Elite Member

    Joined:
    Sep 25, 2009
    Messages:
    1,500
    Likes Received:
    1,504
    Occupation:
    Time...Time is on my side..
    Location:
    EU
    Problem solved lol
     
    • Thanks Thanks x 1