1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Best way to split huge txt file for Scrapebox (over 10 million lines)

Discussion in 'Black Hat SEO Tools' started by muchacho, May 24, 2011.

  1. muchacho

    muchacho Supreme Member

    Joined:
    May 14, 2009
    Messages:
    1,293
    Likes Received:
    187
    Location:
    Lancashire, England.
    I've kindly been given a huge AA list which has over 10 million URLs.

    The problem is, I've tried using the free software 'File Splitter' and 'Text Splitter' but both come back as 'out of memory' error messages.

    This is on a PC with 4 gb of RAM.

    Trying to import into Scrapebox crashed it due to memory as well.

    Is there anyway around this? Ideally I'd like to split it all up into chunks of about 500k txt files to run individually.
     
  2. akashasnani

    akashasnani Regular Member

    Joined:
    Mar 30, 2011
    Messages:
    226
    Likes Received:
    111
    Home Page:
    i aslo facing same problem..!!
    my pc has only 2GB ram..!! :(
     
  3. remmaps

    remmaps Regular Member

    Joined:
    Apr 13, 2010
    Messages:
    212
    Likes Received:
    123
    Home Page:
    try this one gdgsoft.com/gsplit
     
  4. 1337WuLF

    1337WuLF Senior Member Premium Member

    Joined:
    Jan 3, 2011
    Messages:
    919
    Likes Received:
    370
    Occupation:
    IM
    Location:
    On a rock in space
    I got 8GB of ram, I can give it a try for you if you like.
     
  5. bk071

    bk071 Jr. Executive VIP Jr. VIP Premium Member

    Joined:
    Nov 24, 2010
    Messages:
    3,104
    Likes Received:
    7,914
    Occupation:
    I don't have a job
    Location:
    .............
    There's a feature in WinUtilities called 'file splitter and joiner' you may want to give it a try.

    I was able to split a 14 mil file on a PC w/ 1 GB of ram :)

    Google "winutilities"

    Cheers.
     
  6. muchacho

    muchacho Supreme Member

    Joined:
    May 14, 2009
    Messages:
    1,293
    Likes Received:
    187
    Location:
    Lancashire, England.
    Does that split .txt files into smaller .txt files, or is it just a tool to split and restore if you were just transporting them from A to B?

    I'm wanting the output to also be .txt just fewer amount of entries.
     
  7. crazyflx

    crazyflx Elite Member

    Joined:
    Nov 9, 2009
    Messages:
    1,674
    Likes Received:
    4,825
    Location:
    http://CRAZYFLX.COM
    Home Page:
    Sure, use this:

    Code:
    http://www.multiupload.com/N6MYZTPBLE
    It is a log file splitter, that is meant for splitting gigabyte sized text files, even with low PC resources.

    VirusTotal:
    File name:LogSplitter.hta
    Submission date:2010-10-30 16:36:47 (UTC)
    Current status:finished
    Result:1 /43 (2.3%)

    VirusTotal Link:
    http://www.virustotal.com/file-scan...15f2afccbb3e6b8149d7b06bfb9d0112fa-1288456607

    McAfee reports as having this: Heuristic.BehavesLike.HTML.Suspicious.E but I can assure you, I use that program ALL the time, and it is definitely not a virus.

    If when you go to use it, you get this error message after a couple of seconds:

    "A script on this page is causing Internet Explorer to run slowly" then download this fix from Microsoft: http://support.microsoft.com/kb/175500
     
    • Thanks Thanks x 4
  8. imth3one

    imth3one Newbie

    Joined:
    Nov 5, 2008
    Messages:
    47
    Likes Received:
    10
    Search for listmotor. It was made for emails but works great to play around with scrapebox lists
     
  9. muchacho

    muchacho Supreme Member

    Joined:
    May 14, 2009
    Messages:
    1,293
    Likes Received:
    187
    Location:
    Lancashire, England.

    That worked very well cheers, you're a star.

    It kept asking if I wished to cancel the script as it was slowing the browser, but 30 clicks of 'no' and it did the job.

    Thanks again. A very good tool.
     
  10. putrasandy

    putrasandy Regular Member

    Joined:
    Sep 26, 2010
    Messages:
    219
    Likes Received:
    11
    i dont get any error messages.. it work well.. thanks
     
  11. crazyflx

    crazyflx Elite Member

    Joined:
    Nov 9, 2009
    Messages:
    1,674
    Likes Received:
    4,825
    Location:
    http://CRAZYFLX.COM
    Home Page:
    That's the error I was talking about. The link to microsoft and the download on the page will take care of that error for you. You'll never have to click no again, it'll just work away in the background and inform you when it's finished.

    and no problem :)
     
  12. trail29

    trail29 Regular Member

    Joined:
    Oct 14, 2009
    Messages:
    274
    Likes Received:
    63
    try this one buddy. thats what im using.
    Code:
    http://www.hjsplit.org/
     
  13. mirrorer

    mirrorer Jr. VIP Jr. VIP

    Joined:
    Jan 30, 2009
    Messages:
    1,164
    Likes Received:
    1,026
  14. eLeSlash

    eLeSlash Regular Member

    Joined:
    Feb 9, 2011
    Messages:
    459
    Likes Received:
    70
    Occupation:
    Freelancer
    Location:
    Romania
    Home Page:
    I am searching for someone to talk to on yahoo messenger or skype about scrapebox. Help me lil bit :) i need some guidance from some experienced scrapebox users. I got some questions i want to ask. If someone is available please pm me thanx.
     
  15. trail29

    trail29 Regular Member

    Joined:
    Oct 14, 2009
    Messages:
    274
    Likes Received:
    63
    this one will surely help you;
    http://www.blackhatworld.com/blackhat-seo/buy-sell-trade/129096-scrapebox-ultimate-serp-scraper-auto-blog-commenter-prstorm-mode.html

    this one also;

    http://www.blackhatworld.com/blackh...box-tutorial-all-one-place-just-bookmark.html

    hope it helps.