1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Well, Shit. Scrapebox failed on me after a decent scrape of 3.9million URLs...

Discussion in 'Black Hat SEO Tools' started by mrmarchuk, Jul 15, 2014.

  1. mrmarchuk

    mrmarchuk Registered Member

    Joined:
    Jun 4, 2012
    Messages:
    65
    Likes Received:
    30
    Occupation:
    Working From Home
    Location:
    Portland, OR
    So I figured while I flew from Portland to North Carolina, I'd have ScrapeBox going strong, scraping a list of 500k queries to help me find as many WordPress websites in a target niche (Jewelry). I stopped it remotely when I got to the airport from my house (which is why you see 215k previously harvested URLs in the screenshot), and saved the URLs and remaining keywords, and then resumed it.

    I landed in North Carolina and remote-connected to my pc again only to find the following error message waiting for me:
    well shit scrapebox fail.png

    after 3.9million URLs scraped, Scrapebox crashed :(

    Granted, probably a good 75% of those are duplicate URLs, but that would have still been a good 1 million URLs of WordPress websites.

    Has anyone else run into this issue or knows of a fix for it? Would the Premium Automator Plugin help me fix this issue by giving me a way to stop-save-start?

    I'm on my way to Montego Bay, but will have my brother at my place and I can call him to back up the scrapes for me, but he's not very dependable when it comes to important stuff (lol), so I might just have to remote-connect in from Jamaica :\
     
  2. Peter Ngo

    Peter Ngo Jr. VIP Jr. VIP

    Joined:
    Apr 23, 2013
    Messages:
    1,837
    Likes Received:
    1,463
    Occupation:
    I browse BHW for a living
    Location:
    The Internet
    It crashed all the time, but no big deal, go to your scrapebox folder --> harvester session, it is all there.
     
    • Thanks Thanks x 1
  3. Alpha.

    Alpha. Power Member

    Joined:
    May 11, 2013
    Messages:
    505
    Likes Received:
    236
    Location:
    Europe... No Borders? No Worries!
    This.

    I had the same problem when I first got scrapebox.. and after a while found out that scrapebox actually saves your progress! Needless to say, if you've been scraping a while.. you now have a gazillion links!
     
    • Thanks Thanks x 1
  4. Aty

    Aty Jr. VIP Jr. VIP

    Joined:
    Jan 27, 2011
    Messages:
    5,407
    Likes Received:
    3,698
    Home Page:
    Like said above, it is saved in your Scrapebox folder. Problem is you won't be able to open a 3.9 mil url file with SB. I use Free Mailing List Splitter to split it to 1 mil url files.
     
    • Thanks Thanks x 1
  5. BacklinksAce

    BacklinksAce Registered Member

    Joined:
    Jul 8, 2014
    Messages:
    78
    Likes Received:
    11
    Location:
    UK
    Home Page:
    Or switch to Gscraper and have no issues with the size of your scrapes again ;)
     
  6. SerpEvil

    SerpEvil Supreme Member

    Joined:
    Jan 1, 2012
    Messages:
    1,495
    Likes Received:
    446
    Gender:
    Male
    Occupation:
    WebMaster
    Location:
    Top 3 On Google
    gscraper is the best tool for scraping these days and never crashes
     
  7. loopline

    loopline Jr. VIP Jr. VIP

    Joined:
    Jan 25, 2009
    Messages:
    3,377
    Likes Received:
    1,800
    Gender:
    Male
    Home Page:
    Scrapebox V 2.0 coming out soon has no limit in the harvester grid for opening lists. Besides the current version segments the 3.9 million urls into 1 million chunks so you can open them and you can currently use the dupe remove addon to work with lists up to 180 million lines.

    But once 2.0 is out, its 64bit so there will be virtually no limit on the harvester grid.
     
    • Thanks Thanks x 7
  8. kingpronew

    kingpronew Regular Member

    Joined:
    Mar 28, 2013
    Messages:
    369
    Likes Received:
    39
    Occupation:
    CEO
    Location:
    www.luckygiftcard.com
    Home Page:
    This scapebox suks, its flash fast comment just draw away the cmt in the trast, i quitted this tool for long time and just use nohandseo right now.
     
    • Thanks Thanks x 1
  9. mrmarchuk

    mrmarchuk Registered Member

    Joined:
    Jun 4, 2012
    Messages:
    65
    Likes Received:
    30
    Occupation:
    Working From Home
    Location:
    Portland, OR
    Good thing I increased the size of the harvester folder ^_^ I didn't know it saves the stuff in there. Thanks guys!

    Also, I'll be sure to check out GScraper when I have a minute, meanwihle, here's a teaser of where I'm at :p (I'll be sure toupload more pics after the vacation is over ^_^ )

    edit:Yeah I know my hair is all weird. Story of my life.

    [​IMG]
    [​IMG]
     

    Attached Files:

  10. mig99

    mig99 Newbie

    Joined:
    Jun 18, 2014
    Messages:
    33
    Likes Received:
    6
    breathtaking view, ima set it as wallpaper lol
     
    • Thanks Thanks x 2
  11. mrmarchuk

    mrmarchuk Registered Member

    Joined:
    Jun 4, 2012
    Messages:
    65
    Likes Received:
    30
    Occupation:
    Working From Home
    Location:
    Portland, OR
    I've got a few better ones, but the internet is incredibly slow here, so it'll have to wait till I get back :)
     
  12. loopline

    loopline Jr. VIP Jr. VIP

    Joined:
    Jan 25, 2009
    Messages:
    3,377
    Likes Received:
    1,800
    Gender:
    Male
    Home Page:
    You like his hair that much? HAHA
     
    • Thanks Thanks x 1
  13. ranjim

    ranjim Regular Member

    Joined:
    Apr 5, 2012
    Messages:
    386
    Likes Received:
    46
    Will 2.0 be an update or a new product to buy?
     
  14. loopline

    loopline Jr. VIP Jr. VIP

    Joined:
    Jan 25, 2009
    Messages:
    3,377
    Likes Received:
    1,800
    Gender:
    Male
    Home Page:
    2.0 is an update. I think if they were going to charge for it they would call it a different product name all together. Its just an update in the lifetime license line of the product "scrapebox"

    They like to make money but they aren't greedy. Scrapebox is the cheapest "seo" program out there for anything in its class, and they could charge for all sorts of things along the way but don't.

    Anyway, to the very best of my knowledge its a free update.
     
    • Thanks Thanks x 1
  15. mrmarchuk

    mrmarchuk Registered Member

    Joined:
    Jun 4, 2012
    Messages:
    65
    Likes Received:
    30
    Occupation:
    Working From Home
    Location:
    Portland, OR
    Totally possible, but I could see it as a "premium" upgrade as well. As much as I'd love it to be a free upgrade, I'd be totally cool with paying an extra $20 to have the ability to scrape and work with millions of URLs - I'm constantly building site lists for myself so that when I need em, they're ready for me, and my current scrape of 5million URLs is still going strong, I'm not going to stop it until it crashes, but I'd love to be able to stop it, remove duplicates, and then continue scraping.

    lmao, that was on our first day. MY hair is crap like that, goes all over the place (probably because I spend too much time in my office, and when I step outside, it's like "FRESH AIR!").

    Here's a better pic, this time not using my crappy phone camera and with my hair in order
    [​IMG]

    and a few panoramas like the one below on my lounge thread (I'll share em in full resolution when I get home, because the internet here is so slow it makes me want to throw my X1 Carbon in the ocean xD).
    [​IMG]
     
    Last edited: Jul 20, 2014
  16. mig99

    mig99 Newbie

    Joined:
    Jun 18, 2014
    Messages:
    33
    Likes Received:
    6
    lol i was taking the one with the beach view
     
  17. kingpronew

    kingpronew Regular Member

    Joined:
    Mar 28, 2013
    Messages:
    369
    Likes Received:
    39
    Occupation:
    CEO
    Location:
    www.luckygiftcard.com
    Home Page:
    sorry to hear about that, are you using a valid lisense scabebox, or maybe your lap has not enough RAM.
     
  18. loopline

    loopline Jr. VIP Jr. VIP

    Joined:
    Jan 25, 2009
    Messages:
    3,377
    Likes Received:
    1,800
    Gender:
    Male
    Home Page:
    That is a pretty awesome panoramic shot! Place looks amazing, your living like the NR in the 4 hour work week. After my kids are not in school any longer, I may take up residence around the globe at places like that. Enjoy!
     
  19. mrmarchuk

    mrmarchuk Registered Member

    Joined:
    Jun 4, 2012
    Messages:
    65
    Likes Received:
    30
    Occupation:
    Working From Home
    Location:
    Portland, OR
    License is legit, laptop has 6cores and 8gb ram. The issue is with the software being 32bit. Hopefully the 64bit update fixes this in the future
     
  20. K.H.R

    K.H.R Jr. VIP Jr. VIP Premium Member

    Joined:
    Dec 9, 2013
    Messages:
    2,931
    Likes Received:
    953
    Gender:
    Male
    Occupation:
    Study + Outsourcing
    Location:
    BHW
    Scrapebox Is a Grate tool but many people faced this problem.