I haven't used it for a few months and just updated to the most recent version. I don't remember having this problem before, however now everytime I try to scrape a reasonably large list of urls from the SE's, it crashes, and frequently. Sometimes I can get ~30k urls scraped before it crashes, other times ~500k. That might sound like a lot but typically they're only ~10% unique domains and the point is that I'm not able to finish scraping my keyword list.
Any suggestions as to what might be causing this?
Any suggestions as to what might be causing this?