Screaming Frog for Scraping

Hey guys,

has anyone used screaming frog to look for expired domains? It's a really cool tool too check for 404 errors, however I keep running out of memory even though I made lots of tweaks. I'm on a 8 gm ram computer.

Anyway, I can look into that, just wondering if anybody here had any success with looking for expired domains? getting massive seed lists and so on.

Thanks

DJ

honestly, I don't know much about Screaming frog but I can help you with the tools I use:

either

in Netpeak Checker– select parameters (1) On-Page –Status Code and (2) Whois– Expiration Date

or

in Netpeak Spider – enter all domains, choose the 'list of URLs' crawling mode, and select the necessary parameters

Links to the tools: https://netpeaksoftware.com/checker and https://netpeaksoftware.com/spider

hope it helps, PM me if you have questions

https://www.screamingfrog.co.uk/seo-spider/user-guide/general/


Search for 'memory'