Building Online URL Scraper - Need Testers...

shadowpwner said:

You, sir, win a couple internets.

Awesome script, works fine. However, signing up, i had to try twice. This is probably aweber's fault.

Some advice: the screen froze up when i was trying to get 2000 keywords. Could you output it as a downloadable file, instead of on browser text?

Click to expand...

Thanks for the compliments - much appreciated

Yeah it can make the browser go a bit slow with thousands of results - depends how much horse power your computer's got.

I thought about a downloadable file option, which would be awesome, but as all the urls are kept client side it would mean posting them all back to the server just to serve them as a text file (JS can't trigger a downloable file).

I'm still thinking of a good way of doing this - without having lengthy POST requests chew on my bandwidth

For now what I'm doing is grabbing 1000 or so then just refreshing the page to clear the cache.

dizzspaceteam said:

If you need anymore testers, we would like to try it out.

Click to expand...

Sure - it's live

Just go to http://gscrape.com

Lemmie know what you think.

I just noticed that if you have the noscript plugin in FF the site will not function correctly.

To use the site you must add an exception for gscrape.

Just thought I should let you know.

Great little site. Is there any way you can add functionality to search within specific superdomains (only www.site.in, or only www.site.ac.uk or www.site.edu?)