Free Web Site Crawling [LARGE!]

[serge.w1lllliii]

Hi all!

I'm developing web crawler for big websites (1-10 millions pages) more than half a year. Now I testing it on everything.
What I store:
- page info: titles, desc, headers, status code, load time, distance to index, out links, in links, more than 20 attributes
- links: anchor, follow, index, alt text, image, robots blocked and more

I offer a FREE crawl of any website (size less than 10 mln pages). As result I give you CSV, Excel file or access to MySQL database with all contents. Your data will NOT shared to anyone.

Just write me PM.

 

[valdu02100]

Can you MP me ?

 

[bhlearner]

How are you crawling the pages?

 

[Gogol]

How are you crawling the pages?

Sure he is not using JCBs or shovels.

 

[tugeye]

Hi all!

I'm developing web crawler for big websites (1-10 millions pages) more than half a year. Now I testing it on everything.
What I store:
- page info: titles, desc, headers, status code, load time, distance to index, out links, in links, more than 20 attributes
- links: anchor, follow, index, alt text, image, robots blocked and more

I offer a FREE crawl of any website (size less than 10 mln pages). As result I give you CSV, Excel file or access to MySQL database with all contents. Your data will NOT shared to anyone.

Just write me PM.


View attachment 93741 View attachment 93742

Are you still offering the free scrape?

 

[greatx]

Hi can u PM me pls

i really need this also keeping learning for web penetrating

i am lookig forward!

 

[tugeye]

Hi can u PM me pls

i really need this also keeping learning for web penetrating

i am lookig forward!

lol..how the hell do you pm on here?

 

[greatx]

lol..how the hell do you pm on here?

ireally dunno but just wanna try it if he gonna make his promise