How to scrape a certain site?

[undeterminederror]

I need to get all phrases with a specific keyword from a certain site. The site is too big to use google function site:thesite.com "keyword combination".
There is any solution for that?

 

[goawayplease]

Are you trying to get phrases from within the content, or get all the pages that contain "keyword combination?"

 

[brainet77]

are you trying to scrape content from Search Engine or a regular sites?

 

[BozoClown]

You could write a script(i would use perl) to get every keyword on the page, 10 words before it and 10 words after it. Then output as new line in a file. I guess you could do it in less than 15 lines of code.

If I could write the script off the top of my head I would but I can't write an original line of code in perl. Despite that I have come up with perl scripts that do similar harvesting/mining just by loading dozens of mini-tutorials off google then mixing and matching to get what I want. Sounds complicated, but it is very easy to do, just find the first tutorial with code that does something even remotely related.

 

[undeterminederror]

You could write a script(i would use perl) to get every keyword on the page, 10 words before it and 10 words after it. Then output as new line in a file. I guess you could do it in less than 15 lines of code.

If I could write the script off the top of my head I would but I can't write an original line of code in perl. Despite that I have come up with perl scripts that do similar harvesting/mining just by loading dozens of mini-tutorials off google then mixing and matching to get what I want. Sounds complicated, but it is very easy to do, just find the first tutorial with code that does something even remotely related.

You are making fun of me? LOL. wtf is "line of code?" i never wanna hear about that again.
... yes, to scrape a site, not a search engine. and all the phrases, not the pages

 

[ghands]

Hey Bozo, is there any reason that you would use Pearl versus PHP Curl. I use a WAMP server so testing is pretty easy.