G search result returns nothing after I Crawl it with curl !

mainceaft

Senior Member
Apr 10, 2013
821
159
In short I'm developing g SERP crawler I use PHP function curl to grab it contents , but it's only return empty string nothing like the connection can't be done .
The weird thing that I used same broxy I on my brwoser and the SERP page were showing like usual , BTW I use curl function to grab consents from other sites and the result were perfect
+ I change the user agent of curl function with no change .
This is my curl function

Code:
function corl(  $url  )
{
$base = $url;
$curl = curl_init();
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($curl, CURLOPT_HEADER, false);
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($curl, CURLOPT_URL, $base);
//curl_setopt($curl, CURLOPT_REFERER, $base);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, TRUE);   
//curl_setopt($curl, CURLOPT_USERAGENT,'Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:25.0) Gecko/20100101 Firefox/20.0');
curl_setopt($curl, CURLOPT_PROXY,'socks4://192.168.1.9:322' ) ;
$str = curl_exec($curl);
curl_close($curl);
echo "$str  ".PHP_EOL;
    return $str;
   }

the example url is this
 
You're using a socks proxy and are trying to connect like an http proxy. You need to specify socks and also whether it's socks4 or socks5.

curl_setopt($ch, CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5);
 
You're using a socks proxy and are trying to connect like an http proxy. You need to specify socks and also whether it's socks4 or socks5.

curl_setopt($ch, CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5);
In newer PHP version no need for that , as I said I did test for curl function with the same setting and the proxy and its working https://www.ipchicken.com/ gived me exact same proxy address I used in curl .
Anyway when I used my VPS address without any proxy the SERP page works for multiple time , I think G bans proxy IP's immediately even without showing error message .
 
Last edited:
Back
Top
AdBlock Detected

We get it, advertisements are annoying!

Sure, ad-blocking software does a great job at blocking ads, but it also blocks useful features and essential functions on BlackHatWorld and other forums. These functions are unrelated to ads, such as internal links and images. For the best site experience please disable your AdBlocker.

I've Disabled AdBlock