[edit] some of the comments in the code won't make any sense since they were notes to myself about things that may or may not appear in the final code. Sorry for the sloppiness, but this is a big project for me (a non-coder) and I had to capture thoughts as they showed up. The code works, even if my mental machinery doesn't.
I hacked this together out of some scripts I've collected over the years as a brainstorming tool. Basically you create a database and put "seed" keywords in it. Doesn't really matter what the words are, it's more important to have a bunch of them. As a hint, I did a google search for "list of nouns" and tossed a couple thousand into my table.
The tool reads the keyword out of your database then scrapes the 10 results out of google's suggest tool, basically letting google tell you what people are interested in.
Then it drops the keywords into a new table for you.
You can export that table and get keyword ideas, then go and research any that catch your eye through the google keyword tool to see if there's any money to be made on them.
I have it echoing a bunch of stuff because I'm not really a coder and I needed it to spit stuff out so I could could tell where it was in its processes to debug it. Too lazy to take it all out. Besides, I like being called "boss" all the time.
I can't take any credit for the scraping code. I don't remember exactly where I got it, but I definitely stole it from someone else. Thanks to whoever it was by the way.
I also borrowed couple of the other functions from php.net and others. All I did was spackle them together and create the functions to run the whole operation off of a database.
It's designed to be run from the command line, but it also works as a web page. Just change all the \n's to <br> to make it output better on a page.
Most of the results you get back are useless for making money (clarinet reed mildewed for example). Some of them are great though. Just takes a lot of patience to run them through the keyword tool.
I'm working on automating that too, but since I'm not a big fan of filling out captchas all day, I'm trying to integrate it with an Adwords account. I don't want to lose one since they're a pain in the ass to get back, so I'm going carefully with that one so as not to piss off Big G.
I might release that when I'm done with it.
Anyway, this has been tested, without a proxy, on runs up to 2,000 keywords with no real problems other than getting a term that can't evaluate because it has no results. Just restart the bot and let 'er rip when it dies.
I also recommend using it in a "normal" way. Run it on a few hundred keywords, then give it a break for a few hours. Start it again.
It's got a random sleep function in it to keep you from beating the hell out of google's server, but I wouldn't let it run 24/7. That'll eventually get your IP blocked.
it could use some more work, but I'm leaving it alone for now to try to finish up the overall project.
Maybe it'll help you out as-is.
-scuba
Code:
<?php
// $iterations = rand(25, 500) //pick some number of keywords to run each time the CRON fires.
//create a weighting function that says if the number is lower, sleep for a random period between runs that adds up to the total time of the bot allowance. if higher, run faster.
$iterations = 500; //or use the rand function above
$dupes = 0;
$whoops = 0;
$username="dbaseusername";
$password="dbasepassword";
$database="dbasename";
$seed="";
mysql_connect(localhost,$username,$password);
mysql_select_db($database) or die( "Unable to select database");
for ($count=0; $count<$iterations; $count++) //count each keyword up to the max as determined by the rand above
{
$seed="";
$seed = getNextKeyWord(); //assign the value returned by the function getNextKeyword to the variable $keyword to be used in function scrapeResults
echo "THE SEED KEYWORD IS: $seed \n\n";
$keywordlist = scrapeResults($seed);//execute function scrapeResults using the keyword gathered above to get google suggest results
saveResults($keywordlist);
//check the results to see if they already exist in tblSuggest. If not, add them.
echo "\n\nOk, we're gonna try this one now. Wish me luck, boss. \n\n We're trying to take $seed out of action...";
updateTBLSeed($seed); //note: if the scrape is successful, that's the point at which tblMaster should have the suggest value changed to true
echo "Sorry boss... you commented out everything else... just sitting here... <yawn>\n\n";
echo "\n\n\n\n\n\nSLOWING THINGS DOWN HERE!!!!";
$slp=rand(3,55);
echo "\n\n\nSo far I've done about $count runs. There are more to go.\n\n\n";
echo "\n\n\n\n\n\n\n\nGonna nap for $slp seconds. brb.";
sleep($slp);
}
mysql_close();
die ("Well, we got ".$dupes." dupes, and ".$whoops." whoops on this run, boss! \nSorry about that!\n\n\n");
function getNextKeyWord()
{
$result = mysql_evaluate_array("SELECT pk from tblSeed WHERE suggested !='1'"); //select pk values that have not already been used
if ($result == 0)
{
die("Ran out of Seed Words boss...");
}
$numRows = sizeof($result); //count the number of results
echo "Maybe... \n\nLooks like we got us about: $numRows rows, boss \n\n";
$id=rand(0,$numRows); //generate a random number between 0 and count_of_rows in the database.
$query = "select keyword from tblSeed where pk = \"".$result[$id]."\""; //define the pk as the $id_th array element in a query to get the keyword out of the dbase
$nextkey = mysql_evaluate($query);
if ($nextkey=="undefined")
{
die ("Got an undefined here from function mysql_evaluate, boss");//get the keyword
}
else echo "Got one: $nextkey \n\n";
return $nextkey; //send it back to the function call
}
function mysql_evaluate_array($query)
{
$result = mysql_query($query);
if (!$result)
{
die('AAAAARGH. mysql_evalute_array function Could not query:' . mysql_error());
}
$values = array();
for ($i=0; $i<mysql_num_rows($result); ++$i)
array_push($values, mysql_result($result,$i));
return $values;
}
function mysql_evaluate($query, $default_value="undefined")
{
$back = mysql_query($query);
if (mysql_num_rows($back)==0)
return $default_value;
else
return mysql_result($back,0);
}
function scrapeResults($keyword)
{
$ch=curl_init();
$useragent="Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1";
curl_setopt($ch, CURLOPT_USERAGENT, $useragent);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER ,1);
curl_setopt($ch, CURLOPT_URL,"http://clients1.google.com/complete/search?hl=en&q=".urlencode($keyword));
curl_setopt($ch, CURLOPT_REFERER,"http://www.google.com/webhp?complete=1&hl=en");
$data=curl_exec($ch);
curl_close($ch);
$data=substr($data, strpos($data,"[[")+1);
preg_match_all("/\[\"(.*?)\",\"(.*?) results/si", $data, $out);
return($out[1]);
}
function saveResults($list)
{
for ($i=0;$i<sizeof($list); $i++)
{
echo "Slapping this on into the dbase, boss: ".$list[$i]."\n\n";
$query = "INSERT IGNORE INTO tblSuggest (keyword) VALUE ('$list[$i]')";
mysql_query($query);
}
}
function updateTBLSeed($seed)
{
$query = "UPDATE tblSeed SET suggested = '1' WHERE keyword ='$seed'";
mysql_query($query);
echo "Took this one off the list, boss: $seed \n\n";
$cleared=mysql_evaluate("select suggested from tblSeed where keyword = '$seed'");
if ($cleared > 0)
{
echo "It's set to $cleared!!!\n\n";
}
else
{
echo "Dang. It's still: $cleared...\n\n";
}
}
?>