Hell0 all!
First Time Poster, First Timer on this site...
I have experience in HTML and PHP, and I'm trying to write something that will peruse ca category's listings faster than the search dialogue that's there.
When you post a CL ad, you get emailed a listing link (among other things) immediately. Which is public, if you have the link. Thus, I was trying to see if there was a numbering schema to listings or if it was a random number, and I haven't been able to figure it out. The listings seem to be sequential, but not consistently numbered... to my knowledge. I have a hunch that it is global for all craigslists, as for a span of 30 minutes i'll see a number change in the tens-of-thousands, but I'm just not sure.
I am trying to figure out what the listing scheme is for a particular category
so that I can view listings as they post, not 20-30 minutes later. I made a PHP file that would load the url, see if it was a valid listing, and if so, it would post it, otherwise it'd go to the next number, doing 30 at a time to allow for page timeouts.
Here's what I wrote, but it takes 30-40 seconds to do 50 numbers... And I don't want to get my server blocked from 'excessive' access to craigslist servers....
Any Ideas?
First Time Poster, First Timer on this site...
I have experience in HTML and PHP, and I'm trying to write something that will peruse ca category's listings faster than the search dialogue that's there.
When you post a CL ad, you get emailed a listing link (among other things) immediately. Which is public, if you have the link. Thus, I was trying to see if there was a numbering schema to listings or if it was a random number, and I haven't been able to figure it out. The listings seem to be sequential, but not consistently numbered... to my knowledge. I have a hunch that it is global for all craigslists, as for a span of 30 minutes i'll see a number change in the tens-of-thousands, but I'm just not sure.
I am trying to figure out what the listing scheme is for a particular category
so that I can view listings as they post, not 20-30 minutes later. I made a PHP file that would load the url, see if it was a valid listing, and if so, it would post it, otherwise it'd go to the next number, doing 30 at a time to allow for page timeouts.
Here's what I wrote, but it takes 30-40 seconds to do 50 numbers... And I don't want to get my server blocked from 'excessive' access to craigslist servers....
Code:
// Set Time Limit indicated in settings above
set_time_limit($timelimit);
function doesListingExist($start,$end,$limit) {
$page = $_GET['p'];
$listingnumber = $start + ((($page + 1) * $limit) - $limit);
$listingnumberEND = $listingnumber + $limit;
$troubleshooting = true;
if ($troubleshooting) {
echo 'listing num to start with: ' . $listingnumber . '<br />';
echo 'listing num to end with: ' . $listingnumberEND . '<br />';
echo 'limit to ' . $limit . ' per page<br />';
echo 'page number: ' . $page . '<br />';
}
while($currentNum <= $end) {
// Assign Link Location
$urlToLoad = 'http://stlouis.craigslist.org/mob/' . $listingnumber . '.html';
// See if Listing Exists
if (!fopen($urlToLoad, 'r')) {
return;
}
// Open Listing and assign to Variable
$handle = fopen($urlToLoad, 'r');
while (!feof($handle)) {
$buffer .= fgets($handle, 4096);
}
fclose($handle);
// See if Listing is Active
// TROUBLESHOOTING echo "||" . strpos($buffer, '<div id="flags">') . "||";
if(strpos($buffer, 'iPhone')) {
$newstart = $listingnumber + 1;
echo "go to <a href='test.php?start=" . $newstart . "'> next page... </a>";
echo '<iframe src="http://stlouis.craigslist.org/mob/' . $listingnumber . '.html" frameborder="0" style="width:100%;height:900px;"></iframe>';
return;
}
if ($listingnumber == $listingnumberEND) {
$page++;
echo "go to <a href='test.php?p=" . $page . "'> next page... </a>";
return;
}
$listingnumber++;
}
}