1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Working on a crawler for PR Storm (need suggestions)

Discussion in 'Black Hat SEO Tools' started by xpro, May 5, 2009.

  1. xpro

    xpro Regular Member

    Joined:
    Jan 21, 2009
    Messages:
    422
    Likes Received:
    18
    I have started to word on a crawler so it can crawl pages for PRStorm, the part I need help on is that I don't know which sites publish their logs, Is there anyway to find those sites with Google search? something like "Powered by awstats"?
     
  2. entreprenuer

    entreprenuer Regular Member

    Joined:
    Apr 19, 2009
    Messages:
    200
    Likes Received:
    73
    Location:
    Orlando Fl
    well why don't you just crawl for the logs, and If you find the log then it has to be published right?

    Im not smart on this kind of stuff so correct me if Im wrong.
    hope I can help ;)
     
  3. xpro

    xpro Regular Member

    Joined:
    Jan 21, 2009
    Messages:
    422
    Likes Received:
    18
    yea, that would be the best, but how can I find sites that have logs?
     
  4. spaceinvader

    spaceinvader Regular Member

    Joined:
    Jan 28, 2009
    Messages:
    235
    Likes Received:
    248
    Occupation:
    Dude
    Location:
    England
    you mean stuff like this?

    Code:
    intext:?Created by awstats?
    ?Generated by Webalizer Version 2.01″
    ?Generated by Webalizer Version *?
    ?Top * of * Total Referrers?
    ?usage statistics? ?Summary Period: August 2008″ (change the month to get more results)
    Code:
    inurl:yourkeyword +usage_200804 (enter your keyword and change the date for more results)
    inurl:/webalizer/usage_200806.html (change the date for more results)
    inurl:.com/webalizer/usage_200806.html (change the date and .com for more results like)
    inurl:.edu/webalizer/usage_200805.html
     inurl:.net/webalizer/usage_200804.html
     inurl:.org/webalizer/usage_200803.html
     
    • Thanks Thanks x 1
  5. entreprenuer

    entreprenuer Regular Member

    Joined:
    Apr 19, 2009
    Messages:
    200
    Likes Received:
    73
    Location:
    Orlando Fl
    wow! so If I search those strings in a google serp scraper I will find a list of links that can be pinged?