1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Seaching for footprints in a list of URLs

Discussion in 'Black Hat SEO' started by ningning, May 18, 2011.

  1. ningning

    ningning Jr. VIP Jr. VIP Premium Member

    Joined:
    Mar 22, 2010
    Messages:
    926
    Likes Received:
    261
    I have been wondering about this for a while.

    Is there a tool that can search in a list of URLs for a custom footprint.

    This is basically what Scrapebox does when it is scraping with search engines, but I don't want a search on the entire net. I only want to search a specified list of URLs or domains.

    I know I can use site:domain.com but this will not work with a list of several thousands URLs.

    I found a tool called "Ultimate SEO Blog Analyzer" but it crashes constantly on my Win 7 machine.

    Any suggestions?

    Thanks
     
  2. ningning

    ningning Jr. VIP Jr. VIP Premium Member

    Joined:
    Mar 22, 2010
    Messages:
    926
    Likes Received:
    261
    You could also call this a blog analyzer with support for custom footprints.
     
  3. BHopkins

    BHopkins Moderator Staff Member Moderator Jr. VIP

    Joined:
    Dec 31, 2010
    Messages:
    2,311
    Likes Received:
    1,387
    Gender:
    Male
    Occupation:
    ORM and SEO company owner
    Location:
    California
    Home Page:
    Are you looking to separate everything after .com/ to look for similarities?
     
  4. ningning

    ningning Jr. VIP Jr. VIP Premium Member

    Joined:
    Mar 22, 2010
    Messages:
    926
    Likes Received:
    261
    No. Here is an examble:

    I have 5 URLs:

    http://domain1.tld/pagesomething
    http://domain2.tld/pagesomething
    http://domain3.tld/pagesomething
    http://domain4.tld/pagesomething
    http://domain5.tld/pagesomething

    Now I want to look for footprints on these sites. Lets say I want to find out if any of these sites are wordpress blogs. Then I will search for ?powered by wordpress?.

    Now the tool should read through all the URLs and return a list with the sites containing the footprint.
     
  5. dowser

    dowser Power Member

    Joined:
    Jun 5, 2011
    Messages:
    685
    Likes Received:
    122
    Location:
    canada
    Have you set a rule for your firewall? I almost gave up on it and now I find it very powerful and fast.

     
  6. cyberzilla

    cyberzilla Elite Member Premium Member

    Joined:
    Nov 15, 2009
    Messages:
    2,204
    Likes Received:
    3,364
    Location:
    zeta reticuli
    Let me know what exactly you want to extract from your list. I will figure out a way if possible.
     
  7. ActionTaker

    ActionTaker Power Member

    Joined:
    Jun 11, 2011
    Messages:
    606
    Likes Received:
    209
    You basically want the text that is similar on the different blogs.

    I don't know of such software, but finding footprints doesn't take you long.

    Chances are nothing will repeat itself on all the different blogs meaning you are looking for patterns.. I'm not too sure it will be that easy with software.
     
  8. paincake

    paincake Power Member

    Joined:
    Aug 18, 2010
    Messages:
    716
    Likes Received:
    3,099
    Home Page:
    There's a free tool called sick platform reader. It uses a lot of ram and crashes if you use lists longer than 20k but it works. Google it and you'll find it.
     
  9. ActionTaker

    ActionTaker Power Member

    Joined:
    Jun 11, 2011
    Messages:
    606
    Likes Received:
    209
    How does this work? I installed it buy only have the option "Get root PR" "Trim URLs to root" "Separate file per platform"

    OP is looking for software that finds the actual footprints.
     
  10. keinehabe

    keinehabe Supreme Member

    Joined:
    Nov 4, 2008
    Messages:
    1,207
    Likes Received:
    472
    Gender:
    Male
    Occupation:
    -= CEO =-
    Location:
    Heaven
    Home Page:
    try to look at the options from notepad++ :) it's free just need to use your imagination and you find hundreds of nice tutorials on net lol ...