1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

is there a program that shows pages with exactly the same titles on a site

Discussion in 'Black Hat SEO' started by turdface, Jul 21, 2017.

  1. turdface

    turdface Supreme Member

    Joined:
    Sep 3, 2010
    Messages:
    1,442
    Likes Received:
    193
    If a site has a lot of pages with the same title, I want a program that can audit it and show me which ones it is.
     
  2. rickydzine

    rickydzine Regular Member

    Joined:
    Jun 6, 2011
    Messages:
    459
    Likes Received:
    237
    Screaming Frog SEO Spider Tool should be able to do this.
     
  3. MatthewGraham

    MatthewGraham Jr. VIP Jr. VIP

    Joined:
    Oct 6, 2015
    Messages:
    1,064
    Likes Received:
    949
    Gender:
    Male
    Occupation:
    Rolling Face on Keyboard
    Location:
    United States of America
    Home Page:
    You can presumably do this with a Linux terminal command of some kind. Messed with it some and got to this:

    wget -qO- 'https://www.blackhatworld.com' |
    perl -l -0777 -ne 'print $1 if /<title.*?>\s*(.*?)\s*<\/title/si' |
    recode html.. > all-titles-output-files.txt


    But still needs to be edited to get all of the filenames recursively instead of just the one page. Tried to get the code to run recursively but was taking too long to figure out how to do that and got bored.

    Would also need to then remove all non-duplicates.

    If I had more free time I would mess with it more for my own entertainment, but should probably focus on more productive things. If anyone figures out a good solution off of that I would be interested to see it.

    Credit to this person for most of that code:
    https://unix.stackexchange.com/questions/103252/how-do-i-get-a-websites-title-using-command-line
     
  4. turdface

    turdface Supreme Member

    Joined:
    Sep 3, 2010
    Messages:
    1,442
    Likes Received:
    193
    Thanks for the reply previously. I'm looking for something that will go through a site and identify all the sites that have the same titles on different pages, to identify them so that we can go change them.