1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Any way to find ALL the urls of internal pages of non-indexed domains?

Discussion in 'Black Hat SEO' started by geo89gr, Feb 27, 2014.

  1. geo89gr

    geo89gr Junior Member

    Joined:
    Sep 27, 2013
    Messages:
    127
    Likes Received:
    12
    As title suggests, i want to find the list of all previously existing internal page urls

    I have used site:[] but it only returns the indexed pages
    I have also used the internet archive but it is a pain in the a$$ since it does not provide a list of urls and i have to list them manually by following them one by one.

    Any other ways??
     
  2. Chicilikit

    Chicilikit Senior Member

    Joined:
    Dec 21, 2010
    Messages:
    873
    Likes Received:
    153
    If the site is just deindexed, but still exist you can use xenu link sleuth or scrapebox addon, or try to find sitemap. If the site does not exist anymore, I dont know.
     
  3. geo89gr

    geo89gr Junior Member

    Joined:
    Sep 27, 2013
    Messages:
    127
    Likes Received:
    12
    nah, doesn't exist anymore.
     
  4. seocrab

    seocrab Senior Member

    Joined:
    May 2, 2013
    Messages:
    969
    Likes Received:
    825
    Occupation:
    seo freelancer
    Location:
    UK
    If the site is no longer live, archive.org is your only bet. See if there is a sitemap page in the records.

    Code:
    https://web.archive.org/web/*/http://YOURDOMAIN.com/sitemap.xml
     
  5. geo89gr

    geo89gr Junior Member

    Joined:
    Sep 27, 2013
    Messages:
    127
    Likes Received:
    12
    yeah as i said in my original post, i have already used that. I am looking for a more automated way.