1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Tool to get all URLs of a site?

Discussion in 'Black Hat SEO' started by shenZi, Aug 26, 2016.

  1. shenZi

    shenZi Regular Member

    Joined:
    Sep 23, 2009
    Messages:
    232
    Likes Received:
    18
    Hi dudes,

    do you know any tool, to check all url´s of a website? I didnt find the sitemap of the site, so
    it would be great to have a tool, that scrawl all pages and show me the urls.

    Hope anybody can help me!

    Greets
     
  2. redarrow

    redarrow Elite Member

    Joined:
    Apr 1, 2013
    Messages:
    4,286
    Likes Received:
    978
    You can use scrape box to scrape all the links on a website.

    You can also use href to look at alk the back links.

    There might be hidden links and your never no as they hide behinde the htaccess file.

    Most pbn are out there but hidding impossable to see...
     
  3. wpbacklinks

    wpbacklinks Elite Member

    Joined:
    Mar 27, 2010
    Messages:
    3,439
    Likes Received:
    1,362
    Gender:
    Male
    Occupation:
    Affiliate Marketer
    Location:
    Everywhere
    Screaming Frog
     
  4. shenZi

    shenZi Regular Member

    Joined:
    Sep 23, 2009
    Messages:
    232
    Likes Received:
    18
    I dont mean to show me all backlinks of a website. I mean all the URLs of the website :))
     
  5. clevelandslim

    clevelandslim Jr. VIP Jr. VIP

    Joined:
    Nov 15, 2008
    Messages:
    1,856
    Likes Received:
    188
    I prefer you to use Screaming Frog, are an innovative search engine marketing agency offering search engine optimisation (SEO) and pay-per-click (PPC) advertising services. It helps us for technical site audits, keyword research, competitive analysis, link building etc.
     
  6. loopline

    loopline Jr. VIP Jr. VIP

    Joined:
    Jan 25, 2009
    Messages:
    3,724
    Likes Received:
    1,993
    Gender:
    Male
    Home Page:
    With scrapebox you can crawl the site and get all the urls of the site either with the grab urls by crawling a site or the link extractor, I have videos on both here
    https://www.youtube.com/user/looplinescrapebox/videos

    just search

    Failing that you might be able to get it with xenu link sleuth, haven't' tried, but its free.
     
  7. Flyn

    Flyn Newbie

    Joined:
    Jun 17, 2016
    Messages:
    26
    Likes Received:
    1
    Gender:
    Male
    You can use scrap box to view all the urls.
     
  8. Abuzzz

    Abuzzz Junior Member

    Joined:
    Jun 26, 2016
    Messages:
    101
    Likes Received:
    14
    Gender:
    Male
    I use Xenu.It's free and simple.