1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

How can find all URL of a website?

Discussion in 'Black Hat SEO' started by fabiospe, Jan 10, 2011.

  1. fabiospe

    fabiospe Newbie

    Joined:
    Jan 12, 2010
    Messages:
    10
    Likes Received:
    7
    hi to all
    i would find all URL of a site
    there a tool or software or some method to find all url of a website only know the name of this site?
     
  2. mataff

    mataff Junior Member

    Joined:
    Sep 21, 2008
    Messages:
    139
    Likes Received:
    54
    There are many ways but here's an easy one:

    Create a sitemap for that website using: http://www.xml-sitemaps.com/ then use the sitemap scraper addon in scrapebox with deep linking enabled.
     
    Last edited: Jan 10, 2011
  3. ScrapeBoss

    ScrapeBoss Elite Member Premium Member

    Joined:
    Nov 25, 2010
    Messages:
    1,865
    Likes Received:
    669
    Location:
    123.456.789.012.345.678.901.234.567
    Home Page:
    If it is a wordpress blog, you can get all the url through its rss feed. To do that, type this to your browser:

    hxxp://www.xxxxxx.com/feed/rss2

    That will give you all the urls for whatever you want to do.
     
  4. sfidirectory

    sfidirectory Senior Member

    Joined:
    Mar 29, 2010
    Messages:
    899
    Likes Received:
    483
    Occupation:
    Web developer/BTC enthusiast
    Location:
    php artisan make:migration
    Home Page:
    What the others said on this thread will help you out. Have you tried putting the search string "site:www.yoursite.com" (replace yoursite.com with the one you are enquiring about) into Google? Have tried that on my own site and it works, only if the pages are indexed (obviously). Or have you tried using the Yahoo Site Explorer (http://siteexplorer.search.yahoo.com/)? That seems to work for me too.
     
  5. J0kerz

    J0kerz Supreme Member

    Joined:
    Nov 2, 2009
    Messages:
    1,414
    Likes Received:
    435
    Occupation:
    IM
    Location:
    There
    You can also write:

    site:UrlOfYourWebsite

    on Google.