1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Why I am Restricted to 1000 Results(URL Extraction from a Website)

Discussion in 'Black Hat SEO' started by mtayyabrana, Mar 28, 2010.

  1. mtayyabrana

    mtayyabrana Registered Member

    Joined:
    Sep 13, 2009
    Messages:
    91
    Likes Received:
    3
    Occupation:
    Computer related but, But My business in Online bu
    Location:
    Pakistan
    One Problem: When I am Extracting URL's from a Website

    When I am going to Search all URL's from a website,

    Why I Restricted to only 1000 Extracted URL's from Website:



    Is MY Software, have problem, or we can't extract no more than 1000 urls


    Thanks If You Views This Thread Then Please Comment, As You Wish
     
  2. Cloaks

    Cloaks Regular Member

    Joined:
    Mar 20, 2010
    Messages:
    298
    Likes Received:
    90
    I programmed my own URL extractor too..
    Google simply doesn't give more than 1000 results. It's a limit from their side. That's just how things are..
    So it's not your software, it's a limit from Google.
     
  3. mtayyabrana

    mtayyabrana Registered Member

    Joined:
    Sep 13, 2009
    Messages:
    91
    Likes Received:
    3
    Occupation:
    Computer related but, But My business in Online bu
    Location:
    Pakistan
    It means that we can't find all pages of a website that more than 1000 webpages :(
     
  4. Cloaks

    Cloaks Regular Member

    Joined:
    Mar 20, 2010
    Messages:
    298
    Likes Received:
    90
    That is true, but if you use the right dorks when searching then you can still get some good results. Things like site:edu or site:gov or site:us and so on can give you 1000 related sites for each language.. So that would be more like 5000 sites.. And there are so many dorks. Just google "Google Dorks".
    Hope that helps!
     
  5. mtayyabrana

    mtayyabrana Registered Member

    Joined:
    Sep 13, 2009
    Messages:
    91
    Likes Received:
    3
    Occupation:
    Computer related but, But My business in Online bu
    Location:
    Pakistan
    Any way thanks for giving me some good suggestions
     
  6. vette09

    vette09 Junior Member

    Joined:
    Sep 28, 2009
    Messages:
    174
    Likes Received:
    40
    Occupation:
    Traveler
    Location:
    Earth - I travel full time.
    use parameter. it crawls the site and extracts all of the URLs. It'll even check their PR.