1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

anyone here have sites with more then 200,000 pages?

Discussion in 'Black Hat SEO' started by genotip, Jun 4, 2011.

  1. genotip

    genotip Regular Member

    Joined:
    Aug 24, 2010
    Messages:
    256
    Likes Received:
    26
    can you shere a bit how did you indexed all those pages? how much time it took for you? and so on...
     
  2. chiwawa

    chiwawa Registered Member

    Joined:
    Apr 16, 2011
    Messages:
    61
    Likes Received:
    0
    I am also waiting on replies, want to know how they get in indexed.
     
  3. course575

    course575 Registered Member

    Joined:
    Apr 22, 2011
    Messages:
    50
    Likes Received:
    11
    Wondering, did you take a look at your own signature?
     
  4. emerorr

    emerorr Newbie

    Joined:
    Mar 17, 2011
    Messages:
    34
    Likes Received:
    1
    i was have one site with ~990.000 pages ; > Usign analytics and webmaster tools (added sitemap 50k links/each sitemap)
    on few years old main domain, and new subdomain usign free server. Main domain was have 1.800.000 pages indexed total.

    After 2 weeks i was have ~50.000 pages indexed by google and traffic 1600u/daily ... but administrators was deleted this site... they have right ofcourse;]

    All of these pages was have few unique words only. I have some experience with creating websites with huge amount of pages ; > If you want know anything just ask
     
  5. escobarcobar

    escobarcobar Newbie

    Joined:
    Jun 1, 2011
    Messages:
    6
    Likes Received:
    0
    omg what kind of sites have 990.000 pages?
     
  6. emerorr

    emerorr Newbie

    Joined:
    Mar 17, 2011
    Messages:
    34
    Likes Received:
    1
    a spam websites ; > just put some trash and it will give you some result... but for this best way is v.good programming skills (php,mysql,perl) and ofcourse html,css ;>
     
  7. genotip

    genotip Regular Member

    Joined:
    Aug 24, 2010
    Messages:
    256
    Likes Received:
    26
    did you do something that was speeding up the indexing? or just webmaster tools?
     
  8. genotip

    genotip Regular Member

    Joined:
    Aug 24, 2010
    Messages:
    256
    Likes Received:
    26
    pinging is great to index your backlinks, pinging 200,000 pages of the same site is not such a great idea.

     
  9. savagepriest

    savagepriest Junior Member

    Joined:
    Jan 12, 2009
    Messages:
    167
    Likes Received:
    18
    Home Page:
    use interlinked sitemaps
     
  10. genotip

    genotip Regular Member

    Joined:
    Aug 24, 2010
    Messages:
    256
    Likes Received:
    26
    the strage think is that i have 20 subdomains, one of them have great indexing speed - all the others are very very slow.

    there have same code, same design, same sitemaps - everything the same.

    but still - 1 subdomain indexed already 100K pages, all the rest have only few hundreds.

    and i don't understand WHY :)

     
  11. JackSparrow

    JackSparrow Supreme Member

    Joined:
    Mar 24, 2007
    Messages:
    1,469
    Likes Received:
    2,054
    Occupation:
    Semi-Employed Pirate
    Location:
    search 4 blackhatsparrow
    Home Page:
    definitely some sort of sitemap indexing structure.

    I mean building backlinks to the index page of a sitemap directory, will get the search engine spiders crawling through the sitemap directory finding the links of your webpages.

    But you will have to construct the sitemap directory properly or it will look like spam.
     
  12. hispdcha

    hispdcha Regular Member

    Joined:
    May 24, 2011
    Messages:
    289
    Likes Received:
    133
    My money site is a real estate site with rougly 50,000 listings with pagination to show the listing as well as a map and other stuff. So in total my website has roughly 350k pages. I use sitemaps to get them indexed, currently I have 8 sitemaps setup in a sitemap directory page with 100 links per page. So 8 sitemaps = 6*50k Links at 100 links per page = 500 sitemap pages per sitemap.

    This is completely whitehat and Google has some nice tech information about sitemaps. they say keep the number of links at 100 on each page and break sitemaps into 50k URL's or less.

    My other two sitemaps are for my normal content pages and a video sitemap.

    I have 308k pages indexed right now, and it's due to the strength of my site + link network I have built up over the last 3 years.

    As to your original question, how to get that many pages indexed: Sitemaps.
     
  13. genotip

    genotip Regular Member

    Joined:
    Aug 24, 2010
    Messages:
    256
    Likes Received:
    26
    thanks all, i got the feeling the only 1 subdomain pages was indexed fast and google was giving all the effot for this subdomain. when it over now, i can see that all other subdomains starting to be crawled.... i thought subdomains are like diffurent domains on google eyes, but maybe this is not 100% true.
     
  14. crackpie

    crackpie Registered Member

    Joined:
    Feb 8, 2010
    Messages:
    56
    Likes Received:
    42
    Home Page:
    I've got a site with 155K pages indexed. All scraped content but each pages pulls in content from various sources so it looks pretty unique.

    Site structure and unique content makes a huge difference - backlinks obviously help as well.

    If your pages frequently update their content that also helps a lot.