1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

How would you index a big site.

Discussion in 'Black Hat SEO' started by RedKiller, Jan 24, 2012.

  1. RedKiller

    RedKiller Junior Member

    Joined:
    Oct 16, 2010
    Messages:
    183
    Likes Received:
    75
    Hello,

    How would you index a big site (indexing all pages of a site).

    What things would you do.

    Thanks
     
  2. ronmojohny

    ronmojohny Regular Member

    Joined:
    Dec 9, 2008
    Messages:
    283
    Likes Received:
    59
    install a sitemap and register it in google webmaster tools
     
  3. themidiman

    themidiman Power Member

    Joined:
    Feb 25, 2011
    Messages:
    701
    Likes Received:
    1,535
    Location:
    root@pts/0
    It all starts with the structure of the site. If a site is internally linked well, with good unique content, there isn't much you should really have to do. Build a couple links and the bots will take care of the rest.
     
  4. nikao

    nikao Power Member

    Joined:
    Oct 14, 2011
    Messages:
    518
    Likes Received:
    180
    Home Page:
    sitemap, good navigation structure, internal linking and then of course external linking to the important pages within your site:)
     
  5. Fathom

    Fathom Power Member

    Joined:
    Jul 1, 2011
    Messages:
    518
    Likes Received:
    282
    Location:
    Hertfordshire
    Submitting a sitemap to Google webmaster tools does not improve indexation of a site.
     
  6. HarrisonHill

    HarrisonHill Regular Member

    Joined:
    Apr 29, 2010
    Messages:
    318
    Likes Received:
    212
    Actually, it does. It's kind of map of links to pages which haven't been linked anywhere else. Google can't crawl page if it's not linked anywhere, sitemap helps Google bot in this case.
     
  7. Dresden14

    Dresden14 BANNED BANNED

    Joined:
    Mar 7, 2010
    Messages:
    663
    Likes Received:
    76
    just like i would a small one?
     
  8. acotut

    acotut Elite Member

    Joined:
    Dec 1, 2010
    Messages:
    2,294
    Likes Received:
    1,040
    Gender:
    Male
    Home Page:
    Make sure all your pages are interlinked,and then index your homepage.The others will be indexed in the next few days.
     
  9. Normdogg

    Normdogg Registered Member

    Joined:
    Jan 25, 2011
    Messages:
    70
    Likes Received:
    15
    Occupation:
    Network Admin, Affiliate.
    Location:
    Ann Arbor, MI
    Focus on internal linking... and when you backlink, backlink depth pages..
     
  10. nikao

    nikao Power Member

    Joined:
    Oct 14, 2011
    Messages:
    518
    Likes Received:
    180
    Home Page:
    lot of people here who obviously never have dealt with a real 'big site' (talking 10,000+ pages)... its not as simple as 'link to the homepage and google will find the rest'..
    especially if you have new content/pages added on a daily basis.

    Internal linking structure is really important. make sure you have a multiple internal 'hubs' or landing pages / nodes, which are heavily interlinked but also build links to them from outside.
     
  11. Fwiffo

    Fwiffo Power Member

    Joined:
    Apr 7, 2010
    Messages:
    562
    Likes Received:
    325
    Occupation:
    Starship Captain
    Location:
    Pluto / Spathiwa
    big sites will get indexed. It's the small backlinks in odd places that may not have an inlink that most people concern themselves with indexing.
     
  12. Normdogg

    Normdogg Registered Member

    Joined:
    Jan 25, 2011
    Messages:
    70
    Likes Received:
    15
    Occupation:
    Network Admin, Affiliate.
    Location:
    Ann Arbor, MI
    I have several sites over 200k indexed pages, and one over 1 million.

    1. Internal Linking
    2. Sitemap submission
    3. Send backlinks to your depth pages.
     
  13. nik-0

    nik-0 BANNED BANNED

    Joined:
    Jan 19, 2012
    Messages:
    510
    Likes Received:
    96
    What worked best for me when I had to index a site with 5000 pages was to submit category url's into Google webmaster (yes you can do that manually, besides the sitemap). Within 2 days I had 95% indexed.
     
  14. Fathom

    Fathom Power Member

    Joined:
    Jul 1, 2011
    Messages:
    518
    Likes Received:
    282
    Location:
    Hertfordshire
    You are confused, I was talking about submitting a sitemap to Google webmaster tools not a sitemap in general.

    I do try to make it easy for you guys with my one liners.
     
    • Thanks Thanks x 1
  15. sujoydhar

    sujoydhar Senior Member

    Joined:
    Aug 20, 2011
    Messages:
    930
    Likes Received:
    1,025
    Occupation:
    Student
    Location:
    www.IAMSUJOY.com
    Home Page:
    try pinging website !