1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Only 100 out of 9000 pages indexed after 2 months?

Discussion in 'Black Hat SEO' started by qwiky, Oct 15, 2011.

  1. qwiky

    qwiky Newbie

    Joined:
    Apr 15, 2009
    Messages:
    7
    Likes Received:
    0
    I have a site that automatically makes about 50 unique pages daily from scrapped content. But I have not been able to get my pages indexed at all. After over 2 months, only about 100 out of 9000 pages have been indexed. A sitemap.xml file is automatically updated daily and pings Google once it is updated. The domain is newly registered, PR0, and it has less than 10 backlinks. What am I doing wrong? I see that some autoblogs can get thousands of pages indexed in less than a week but I can't even get 5% of my pages indexed.

    I am following some of my closest competitors and they have over 100000 pages indexed, PR0 and 0 backlinks. How is that possible?
     
  2. confined

    confined Regular Member

    Joined:
    Jan 4, 2009
    Messages:
    216
    Likes Received:
    91
    google recognizes your pages as crap. since they are crap. and thus, they don't get indexed because google says it hates crap. make it more unique, sprinkle some shit in with your crap
     
    • Thanks Thanks x 3
  3. PavLev

    PavLev Newbie

    Joined:
    Mar 27, 2011
    Messages:
    11
    Likes Received:
    0
    what would be some ingredients in said sprinkles?

    keyword density should be at what %? 10, 20, 30?

    what makes it more unique?

    thx
     
  4. Winchester

    Winchester BANNED BANNED

    Joined:
    May 5, 2010
    Messages:
    976
    Likes Received:
    1,879
    It's not the keyword density in this case, could be quality of the post (ie spelling/grammar) or originality of the content which is a big one
     
    • Thanks Thanks x 1
  5. Ewokson

    Ewokson Jr. VIP Jr. VIP

    Joined:
    Jul 12, 2011
    Messages:
    242
    Likes Received:
    70
    How unique are these articles? Are they tons of article pieces put together or are they put together and spun? Most likely google sees they are not unique/readable and doesn't index them.
     
  6. flibbertigibbet

    flibbertigibbet Regular Member

    Joined:
    Apr 11, 2010
    Messages:
    388
    Likes Received:
    188

    This is such a MYTH. Google is not an intelligent person- it's a computer algorithim. It can't think or feel or make decisions about what is crap and what isn't crap based on the words on the page. Someone has to TELL it what is good and what is bad, and that's impossible to do for every site on the web at any given moment. The unique content idea is a myth, too.

    Google still indexes crap all the time. I know this because I did a test with a 12,000 + page wp site a few months ago and ALL of the pages were indexed in less than a month. Bounce rate sucked. There was virtually NO content (just a wp header, and an H1 tag), and it still got a significant amount of traffic). There was literally NOTHING on the pages. It was kinda cool to see, actually. It was just a test though, so I took it down.

    OP, you're just not using the right services or combination of links to get indexed quickly. Check out the search feature here or check in the BST section for people who really know how to index that shit- It CAN be done! ;)
     
    • Thanks Thanks x 1
    Last edited: Oct 15, 2011
  7. qwiky

    qwiky Newbie

    Joined:
    Apr 15, 2009
    Messages:
    7
    Likes Received:
    0
    I am using a custom script to generate content. Are wp sites easier to get indexed? I haven't tried pinging each of the pages yet, I will try that next.
     
  8. Autumn

    Autumn Elite Member

    Joined:
    Nov 18, 2010
    Messages:
    2,197
    Likes Received:
    3,041
    Occupation:
    I figure out ways to make money online and then au
    Location:
    Spamville
    You might have heard of this massive field of computer science called "machine learning." Why is there so little gibberish in the high traffic serps these days? It's not because there's someone manually weeding it out.

    You and confined probably just have different definitions of "crap."
     
    • Thanks Thanks x 1
  9. Kickflip

    Kickflip BANNED BANNED

    Joined:
    Jan 29, 2010
    Messages:
    2,038
    Likes Received:
    2,465
    Do you really believe that Google can't program a spider that can basically read sentences to see if the sentences are crap? There are so many ways to tell Gibberish from Real Quality Content. Do you also believe that Google has humans sitting around GUESSING when you made a spelling mistake so that it can suggest a different phrase with correct spelling?

    It doesn't have to judge every site at any given moment, it only needs to judge a site when the spiders crawl the pages and send the information through their algorithms and compare the results to websites with similar content.

    Here is an example:
    Content Contains Word Not Found in Oxford Dictionary = -1
    Content Found on a Previously Crawled Webpage = -1
    Content Contains Phrases Commonly Searched in Google = +1

    Wow! I just wrote my own algorithm that make decisions on what IS and ISN'T crap!
     
    • Thanks Thanks x 1
  10. phpbuilt

    phpbuilt Jr. VIP Jr. VIP

    Joined:
    May 16, 2011
    Messages:
    1,650
    Likes Received:
    5,208
    Occupation:
    $ from websites I own.
    Location:
    putting monkeys in paypal
    content found on a common trusted CMS ... wordpress +1, joomla +2, vbulletin +5
    entire content of site less than 20% duplicate as found on other sites +5
    words on same page are LSI related to each other +20
    pages interconnect naturally in-content +5
    backlinks to page do not decay at an unnaturally high rate +2

    And there's probably hundreds more little calculations they do. If you're making bank with having garbage indexed, enjoy it while you can and save your pennies for creating real content because Google is going to keep getting smarter.
     
    • Thanks Thanks x 1
  11. flibbertigibbet

    flibbertigibbet Regular Member

    Joined:
    Apr 11, 2010
    Messages:
    388
    Likes Received:
    188
    I think you guys were mis-understanding what I meant by Google not being able to determine what is and isn't crap based on the words on the page. That was my fault, as I didn't go into much detail in that post.

    I didn't mean that the spiders/algorithms couldn't check for misspellings or check for sentence structure. I didn't mean that Google can't check for duplicate content (the exact words in the exact same order on 50 different websites). We all know that it can. I know artificial intelligence works. I'm not stupid.

    What I meant by garbage is that there are people putting up a ton of articles based on a keyword and ranking in #1 spots that either

    1) ramble on about the keyword and don't really give people a whole lot of USEFUL information. They just pick a few paragraphs from other sites like amazon and PARAPHRASE what they read and then re-write it or spin it for their own page. It's TECHNICALLY "unique" because they're spinning it or reordering the sentence structure or throwing in a few synonyms here and there, but it's not really unique. The whole purpose of the site is just to get some measly adsense clicks or an order from amazon or something.(I find a LOT of xfactor sites DAILY still up that do EXACTLY this)

    or

    2) they put up a site that targets a specific keyword and the article is decent but they didn't actually research what the searcher WANTED and thus are talking about something COMPLETELY different from what the visitor wanted. They just ASSUME that their content is original, unique, and great for the visitor, but in all actuality, it's crap to the visitor. So the visitor doesn't get the information they needed and they just bounce and find another site to go to.

    by "garbage" I mean that a lot of IMers don't know how to market properly- they don't know how to give visitors what they want.

    Google can't tell the difference between an xfactor page that has a lot of paraphrased stuff verses a really great page that has original thought and that is really well planned out and researched. That's all I meant. The links to either page does determine this eventually, but it's far from perfect.

    As we all know, the serps are manipulated every day by us bhers.

    I guess I'm just ranting that IF you're gonna do blackhat seo to manipulate search engines and get to #1 in the serps, obviously you're kick ass and deserve to be there cause you know what it takes to be #1 from a link-relationship perspective. If you DO make it to #1 though- make sure that your content deserves to be there too.

    Crap gets indexed all the time. "Unique" is in the eye of the beholder...or rather the website visitor. Check out this video by Brian Eisenberg, maybe he explains it better than I do:

    http://www.youtube.com/watch?v=bZ65KZb7xag
     
    Last edited: Oct 15, 2011
  12. phpbuilt

    phpbuilt Jr. VIP Jr. VIP

    Joined:
    May 16, 2011
    Messages:
    1,650
    Likes Received:
    5,208
    Occupation:
    $ from websites I own.
    Location:
    putting monkeys in paypal
    I think you're confusing "being indexed" with "ranking". They aren't connected at all. I'd rather have 1 page indexed and ranked #1 on a 3000/mo exact keyword and getting 300% extra traffic off longtails ... than to simply boast that I've got 10k pages "indexed" but not ranking for anything.

    I have sites with dupe content provided by affiliate programs that have 10's of thousands of pages indexed that make relatively nothing, because they're competing with every other page that is exactly the same.

    In fact, I hate low quality pages on my sites. If a page is in my link structure, it's taking PR and link juice and diminishing the other pages on my site just to exist there. I've got one website with about 30 PR5 pages, 80 PR4, 150 PR3, hundreds of PR2 etc. ... if I put a stinker of a page on the site it's taking the PR and link juice from other pages and doing absolutely nothing useful with it.

    Are there sites ranking with crap? Sure, but they're fighting an uphill battle and it'll only get worse in the future. In general, sites with crap don't rank, and that trend will continue and become more obvious as time goes on. In general, sites with quality content rank better and that trend will increase as time goes on.

    Keep the spammy stuff to the web2.0s that are promoting your money sites ... low quality content is useless unless it's used to prop up your quality content.
     
  13. qwiky

    qwiky Newbie

    Joined:
    Apr 15, 2009
    Messages:
    7
    Likes Received:
    0
    Well I do need to at least get my pages indexed before I can start work on getting them ranked. I am very curious how some of my competitors are able to get so many pages ranked with 0 PR and 0 backlinks. I know they are getting good traffic by checking their Alexa ranking.
     
  14. Autumn

    Autumn Elite Member

    Joined:
    Nov 18, 2010
    Messages:
    2,197
    Likes Received:
    3,041
    Occupation:
    I figure out ways to make money online and then au
    Location:
    Spamville
    My whole business model is built on ranking generated content that is basically grammatically correct, readable but meaningless (ie. it doesn't actually make sense). I built a whole scrape / spin autoblog system but ultimately scrapped it and went back to cloaked doorways with generated content because the numbers just didn't add up - offering zero "good" content and showing a big compelling ad outperforms showing content, pretty much without exception.

    That issue of determining whether content is good or worthwhile is the single hardest task in machine language processing. All the other grammatical stuff was solved years ago (pre-internet in the cognitive psych world) but machines still can't determine what is "good" and what isn't, which is why we have proxy measures like backlinks and especially the less-gameable stuff that's coming through like bounce rate and other measures of engagement.

    I wholeheartedly disagree that showing content is a good way of marketing. I've been religious about running the numbers since I first started building porn sites in 2001 and without a doubt, getting surfers into your sales funnel asap and NOT showing them any free shit is the most productive use of your traffic.

    If your basic ads and sales pitch are good, there's no need to warm them up with free shit. You should be able to grab their attention and make them get out the credit card. If your sales pitch sucks, no amount of warming them up or getting them to trust you by offering free content is going to help you convert them.

    One of the core mistakes I see a lot of IMers making is thinking that they are free content providers here to provide a service, rather than salespeople.

    Always be closing!
     
  15. Autumn

    Autumn Elite Member

    Joined:
    Nov 18, 2010
    Messages:
    2,197
    Likes Received:
    3,041
    Occupation:
    I figure out ways to make money online and then au
    Location:
    Spamville
    Their domains won't be "true" PR0s in that they will have enough link juice coming in to be getting indexed and ranked, it just hasn't shown up on the toolbar yet.

    If they genuinely have 0 backlinks then either they have an awesome aged domain, or they're putting their backlinks through 301 redirects that don't show up in most of the backlink tools.
     
  16. kokoloko75

    kokoloko75 Elite Member

    Joined:
    Jan 1, 2011
    Messages:
    1,628
    Likes Received:
    1,935
    Occupation:
    Design director
    Location:
    Paris (France)
    Try to ping all URLs, then to create a RSS feed with all this URLs, and finally ping and submit this RSS feed.

    To ping all URLs :
    Code:
    http://www.pingfarm.com/
    To create RSS feed with your links (Warning : Google ignore if over 50.000) :
    Code:
    http://www.bulkping.com/rss-feed-generator-creator/
    To submit your RSS feed :
    Code:
    http://www.bulkping.com/free-rss-submit-online/
    This process works very well for me.

    Beny