isn't a/b testing considered cloaking?

My company runs a lot of A/B tests (sometimes A-G) we may have 5 versions of the homepage, or 5 versions of the product pages for example. However the spiders always get bucket A no matter what. We never show them alternate tests. This could be considered cloaking in the eyes of google -- your human visitors get a different experience than the search engines.

I'm wondering what the time limit threshold is before we could actually get into trouble with this. Any major internet company does A/B testing, so I'm just curious what you guys think. If the tests ran for 6 months I would think it could actually impact our rankings...but a few weeks...probably not a big deal, as long as eventually spiders got the same experience.

Unless Google Analytics realizes you are simply A/B testing and doesn't penalize you?

Or maybe this a trick some marketers use to get their Adword ads past the filters?

You can do split testing from Google analytics and it is so widespread that I don't think Google frowns on it.

Anyone smart knows that split testing or multivariate testing is an absolute must to maximise conversions. If the intention is simply testing the best version of a page then I think Google is smart enough to see what you are doing.

Just my 0.02 cents

It's only cloaking if you are serving different content to google than what you are serving up to users. Multivariate (I love saying that word) and A/B testing should randomly serve a different version of your page to visitors but that is not the same as hiding the content from Googlebot. To avoid issues, use the rel="canonical" on your test pages to let G know which is the real page to display in results.

There's a post about this very thing on Google's Webmaster Central from 5 days ago.

Check it out: http://googlewebmastercentral.blogspot.com/2012/08/website-testing-google-search.html