Multivariate Testing & Cloaking – Website Optimizer, Offermatica et al

The Age Old Question about Landing Page Multivariate Testing and Cloaking Raised its Malignant Head Once Again Today. I’m going to put a few of my thoughts down on the subject, and I’m going to include some Google thoughts on the matter.

FIRST, I’ll pull from Google Webmaster guidelines – I’m not considering ‘sneaky JavaScript’ or ‘Doorway Pages’ here, just plain old cloaking.

• ‘Cloaking refers to the practice of presenting different content or URLs to users and search engines. Serving up different results based on user agent may cause your site to be perceived as deceptive and removed from the Google index.’

Now, to go a little further; Google on Google Website Optimizer and cloaking says this:

• ‘…Google doesn’t view the ethical use of multi-variate testing tools such as Website Optimizer as cloaking…We encourage constructive testing — optimizing your web pages benefits advertisers as well as users, by increasing conversions and by presenting the most desired information more efficiently.’

So far no problems… I’ll let you in on a secret – LOTS of folks do multivariate testing. Not all of them use Google Website Optmizer.

so…..So far we’ve had no problems with caching & crawling because we try really hard to stick to the guidelines provided by Google for multivariate testing, but one of the guidelines does make ones eyebrow’s scootch up a bit – I’m not saying any of the below examples pertain to anything I’ve ever worked on, I am saying it’s something I’ve thought about in a personal capacity (most SEO folks have one or a few sites they manage out of hours if they have a job, I’m not saying I’m one), and it’s something I’ve pondered on, especially regarding brands who might find themselves in a brand/business changing position.

I’m not talking about this one as far as raised eyebrows goes – it’s a no-brainer if you’re maintaining whitehat techniques::

• ‘…if we find a site running a single non-original combination at 100% for a number of months, or if a site’s original page is loaded with keywords that don’t relate to the combinations being shown to visitors, we may remove that site from our index.’

I am talking about this one:

• ‘Your variations should uphold the ‘spirit’ of your original page’s content — they shouldn’t change its meaning or people’s general perception of it.

Okey dokey then. So let’s go with an example here:
Assume I have a 5 year old home page. I’m doing a redesign based on a usability study and other feedback, and I now have 2 completely different home page designs, with entirely different messaging; one isn’t even brand based.

1. The intent is to ensure that there is a difference in perception of all variables due to the desire to test 2 radically different designs. By definition, the ‘spirit’ part of the requirement just died a horrible death.

2. As to the ‘meaning’ of the page – define meaning please…?

a. Online marketing objective (meaning)
b. Subject content and relevance (meaning)
c. Connotative association and relevance (meaning)
d. Etc

3. Now onto a consideration of people’s ‘perceptions’ of my old page & my new test pages. Is a perception based on:

a. Content
b. Brand
c. Usability
d. Aesthetics
e. Etc

Please explain how a spider (an algorithm; a software program) can determine the perceptions of my very human users. I wonder if this is more a case of (like déjà vu isn’t it?) … Is the perception based on:
a. Coding
b. Compliance
c. Perceived similarity (in terms of content, brand etc)
d. Etc…?

Assumption For the Case of Assumption:

Let’s assume the multivariate testing is comparing entirely new ideas, even brand differences in design and naming – ‘Pizza Hut’ to ‘The Hut’ – for example (that’s real), and lets’ take it further and say the ‘new’ ‘The Hut’ was moving more into schwarma territory while retaining a minor focus on pizza (that’s not real, I needed an example and that worked for this illustration).

So, are we cool? Even if we test the original (for the heck of it) against one completely different version for a full 4-5 months? Tradition holds that 8-12 weeks are enough, but if you’re changing your business model and you are in Alaska (or most parts of Canada  ) you may want to see the reactions of folks in both the hot months and the below zero months which means a longer test.

The Apparent Point
Be careful of being too strict with the rules, be careful of not following the rules, be careful of not changing what you need when you need it, and be careful of which test program you use. Don’t piss of Google et al.

The Reality
Multivariate testing is a bit of blind sport and spot for the engines when it comes to cloaking. Do it if it’s in your remit. It’s worth it. Read the guidelines with a pinch of salt, be careful, but be bold. The learnings you’ll get with a well conceived test plan will probably outweigh any potential penalties you may incur… and I stress MAY. I can tell you that so far I have never had a site banned for MV testing, and I don’t use Google Site Optimizer.

I can also say that I play by the rules, always. Yeah, laugh you black hat millionaires, I’m not as gutsy as you, and I’m also a dedicated rule follower, notoriously honest and I stand by my word. Not that you don’t.  I wish you well 🙂

.

.

Disclaimer: nothing I say here may or should be attached to any employer, current or previous. This is my personal blog. The views expressed on this page are mine alone and not those of my employer. But still, follow me on Twitter!