SEO Testing – Including SEO in Test Strategy

I recently that the honour of presenting at as a keynote speaker. I am delighted that no-one felt the overwhelming need to leave during my 50 minute discourse, and while I’m sure the fact that I was handing out money played a small role) ‘just click ‘back’ to get back to the post’, I have to say it was a pleasure to have such an engaged and interactive audience!

That said, I wanted to mention how I sneakily included some SEO conversion testing into the presentation, particularly because the test is rather unusual as far as I know, and requires a higher level of dedicated SEO knowledge than I think it fair to attribute to most of the eMetrics attendees. I actually don’t know of anyone else who has tried it – and in typical test fashion, I have repeated the test three times, one is running currently, and the results of the first 2 were highly similar – significantly so in boosting both SEO traffic (I’m assuming SERP CTR) and SEO unit conversion rates.

Data Research Leading to Hypothesis (1):
PPC drives a tremendous amount of traffic during offers and campaign periods, and SEO traffic appears to be somewhat cannibalised. Can we alleviate PPC costs by ensuring that the SEO results show the offer for the duration of the campaign?

Hypothesis (1):
By reflecting the campaign offer in the SEO title tag results we will: increase SEO traffic and decrease campaign attributable PPC costs, without negatively impacting total forecasted search sales.

Test 1: 4-6 Week Duration (Peak Season):
What We Did:
~ We swapped out the original title tag with one reflecting the website wide $40 off offer (20% off).
~ We did not change the meta-description wanting to minimize the potential impact of controllable variables.
~ We pushed the changes and the xml sitemap (Google) the day the campaign went live.
~ We reversed the change a few days ahead of campaign-end based on our known/anticipated crawl rate – with a safe confidence interval – with the assumption that the page would be recrawled and indexed with the original title tag before the campaign ended. (Our timing was well researched and worked out fine).
~ We work in a CMS, so those types of changes are very easy to make considering it is a large corporate environment (anyone who has worked in such an environment will know the types of challenges and time constraints inherent to making quick changes to a site).

IMAGE – SEO Test 1: Title Conversion Testing (sorry, wp is being ridonkulous this evening – just click your ‘back’ button to get back here!)

~ SEO traffic increased just over 100%
~ SEO units sold increased 150%
~ PPC traffic and units cannibalized slightly, but the net was highly positive.

Post-test Analysis:
~ While the SEO test was underway, we also ran supportive/acquisition-based PPC adcopy tests for the duration of the campaign and found that ‘20% off’ had a higher CTR and conversion rate than the ‘$40 off’ messaging, so we decided to apply that to the next SEO test.
~ With such a strong result for the title only change test, we decided to try changing both the title and the meta-description for the next test.
~ Once again, the test would occur during a website wide promotional offer period, so we waited…

Test 2: 5-7 Week Duration (Off-peak):

Hypothesis (2):
By reflecting the campaign offer in the SEO meta-description AND title tag results we will: increase SEO traffic and decrease campaign attributable PPC costs, while positively impacting total forecasted search sales.

What we did:
We swapped out the original title tag and meta-description with one reflecting the website wide 20% off offer. We followed the same process for publishing, pushing xml sitemaps, and reversal. Again our timing was well within required/predicted limits.

IMAGE – SEO Test 2: Title + Meta-description Conversion Testing

~ SEO traffic increased just over 50%
~ SEO units sold increased 90%
~ PPC traffic and units for this test were not impacted significanty in either direction. A difference to both pre-seo-tesiing, and test-1 results.

Post-test Analysis:
~ The PPC adcopy test was once again run in conjunction with the SEO test and netted the same results, directionally this time. We are comfortable with continuing to run PPC tests during promotional periods, and to continue to use % offers for SEO results until or unless we see significant changes in PPC test data.
~ We were unconvinced of the impact of the meta-description change, so we will retest.
~ We are currently running a 3rd test with yet another copy variable change – ‘>’ instead of ‘|’ in the title tag. Campaign is still live and test is still running. (The arrow vs pipe test is being run elsewhere as well, results will be analysed separately, and as a whole.)

Hypothesis (3):
By reflecting the campaign offer in the SEO meta-description and title tag results we will: increase SEO traffic and decrease campaign attributable PPC costs, while positively impacting total forecasted search sales – repeating test 2 results.

IMAGE – Third/Current Test is this….

Enjoy! And let me know if you have any questions.


Please note: These tests refer to only one of our product pages, and result for other products/pages or offerings may be different for numerous reasons. The content on our primary destination page for these tests changed very little as the product page is a core SEO resource. Offer information was added while other content remained standard. We (SEO) took a risk on losing some generic term rankings (we test those over longer periods), but as the test was for such a short period of time, we assumed the impact to converting traffic would be minimal. Our intent was to make first-use of the organics to support web based initiatives to ensure that search engines users would see the same results in both the paid and organic results for as long as we could realistically maintain them risk and liability free. While it is understood that organic results may not always lead you to the page or information you want, as a senior marketing manager for a corporate entitity with values including ‘Integrity Without Compromise’, and ‘Delight Customers’, I do not take chances that may negatively impact our user’s experience on the search engines, or secondarily, during their first-experience with our brand.

Spread the word:

We'd be honored if you'd help support SEMI by Stumbling us or voting for us on Sphinn.

Stumble it!

7 Comments so far... perhaps you would like to leave one?

No comments yet.

RSS feed for comments on this post. TrackBack URL

Leave a comment

You must be logged in to post a comment.