by Doug

04/06/2012

Internet Marketing

Google Content Experiments: A look under the hood.

Saying goodbye to GWO

So, you may have received an email from the GWO team with the opening line ‘We’re saying goodbye to Google Website Optimizer.’.  That was one punchy way to end a week!  The blog post released at the same time contained the usual slick marketing video we come to expect from Google product launches:

<iframe width=”560″ height=”315″ src=”https://www.youtube.com/embed/TGrujIh2H0I” frameborder=”0″ allowfullscreen></iframe>

The marketing is slick but what about the product?

Things we like about Content Experiments (CE):

  • Very simple experiment setup.
  • Good usage of GA goals is essential for success more than ever
  • No referral information lost on redirection – including dynamic parameters like gclid.
  • Google Analytics Premium users will see fresh experiment data very quickly.
  • Will unsampled CE reports be available for GAP users?

 

Things we’re slightly concerned about:

  • Using more goals for experiments will help users rapidly hit the goal limit.
  • We’d recommend using a separate testing profile
  • 5 variation limit – this is a big hindrance at this time.
  • Large volume sites afforded the use of many variations – GWO had no limit…
  • Non-GAP users will wait longer for experiment data – this is risky when tests could be costing or generating a lot of $$$/£££.
  • Lack of MVT…or is there!?!

We’ve tested three types of experiments:

1) Simple A/B with a redirect.
http://www.conversionworks.co.uk/services/

Setting up a vanilla Content Experiment couldn’t be simpler – we just followed the on screen instructions and the test was live in minutes…

Experiment setup screen shots would normally go here…help yourself, it really is easy!

I need to test more variations though – 5 variations is too few for a lot of our larger clients. Another concern I have on this type of experiment is the risk of causing any ‘duplicate content’ SEO issues. Should variations be excluded from the Google index by meta tag or robots.txt? I could put all experiment variations in a /test/ directory and then deny access to all bots on that folder.  I’ve put this question to Google and look forward to some clarification.

2) Simple A/B with query string parameters
http://www.conversionworks.co.uk/

This experiment is slightly more fruity. Rather than redirect to a new page, I redirect to the same page with a decorated URI. I’m using javascript to handle the content changes (look at the menu – ‘How it Works’ vs. ‘Our Process’ – a very contrived test). This poses less SEO risk I guess but adds page complexity. This is almost like a single variation MVT. The content switch could be handled by server side code too and results in quite major test variation differences. Powerful but (again) limited in variations. Notice that the CE script has been modified to do the redirect conditionally to prevent infinite redirections:

if(document.location.search.indexOf('menu=')<0){utmx('url','A/B');}

3) Hacked MVT in A/B framework (beta!)
http://www.conversionworks.co.uk/ce/

This test is a hack that is very much in beta stage right now but it’s worth sharing nonetheless.  The question is, ‘What if I wanted to test a common header, footer or menu across all my site pages?’ CE doesn’t make this so easy. I don’t want to have 5 different variations of all pages on the site. This is where we would us an A/B in the MVT framework in GWO. In CE we have to use and MVT in an A/B framework! Here’s how.

Take the CE script and change the last line to remove the redirect functionality:

<script>var combination = utmx('combination');</script>

We can now consume the combination variable and decide how it changes our page content.

We like the new ga_expt.js library. It looks like there are MVT methods in there already….hmmm!

if(combination==0){jQuery('a').attr('href', '/ce/');}
if(combination==1){jQuery('a').attr('href', '/ce/1');}
if(combination==2){jQuery('a').attr('href', '/ce/2');}

So, this test is also quite contrived, it just changes all the links on the page in a crude fashion – just proves the point though.

Goal types are a little limited. Quite often we want to test to reduce bounce so I’ll have the classic ‘pages/visit > 1’ bounce goal. Shame I can’t use this on a CE test though. So, I use an event driven goal using this script to capture clickage:

jQuery('a:contains("How It Works")').live('click', function(){
  _gaq.push(['_trackEvent','Testing','How It Works','How It Works Click',0,true]);
  setTimeout('document.location = "' + jQuery(this).attr('href') + '"',500);
  return false;
});

It would be even nicer to be able to use hitCallback in this scenario…

There’s our three experiment types so far.  CE is not the same as GWO.  In many respects it’s a LOT easier than GWO but the current limitations mean that whilst easier, it’s also less powerful.

This is early days though.  The initial signs of polish are good.  Watch this space…

 

UPDATED 6 June 2012:

Remember to exclude the utm_expid and utm_referrer parameters from your Google Webmaster tools configuration.

Also, on the subject of SEO, using canonical links in your test pages will be a big help to Google in understanding what version of the content is the original.

Additionally, have you noticed the date picker is disabled on the CE report?  You can create an advanced segment based on visit date to segment results by day/week/month if required – a great idea by Phil!

Comments

Leave a comment

Your email address will not be published. Required fields are marked *