Upcoming Buzz in Google Website Optimizer

Good Evening,

As a portion of the people whom I deal with on a daily basis know, I am a test-happy web analyst for CableOrganizer.com in South Florida, US. I am constantly trying to find new ways to manipulate and blow the gears off existing technology for its improvement. One of the ways in which my job allows me to play with this is the free tool offered by Google called the Website Optimizer. It is a analytics tool (which can be attached to all the other Google goodies) with which to perform multi-variate and split (A/B) testing of page elements etc. on pages.

My initial foray into this multi-variate testing world brought me to this tool and a Beta-test through Robbin Steif at LunaMetrics in Pittsburgh, PA. Through a series of conversations and e-mail correspondence, Robbin was able to help me understand the principles and relatively complex math and ideas behind the tool. I couldn't help it, two weeks into the conversations I started to think about its limitations and what I could do to make its limitations our strengths.

With that, I set out with a handful of wrenches and poured them into a high-performance engine. (no pun intended). What I came up with is this:

1. Although a tool set with the idea that the tests must run a full-factorial analysis, when paired with the Multi-Variable framework available in Omniture's SiteCatalyst, you can developed and test with Taguchi Methods...how you say? That's a future post, and trust me, its a scorcher.

2. Proxies can be excellent indicators to conversion. By this I refer to using multiple pages tagged as a success metric within the Google Website Optimizer. If you are testing a landing page and you are an eCommerce company, chances are, doing a full factorial test with a single conversion metric page(success or goal in the optimizer interface) will limit your ability to test efficiently in one of two ways: A- you can only test limited elements if your page traffic isn't phenomenal and the test will still take quite some time. B - the effect will not be fully realized unless you are able to show that your page metric is a true indicator of conversion. (so you end up using proxies anyway, so why not stretch the limits and measure the ripple).

3. While not an ideal tool, it is free, and it can be manipulated with a little work to do nearly anything. You can pass elements of all kinds in and out between the scripts and the scripts determine the element. Another post on this will be available after we have successfully completed the tests we currently have slated for CableOrganizer.com this month.

4. Although the test doesn't produce a final result until the clear winner has achieved a certain level of successes and the sample size is statistically valid, if you create a clearly different set of variables your optimal choice should be clear.

Website Optimizer is great for creating a series of internal and personalized best practices with which you can rewrite your important landing pages and make an immediate impact. Using the tool is relatively simple and can certainly benefit anyone who can use it. I would be glad to help anyone who has an interest in making this tool work for their needs or I can certainly speak as a reference to the competence and skill of Robbin and Taylor at Lunametrics.

Please feel free to contact me through posting and I can get you information on our methodologies.