Expanding on Taguchi Methods in Google Website Optimizer

I've hypothesized for quite some time that the ability to run a Taguchi Factorial test in Google Website Optimzer was not only possible, but beneficial in the time that it saves when dealing with creating and managing several multi-variate tests. Having taken the time to do the research and the implementation, I am very proud of the fact that CableOrganizer.com is currently running exactly that type of test with the Google Tool at a fraction of the time of a full factorial test and a fraction of the cost of the "professional" tests out there.

Here's how:The Google Optimizer uses scripts with identifiers to "split" areas of the pages being tested in their coding. Essentially, that creates all that is included in the tested element. These are completely determined by the user, so it creates an opportunity to manipulate and build on what they have given to improve it for our uses. What we've done is created a single element which included the use of 7 individual elements. These were based on the series of elements which Paul has mentioned on several posts on the Yahoo! group somewhere in this catalog http://tech.groups.yahoo.com/group/webanalytics/group/webanalytics/.

These included, for this test: (which we, by the way, used the MultiVariate setup calculator in SiteCatalyst to develop the Taguchi recipes.)

So, yep, 7 elements with two variations each and then a control to run along linear with a traffic partition. The modeling calculator above lists 16 recipes which are to be run versus the control. This is instead of something like 350 with regard to the full factorial analysis methodology.

Thereafter, I went about creating the elements which needed to be developed and laid out for each of the pages. I created 9 images which would take the place of the three in the original, plus 6 to test the number of images and how they impact the performance. It took some fooling around to get it to work right, but, I was able to develop 16 unique recipes in Dreamweaver which allowed for me to name each of the variations in accord with the Taguchi output from SiteCatalyst.

One by one, I took the code snippet out of the Dreamweaver code view, and placed it in each of the corresponding named variation elements inside the Google Optimizer. When I was finished, it created 17 variations on the same 'area' of the site. I made sure to look into different sizes of resolutions to see how this would be affected by certain viewers, and looked through our two most popularly reported browsers (MSIE and FireFox) and clicked the last things into place. By 4am, I was sick of looking at the page and tired of adjusting tables and alignment, but I got it right and hit the button. Having alloted 50% of the traffic for the test, it was neat to finally see the numbers start to roll in a couple hours later over coffee at the office.

According to the Google AdWords Testing Calculator

(available at https://www.google.com/analytics/siteopt/siteopt/help/calculator.html), which was a tool brought to my attention by Robbin Steif from LunaMetrics when she was training me on multi-variate testing and conversion science, this test, to perform based on the inputs and the methods of calculating proxies internally, will collect and distribute valid results within a weeks time. As a safety and to reduce the level of doubt, we'll let it run itself all the way through, but it will still only take two weeks at the most to produce a final result. This is a great thing for a small/medium business with moderate to light traffic on the site. If, for no other reason, it helps to level the playing field with companies running tremendous traffic and the luxury to afford having an outside agency perform these experiments.

Maybe you wonder what the value is in something like this besides the ability to say, we did that, or why does this even matter. That's a fair statement. For me, it was necessary to do this because we need to be able to get value out of everything we do. For me to be able to test how product descriptions were presented in the earlier pages in our navigation paths, it meant being able to test and adjust several elements all at the same time without losing customers to a page that was schizophrenic in its presentation. You see, you can't test things like that if you are doing full-factorial analysis on say a category page or an intermediary for the landing page-to-goal. It requires that you have control over several repetitive elements simultaneously and meant that the Google Tool was a freebie we'd have to pass on unless we could make it work. Having said that, I'm thrilled that I gave this a try, the sense of satisfaction is well worth the toil which went into mapping and preparing this highly complicated test.

If you have questions or require more information on this subject, I would be glad to help if you would be so kind as to send me an email. I will respond within a reasonable time frame, so long as doing so doesn't place me in a compromised position with how I spend my time on the clock.

Be sure to check out the site too, CableOrganizer.com is always working on some wild experiments and testing out new uses for analytics principles. Maybe you could get some good ideas.