Optimizing Mobile Sites Using Multivariate Testing and A/B Testing

By: Roberto Bell

What is Mobile Multivariate Testing?

Common methods for running controlled experiments on mobile web pages range from simple A/B testing to sophisticated multivariate testing, also known as multivariable testing. In A/B testing, one or more new versions of a page or single site element compete against an existing control version. For example, two versions of a headline might compete against an existing headline. Mobile multivariate testing, on the other hand, is like running many A/B tests concurrently, where there are multiple elements being tested at the same time. For example, two different product images, plus two different headlines, plus two different product copy text, for a total of 27 possible combinations (including the original control versions).

What's important to understand about mobile multivariate testing is that it not only shows you which combination of elements lead to more sales or pull more leads, but it reveals which individual elements impact visitor behavior vs. those that don't. For example, did variations in product image impact visitor behavior more, less, or the same as the text? Understanding how each site element causes visitors to interact with your mobile site is the essence of a test-learn-repeat process that marketers can use to synthesize new ideas and continually improve their mobile site's ability to achieve and exceed their marketing goals.

Mobile Multivariate Testing as a Process for Ongoing Improvement

The process of mobile multivariate testing unearts not only what works and should be implemented, but also what does not work and should be avoided. Every new idea, whether content, feature or campaign related, should be put to the test to determine if it helps or hurts the visitor experience. While some new ideas increase sales, others fail - sometimes substantially. But even with these failures, there is knowledge gained in what to avoid the next time. The ability to test a new idea and "look before you leap" is an unmistakable advantage that breaks the constraints on mobile marketing innovation. Only once a solid testing capability is in place, and the impact of any mobile site change able to be quantified, can mobile marketers truly optimize their mobile site's effectiveness.

What are Common Errors to Avoid When Optimizing Mobile Content?

There are five types of errors that are easy to make when running mobile multivariate tests. Fortunately, they can be avoided with some advance thought:

1. Improper factoring caused by little or no isolation of individual test changes; for example, changing a headlines text, font color, and font size, all at the same time as a mobile A/B test instead of a multivariate test. Why is this problematic? Because many changes aggregated into an A/B testing makes it impossible to isolate the impact of each individual change. For example, was it the font color and/or the text that caused the user to behave differently?

2. Running a test for too much or too little time. Stopping a mobile multivariate test early because you think you have the winning combination can increase the risk for statistically invalid results, and may increase time bias from uncontrolled events and/or conversion cycles. Running a test too long increases the risk of wasting time waiting for low-impact results and consumes test sample that could be applied towards another test.

3. Tracking or analyzing wrong metrics (KPI's). For example, measuring a KPI that is too far upstream (i.e., in a conversion funnel) from the ultimate goal, or measuring only one KPI when there are multiple indicators and/or goals that matter. There's also the risk that a measured KPI improves, but at the expense of another (perhaps untracked) KPI, or that the measured KPI is actually a bad predictor of the ultimate goal.

4. Not using behavioural targeting or audience segmentation. This means optimizing your mobile site or campaign for "anyone and everyone" by not targeting tests to include good visitors (and exclude bad visitors) and not slicing the results by meaningful segments. Why is this problematic? Because not all visitors are the same - they're at different stages of the buying lifecycle, using different mobile devices, and some may be mistakenly in the wrong site altogether.

5. Not taking action on results of the test! This could range from not making the winning changes to your mobile site, or not taking what you've learned and running another follow-on test (i.e., the essence of an iterative test-learn-repeat methodology). The risk here is that there is no momentum gained, no ongoing strategy applied, no realization of test results, and worst of all underwhelming ROI.

How can Multivariable Testing Optimize my Mobile Web Initiatives?

Multivariable testing can yield some very meaningful results in enhancing mobile marketing effectiveness. For example, we worked with a well-known online auction house to perform a series of multivariate test campaigns to understand which site factors were most impactful in visitor bidding rates. The team tested variations in elements such as mobile page layout and messaging, individual item landing pages, and calls-to-action, resulting in the following results:

* 429% increase in bidding actions
* 83% increase in product browsing actions
* 166% increase in individual lot views
* 590% increase in opt-in subscriptions

If you are looking optimize your mobile marketing initiatives, multivariate testing can and should be part of your arsenal of analytics and optimization products.

Article Source: EZINEPRIME - FREE WEB CONTENT and ARTICLES

Eric J. Hansen is the president and founder of SiteSpect and architect of the SiteSpect SaaS solution, the first and only mobile multivariate testing and A/B testing platform that helps mobile marketers improve site effectiveness through non-intrusive optimization.

Add this article to your favorite Social Bookmarkers.

Print This Article

Post Comment

Add To Favorites

Email to Friends

Ezine Ready

	EzinePrime © 2006-2007 HubPrime.com Notice: We collect personal information on this site. To learn more about how we use your information, please see our Privacy Policy.
Home \| About\| Contact Us \| PrimeLinks\| Privacy Policy \| Terms of Service \| RSS
by HubPrime™