Marketers understand the value of using metrics to gain customer insights, but often have a limited background in data analysis. This can have a profound impact on the accuracy of information used to make important decisions.

Digest...

There’s no need to turn a marketer into a data analyst or vice versa, but instead, let’s cover some very important ground first before any testing begins.

 

Question #1. How are we using quality assurance to mitigate risk?  It stands to reason you want to address data accuracy and appropriate metric interpretation before you begin testing.

And quality assurance (QA) is a key step in doing that.

 

Question #2. How will results vary? Would you want to make a multimillion-dollar decision knowing there is a good chance you could be off by a fairly large margin?

Probably not, and this is why understanding how variance between your metrics platform and the true value is important.

 

Question #3. Do we know how the metrics we use are calculated on our platform?   Misinterpretation of metrics is epidemic in online testing – and it’s often preventable.

I say this because the accuracy of interpreting information is generally driven by an understanding of it.

 

Question #4. How can we avoid placing too much faith in one calculated metric?   

I often talk to people who use average time on page as a measure of engagement and they put far too much faith in it. The problem with this is from a theoretical standpoint, any average is highly susceptible to outside forces because an extreme outlier will skew the average value.