Split Testing and NOT Jumping to Conclusions

I’ve been running a test on a site where I pulled off all of the sidebar testimonials.  Testimonials work, right?  My idea was that perhaps the sidebar being removed would increase readability and not make the sales copy quite so overwhelming.  This was a test that I was hoping would prove profitable only because I do want to clean up the copy.  However, testimonials do work.

And for the first two weeks it was showing that testimonials are effective.  Pulling them off showed a drop in conversions of 18%.  This held steady for two weeks.  And then Monday what do I find?

splittest-swing

Suddenly instead of thinking it’s in the bag with an 18 percent dip in sales, I’m seeing it dip less than 2 percent.  And keep in mind we’re talking about conversion well over 400 across both variations — so it’s not like I’m dealing with some small sample.

At any rate, the important part about split testing, and something I learned from this little experience, is to not jump to conclusions. Well it very well may be (and likely is) true that the testimonials boost conversions, I wanted to know for sure.

And at the moment, I don’t.

3 comments On Split Testing and NOT Jumping to Conclusions

Leave a reply:

Your email address will not be published.

Site Footer