A/B testing stats not directly comparable

Highlighted
All Star

A/B testing stats not directly comparable

In CC's built-in A/B test feature, you only get a full data set for Test A. Bounces, unsubscribes, and spam reports are NOT supplied for Test B. And since CC calculates the open rate automatically as "opened" / "sent minus bounces," the open rate for Test B is artificially lower (because B test doesn't count Bounces). Am I missing something? Is CC aware of this problem? 

Tags (2)
2 REPLIES 2
Highlighted
Moderator

Re: A/B testing stats not directly comparable

Hi @Brownswiss 

 

Thank you for reaching out to the Community about your email! I can understand how important it is to view your reporting for a sent campaign, especially for one using our A/B subject line tool. Though your most recent campaign did not have a large enough difference in opens to trigger a winner, I was able to see the full data for both subject lines.

 

2020-03-11_17-12-35.png

 

Where you not able to see this information through the separate available reports on your email details page? I also was unable to see any decrease in opens. But if you can email us at social_support(at)constantcontact(dot)com with your username and any examples you may have, we would be happy to continue troubleshooting for you.


Frankie.P
Community & Social Media Support

Did I answer your question? If so, please mark my post as an "Accepted Solution" by clicking the Accept as Solution button in the bottom right hand corner of this post.

You're not alone on your email marketing journey. Connect with a Marketing Advisor to see how they can help you develop the right strategy for your business and get the most out of your Constant Contact experience.
Highlighted
All Star

Re: A/B testing stats not directly comparable

Thank you! I did see the results as you indicate, but hadn't realized that bounce rates, etc. were available under "Sent" dropdown. Thanks again.