Why is there no A/B split testing?

Member

Why is there no A/B split testing?

Why can't we do A/B split testing in our emails so we can test things like subject lines and offers?

 

 offers this,  offers this...every other ESP offers this basic functionality. 

 

Many of us have been asking for this feature for over 10 years now, and all we hear is, "it's something we're looking at."

 

Please, stop looking and implement it. Honestly, I'm happy to bring over pizza and beers for your developers and I'm sure we can have a working beta version completed in an afternoon.

 

 

15 REPLIES 15
Super Participant

Those tools are much more manual and painful with this vendor. You can expect to have references to competition removed from your posts here.


P. S. . Changes take years here and they'd only mess up something like that anyway.
Moderator

Hi @richflyte

 

You pose a good question that I do not have the answer to at the moment, but I am reaching out to our teams here to try and get more information for you. As soon as I receive a response I will post what I find out in a reply to this thread. To @Clique_Marketing's point, we do remove the names of other ESP companies from posts in the Community when we come across them. We try to respect our competitors and remove mentions of them (regardless of whether they are negative, positive, or neutral) as this Community is for the discussion and support of Constant Contact products and services. However, I do have a copy of your original, unedited post which I am sharing with our product teams so that they can see your post as you intended it to be seen. 

 

 

Natalie B.
Social Media Marketing Manager

Hi @richflyte

A/B Testing is a feature we continue to evaluate on our roadmap.  Our past research has indicated that multi-variate testing is too complex for our target customer.  However, subject line testing is a feature we'd like to offer to help our customers be more successful. The priority of new features like this is greatly influenced by our customers.  While we would love to work on every feature request, especially a cool one like A/B Testing, we only have the capacity to work on the most requested features.

   

I appreciate the feedback and encourage you and others to continue requesting this functionality.  Subject line testing is certainly a personal favorite of mine and I hope it generates enough customer demand to push it to the top of our list.  I’ll be happy to have you over for pizza and beers with the developers if you can help make that happen.

 

 

Paul S

Product Manager

Paul Spitzer
Product Manager

Paul,

 

Thanks for your response. Without naming your competitors, they all seem to be offering those type of A/B split testing. I've even seen CTCT write on the importance of A/B split testing in the past.

 

While you have some beginners as customers, many of us are smarter than you give us credit for. Split testing on subject lines would be a good start, but as marketers we need these basic tools to evaluate our campaigns. 

 

Thanks. 

I've been with Constant Contact for over 10 years.  I agree that it is ridiculous not to have automated A/B testing available. It is absolutely necessary.   This feature does not have to be complicated, nor does it need to be accessible to those that do not want to use it.  Please make it happen! If I have to, I will find another vendor because my time and effort is being wasted. 

Hello @MichaelM6968. We appreciate you being a long-time customer with us here at Constant Contact. I do understand how this would be a useful feature for our customers, including yourself. We will continue to send over this feedback to our product team and hopefully, this can be a feature could be implemented into accounts in the future. 


Caitlin M.
Community Manager
Did I answer your question? If so, please mark my post as an "Accepted Solution" by clicking the Accept as Solution button in the bottom right hand corner of this post.

You're not alone on your email marketing journey. Connect with a Marketing Advisor to see how they can help you develop the right strategy for your business and get the most out of your Constant Contact experience.

Michael,

  

Dont' hold your breath for this.  It was first asked for many many years ago.

 

Read the Product Managers response a few posts up.

 

You'll have to do it yourself if you want to continue to use this product.

 

 

 

 

 

Clique_Marketing - I'm not holding my breath, I'm moving on.

Hi Paul,

 

I appreciate your comments but really, A/B testing has been suggested and requested for almost as long as I have been a customer and I started in 2004.  For those long-term customers who were here before August 2013 I appreciate the comments of the clients who have commented on the reduced functionality of CC in favor of a simple product suitable for the novice user.  You mentioned in your response the "target customer". I would really like to know the characteristics of the CC "target customer". Knowing the level of sophistication of the "target customer" will help me understand your constant dumbing down of the program. If I know that you are truly designing it for the novice user it will keep me from griping so much every time your team adds a "new and improved" feature that actually takes away functionality.

Please forgive the negative tone. I write because I care about CC and want it to do well. My organization uses it almost daily and even with all its flaws and reduced functionality it still offers enough robust features to continue its use.  BUT, the steady loss of features over the years in favor of simplification has made it a less sophisticated product that is causing more frustration, generating more calls to customer service, and taking longer to produce a campaign. 

 

I would be happy to join richflyte in bringing the pizza and beer to get things moving.

 

Thanks for listening

Dave Ross

CC user since 2004

Dear Constant Contact Community Managers,

 

'Multi-variate testing too complex for your target customer...'

 

A: That's a fair answer, but it was also given just over 2 years ago, with no updates or new articles since

B: Please consider that for many of your target audience, whether the functionality would be confusing or not (I promise I will rise to whatever nosebleed-enducing quantum mechanics you provide), we're still left with assignments, in the form of employee goals, or outright requirements of our jobs, that leave us A/B testing manually. If it's not a functionality set you can offer in your software, this is a content gap you can fill with your great technical writing, and we'll use your advice (I promise!).

 

We're definitely going to conduct A/B testing, but, for instance, because there is nothing in your KnowledgeBase beyond the "it's on our roadmap" answer (I used to work in software, I've given this response countless times), there's no thought leadership on how to do it without built-in functionality.

 

I found such thought leadership on rival platforms, and am left taking their advice (for manual A/B testing) and implementing it on your platform. I use your benchmarks in my reporting; I use your guidance on best time of day to send in market planning; I use as much of your functionality as I can pack into each email campaign; but I'm left using your competitors' thought leadership to execute my A/B Testing.

 

Please consider writing articles about:

  • what constitutes valid A/B testing principles (and what gets called "A/B Testing" but isn't)
  • best practices for ensuring quantifiable results
  • how to measure A/B test results (e.g., using the Average function in Excel or Google Sheets to concatenate results from disparate campaigns)

You provide a great product--one that would be well-bolstered by multi-variate testing--and moreover you provide great thought leadership.

 

Please consider adding A/B Testing principles to your table of contents.

 

Thank you!

 

Hi @MaureenG8

 

Thank you for reaching out to the Community with your feedback on our A/B  subject line testing tool.  I did want to first mention this tool is no longer in beta and is currently available on every 3GE template.

 

I also can definitely understand how helpful it would be to test other variables. I apologize, this is not a feature we currently provide. This is however a great feature request we have tracked in your account and submitted to our engineers! In the meantime here are some of our articles that goes over A/B testing in our system:

 

What is A/B Testing? Why It Matters & How to Get Started

Stage an Epic Battle with A/B Testing

Manual Subject Line A/B Testing and List Size

 

I hope these articles help!


Frankie.P
Community & Social Media Support

Did I answer your question? If so, please mark my post as an "Accepted Solution" by clicking the Accept as Solution button in the bottom right hand corner of this post.

You're not alone on your email marketing journey. Connect with a Marketing Advisor to see how they can help you develop the right strategy for your business and get the most out of your Constant Contact experience.

Just saw they are (finally) beta testing a/b split testing for subject lines. YAY!

 

Unfortunately I'm not seeing a way to do testing on an email created by copying an old one. You have to create a new one by picking a template. I create the majority of my emails by copying my last one so I don't have to start fresh each time as much of the info stay the same.

Hello @naturalresources. We're very excited about this beta test for A/B subject line testing too! At this time, there is not a way to copy an existing campaign into the A/B subject line test and it must be manually re-created when picking a type of campaign to send out. We are collecting feedback from our customers during this beta test and I will certainly submit yours over to the appropriate team. Thanks so much!


Caitlin M.
Community Manager
Did I answer your question? If so, please mark my post as an "Accepted Solution" by clicking the Accept as Solution button in the bottom right hand corner of this post.

You're not alone on your email marketing journey. Connect with a Marketing Advisor to see how they can help you develop the right strategy for your business and get the most out of your Constant Contact experience.
Regular Participant

I too have been a CC customer for longer than I can remember.  10-15 years?

 

Anyway, I would also like to see A/B testing for mailing time.  For example, instead of making us split our own mailing lists and mailing to each portion at different times, why not allow us to select one list but give us the option to schedule the campaign for 2 (or more) delivery times in the future?  That would not be hard to implement at all.    Or, at least give us a tool to split a list in X number of equal sized portions.   

 

Oh I wish upon a star. 

Hi @NoelD6

 

Thank you for this feedback on our A/B subject line tester emails! I apologize they do not off the ability to test the times of when they will be sent out. These email templates are still currently in our Beta testing and focused solely on subject lines. We are however always looking for ways to improve our user's experience in our system and always welcome feature requests like your own. In fact, I have tracked this request in your account and passed this feedback on to the appropriate teams. In the meantime, we do offer a tool to split your contacts into equal sized portions.

 

2018-01-12_15-41-26.png

 

When scheduling your email in our A/B subject line templates, the percentage of contacts for each subject line can be set to 50%. Doing this allows you to have equal sized portions.

 


Frankie.P
Community & Social Media Support

Did I answer your question? If so, please mark my post as an "Accepted Solution" by clicking the Accept as Solution button in the bottom right hand corner of this post.

You're not alone on your email marketing journey. Connect with a Marketing Advisor to see how they can help you develop the right strategy for your business and get the most out of your Constant Contact experience.