Back to Glossary

A/B Testing Defined

A/B testing, also known as split testing, refers to a randomized experimentation process wherein two or more versions of a variable are shown to different audience segments at one time to discover which option performs better. A/B testing is a popular marketing technique used to optimize push notifications, emails, SMS, in-app messages, websites, ads, and more. Using this technique in your messaging strategy can help you align your communication with user expectations and gain insights that can be replicated in other campaigns.

In the OneSignal dashboard, you can use the A/B testing feature to test out different message elements to discover what content resonates most with your target audience. Testable notification elements include the message title, subtitle, content, images, icons, emojis, badges, sounds, logo, launch URL, and CTA button. In addition to testing individual elements, OneSignal also offers advanced A/B testing, allowing you to test up to 10 variables simultaneously.

How to Use it in a Sentence

A/B testing can help you determine if your audience is more likely to engage with a percentage-based or dollar-based promotion.

ab-testing-optimize-user-engagement

How can it be used for mobile apps?

A/B Testing allows you to test two or more variations of a message, to identify which message variation solicits the best response from its recipients. Running these tests can ultimately help you improve your message open rate and click-through rate, leading to greater overall app engagement and user retention.

Common A/B Testing FAQs

Yes! Our data shows that on average, A/B testing leads to a 15% improvement in message engagement.

With the right messaging provider, A/B testing is simple. You don't need any experience to get started — all you need to do is create the test and learn from the results. To learn how to create your first A/B test in your OneSignal, check out our step-by-step A/B testing instructions and log in to your OneSignal account to get started.

Testable message elements include your message title, body copy, subtitle, images, icons, badges, sounds, logos, launch URL, and CTA button. Learn more about how to execute more advanced A/B testing strategies.

Yes! All OneSignal plans have access to A/B testing (including our free plan!). Advanced A/B testing is available on our Professional and Enterprise Plans.