A/B Testing Your Automations – Postscript

Tip: Select any of the images or GIFs in this article for a closer look.

A/B testing is a powerful tool you can use to help ensure you send automations that'll see high customer engagement.

How A/B Testing Works

A/B testing is a go-to marketing tool to take the guesswork out of which products, messages, or automations will be well received. By testing multiple options, you gather useful data that will help you make informed decisions.

Here's how it works:

Create two (or more) different automation messages.
Compare your results. You can see which message has a higher earning per message (EPM) or click-through rate (CTR). That message is your more effective automation.
Send that effective message to the rest of your customers.

Why You Should A/B Test Your Automations

You can get creative and test out different messages. Include images or GIFs, change up message copy, try out different timing intervals, and offer different calls to action (e.g., different promo codes, free shipping offer, etc.).
Learn which messages your subscriber base engages with and apply those learnings to future automations.
Identify successful automations and use them for your larger customer base with confidence they'll perform well.

Create Your Test Automations

Select Messaging in the side menu of your Postscript dashboard, then select Automations.
Select Create Automation in the top-right corner of the page, then select Custom Automation.
Give your automation a Name. Remember you will have multiple, so create a naming convention that will make it easy to tell the various groups apart (e.g., A/B Test Group 1, A/B Test Group 2, Welcome Series Test Group 1, Welcome Series Test Group 2, etc.)
Select Yes from the User Filter drop-down.
Select A/B Test Group from the first drop-down.
Change the following drop-downs to reflect the automation group number and the total number of groups. For example, Group 1 out of 2 Groups or Group 3 out of 5 Groups. You can create up to five groups.
Select Save & Continue to Messages in the bottom-right corner of the page.
Now it's time to customize your message.
- Change up the message copy to see if different personalization strategies (e.g., using the subscriber's first name, including product links, etc.) will produce higher engagement rates.
- Trying out one group with a GIF or image can help you see whether your subscribers engage more with SMS (no image/GIF) or MMS (image/GIF) messages.
- Play with the timing of the automation messages to see if it impacts customer engagement.
- Offer different coupons or promotions to see which gets engagement. Higher discounts, mystery offers, or free shipping are great examples.
Save your automation.
Repeat these steps for each test automation. Remember to change the automation name, user filter, and message for each.

Once you've created your test automations, you will need to enable them.

Locate your automations on your Automations page. You can scroll to find them, or use the search field to filter the listing.
Select the Enabled toggle toward the middle of each automation line.

Analyze the Performance Data

With automations, you'll want to wait until you've sent 500-5000 messages (depending on the number of subscribers you have) before you look at the performance data. This allows you to get a good sample of engagement behavior.

Select Automations in the side menu of your Postscript dashboard.
Locate your automations. You can scroll to find them, or use the search field to filter the listing.

When it comes to determining the effectiveness of one automation over another, ask yourself these questions:

Which automation has a lower unsubscribe rate?
Which automation has a higher click-through rate (CTR)?
Which automation has a higher conversion rate?
Which automation has higher earnings per message (EPM)?

For the automations pictured above, our Abandoned Cart Test Group 1 automation performed better overall.

Tip: If your automations contain more than one message, you can check out how each message in the series is performing by selecting the automation name.

What to Do with the Data

Once you know which automation performed better, you can look at the characteristics of the message.

Did it include an image or GIF?
What type of offer did you include? Was it a percent or amount off, a mystery discount, or a product-with-purchase offer?
Did you personalize the message with the subscriber's name or a link to a specific product?
How many messages did you include in an automation series?
What timing intervals did you use for your messages?

From here, you can clone the more effective automation and use it as your primary automation (e.g., abandoned cart, welcome series, customer winback, etc.). You can also apply these learnings to future automations that you build for this subscriber list.

Get Support

Have questions? Please feel free to reach out to our wonderful Support team at support@postscript.io or via live chat. You can also submit a support request here!

Need ongoing channel strategy guidance? Please fill out this form and we'll connect you to one of our certified partners.