Infinity Testing combines predictive analytics and generative AI technology to generate and test thousands of on-brand message variants that evolve over time. Previously, brands had to test a 50/50 split during both the Exploring and the Optimizing phases where 50% of messages were the brand-created inspiration message and 50% were AI-generated messages, even after seeing clear revenue gains from Infinity Testing.
Now, once a flow reaches Optimizing Phase, brands can fully leverage AI for all messages to maximize ROI and revenue. A Synthetic Control replaces the original inspiration message with a top-performing AI-generated message as a baseline for comparison, ensuring brands get the highest possible return on investment. Synthetic controls will only be applied to Infinity Tests in Automation Flows.
Watch the short video below for a look at how Infinity Testing for automations works and for a refresher on the Exploring and Optimizing phases.
In this article, we'll discuss the details of Synthetic Controls for Infinity Testing in automations.
Key Information
During the Exploring Phase, 50% of messges sent are the Inspiration, or control message that the brand generates. The other 50% of messages sent are AI-generated variants.
Once a flow reaches the Optimizing Phase, 100% of messages sent will be AI-generated messages. This breaks out into approximately 20% being the Synthetic Control (a top-performing AI-generated mesage from the Exploring Phase), while 80% will be new AI-generated variants.
FAQs
Will a Synthetic Control only be added once the flow reaches the Optimizing phase?
Yes! A Synthetic Control will be added to a flow few days after moving into Optimizing. This allows us to have strong confidence in the lift from AI before moving to 100% AI-generated messages.
The Synthetic Control isn’t my actual Inspiration/original control message though, so how can I have full confidence in the data?
Before we add the Synthetic Control message, we compare it’s results to the inspiration message. This allows us to understand its performance in relation to the original inspiration. For example, if the selected Synthetic Control message has a 5% MCVR and a 25% MCVR lift compared to the original inspiration message then that becomes the new baseline. Therefore, as we continue to test, we compare against those metrics. If new AI messages have a 6% MCVR, then we can say that it has 50% lift against the original control.
Can I change the split in the Optimizing phase from 0/100 to something else? Like 25/75?
At this time, brands cannot customize the split in the Optimizing phase.
What happens if I edit a flow and it moves back into the Exploring Phase?
If a flow moves back into the Exploring phase, then it will move back to 50/50 split (50% of messages sent are the brand-created inspiration message/50% AI-generated variants) so that the AI model can continue to learn. Once the flow eventually moves back to Optimizing, then the Synthetic Control will be added again.
If we’re sending 100% AI messages, how can I know if AI is still providing lift?
With a Synthetic Control, we replace the your current inspiration message with a top performing AI-generated message and use that as the new baseline control to compare future lift against. Approximately 20% of the sends will use the new AI Synthetic Control message, and ~80% will be new AI Test Messages. As a result, we’re able to compare against the control while still sending all AI messages to maximize your revenue and ROI.
How is the Synthetic Control selected?
The AI model selects one of the top performing AI-generated messages from the Exploring Phase.
Can a Synthetic Control message ever change? If yes, how often?
Yes. As time goes on and performance changes, Postscript may adjust the Synthetic Control to ensure that we’re always aiming to maximize lift while maintaining a comparable to the original control. There is no set cadence that a new Synthetic Control message will be selected as it is related to a flows performance.
Get Support
Have questions? Please feel free to reach out to our wonderful Support team at support@postscript.io or via live chat. You can also submit a support request here!
Need ongoing channel strategy guidance? Please fill out this form and we'll connect you to one of our certified partners.