As media costs rise, everyone is looking for ways to optimize their campaigns in order to be more cost efficient and increase their return-on-ad-spend (ROAS). But it can be expensive and time consuming to gather the data-driven insights needed to optimize your campaigns.
So what’s the “silver bullet” of ad optimization? Rapid testing cycles that provide more insights while reducing ad testing costs. Unfortunately, it’s not as easy as just cutting spending and deciding to run tests for a few less days. There’s a catch: the tests still need to produce statistically meaningful insights.
So how do we strike that balance to achieve quicker testing cycles and robust results? By using proven, structured testing frameworks like “fractional factorial testing”.
Fractional factorial test design provides a huge competitive advantage because it allows you to accelerate your learnings while minimizing media costs, ultimately helping you complete optimization cycles more quickly (AKA: saving money while increasing revenue.) It’s a major part of why we’re able to multiply customer acquisition growth, profitably, for our clients. And if growing quickly and profitably is one of your goals, it’s something that’s critical for you to understand, too.
Let’s go through an example of this testing framework:
Step 1: Design The Test
When structuring your test, you want a format that allows you to accelerate your learnings, so you can 1) get more insights for less money, and 2) go through your optimization cycles more quickly.
Take the above example: We are interested in learning how all fifteen ad & audience combinations perform, but we can learn this without testing all fifteen combinations. Instead, we structure our test so that we spend on the green boxes, and we don’t spend on the blue ones.
This provides us with a very clean and clear understanding of which ad(s) perform best.
And also a very clear understanding of which audiences perform best.
We could, in theory, run tests on every cell — but that wastes a lot of time, and most importantly, money. Instead, we run tests on the green ad & audience combos until we reach statistical significance. From there, thanks to our fractional factorial test design, we can easily infer with a high degree of confidence what performance likely would have been on the blue boxes. This empowers us with twice the insights, while only having to invest half the media spend we otherwise would have needed for these learnings.
Step 2: How to Analyze For Maximum Impact
For this example, let’s assume a Cost-per-acquisition (CPA) target of $100.
Once we have our test results, we can begin to look for clear opportunities
- CPA opportunities: In the example test results above, if we look vertically at the ads that ran to Audience 1, we can see that Ad 3 has a CPA of 50% less than Ad 1, so that’s a significant improvement and something we can capitalize on moving forward.
- Audience opportunities: Then, if we look horizontally at all the audiences that ran Ad 1, we can see that Audience 1 and Audience 2 met the CPA target (and Audience 2 even shows an improvement over Audience 1), so it’s likely that focusing on these two audiences can help drive some quick wins.
3.Inferred Opportunities: Now for the most valuable part! By leveraging our fractional factorial test design, we can also infer how the blue cells likely would have performed had we invested media dollars into them. Because Ad 3 performed about 50% better than Ad 1 when tested on Audience A, we can infer that we’ll see about a 50% improvement with Ad 3 across the other audiences as well. In the same way, we can infer that Ad 2 will likely be more expensive than Ad 1 across the board, so it is not worth spending more against Ad 2 since we can already understand what future performance. Here’s what it looks like:
With these insights, we can immediately go and spend on ad-audience combos that we didn’t have to waste money testing — with confidence that we’ll get a very good result.
Repeat this process a few times and you’ll very quickly find that your speed of learning is months or even years ahead of your competitors — compounding into a huge competitive advantage.
Case Study: Scaling With Structured Testing
The proof, they say, is in the pudding — and the above figure shows how we scaled a client in a new channel, while bringing their CPA ($150) down below their target of $125 in just 60 days. This dramatic drop in CPA and increase in volume is a direct result of the “educated inferences” afforded by structured testing, saving significant time and money on a campaign.
Here’s another client we were able to leverage this testing framework for to cut their CPA from above $600 to under $100 in just 90 days. Their target was $125, and the first test actually came out to around $600 CPA — the point at which many media buyers would have given up. But by leveraging our proven testing process, and continuing to run optimization cycles within the framework, we reached their target and were able to then scale up their volume significantly.
