The $177,843.34 Facebook Ad Test: Collection vs. Single Video

If you’re looking to immediately increase your e-commerce profits from Facebook ads, this may be one of the most important articles you read this month.

As someone who personally oversees well over a million dollars a month in ad-spend, I’m always looking for an edge.

When I first heard of collection ads in early March this year, I admit, I was excited.

And impatient.

---

Facebook Collection Ads:
Can They Achieve More – with Less?

 

Facebook claimed to have created a “special ad” – with the potential of “reducing the friction (and cost) of advertising coming from mobile phone and video ads.”

Anything to counterbalance the increasing costs we’ve been facing with Facebook ads would have been more than welcome.

This sell was based on two sound premises:

1. Mobile usage is increasing

Collections claimed to deliver a faster loading, more seamless mobile browsing experience. On releasing collection ads, Facebook cited research claiming:

For most small e-commerce business sites – mobile experience is shockingly poor – so hats off to Facebook for solving part of this common problem for advertisers.

 

2. Video is increasingly important

And, is continuing to grow as a major influencer in buying decisions.

So, I was naturally curious and went on a hunt to answer the following question…

What Exactly is a Facebook Collection Ad
and Can It Increase E-commerce
Advertising Returns?

Facebook collection ads are a mobile only newsfeed ad, which combines a video (or image) with a product catalogue of up to 50 related products to your featured video (this is the grid layout- which was the original release).

Four products (of your choice) are shown below the video, and if potential customers want to know more, clicking on your product catalogue below the video opens more choices.

Here is the example Facebook gave for Adidas:

The ability for your prospect to watch a video, while having your featured products conveniently displayed under that video, seemed like the perfect solution to make trends 1 and 2 above, more efficient.

This would have successfully achieved what Facebook was after: a much quicker, seamless, and more visual ‘discovery to action’ journey – in one single mobile newsfeed experience.

In other words, Facebook is achieving more with less.

This profit generating theory was backed up by citations from great brands with highly impressive results:

• Adidas – claiming a 43% decrease in cost per conversion and a 5.3x return on ad spend.
• Tommy Hilfiger – claiming an ROI increase over 200%, an increase of 2.2x return on ad spend, and 37% lower cost per visit to its e-commerce site
• Michael Kors – claiming a 42% drop in cost per conversion/purchase, 2.6x YOY increase in return on ad spend.
• Sport Chek – claiming a 1.92x increase in conversion rates, 46% lower cost per conversion and a 28% increase in online sales
• MeUndies – claiming a 40% increase in conversion rate, a 46% lower cost per conversion and a 28% increase in online sales

Unignorable results for sure. But, the question I needed answering was…

Do these outstanding results translate for ‘non-big brand’ Facebook advertisers?

Most of our clients have an ad-spend budget of between $50 – $6,000 per day, and they expect their advertising dollars to work very hard for them.

As someone who lives and breathes performance marketing campaigns, I’m highly skeptical when anyone talks about how well their advertising did for a ‘well known’ brand.

Major Brands have millions of dollars and years of carefully positioned association behind them. This gives them a crazy advantage (we know… because we have a few big branded clients). So – expecting similar results to your own new shoe store, for example, is simply put, unwise – from my perspective.

This is especially true if you don’t have an big budget like Adidas to optimise your mobile experience.

Facebook’s claim that collection ads were proven to be successful in reducing the friction for buyers has obvious benefits for all, and was worth the test alone.

So, the questions we started with and set out to answer were:

• Do Collection Ads outperform traditional ads for the average advertiser?
• If so, how well do they perform?
• And under what circumstances?

After 13 separate campaigns, 2.5 months and a $177,843.34 in ad spend, we got our answers.

They blew my mind.

Here’s what we did step by step:

---

The $177,843.34 Test Set Up
(Problems I Had To Solve):

Having an actuarial background (think lots of statistics and probability), I can sometimes get a little particular  about making sure any test I do gives me data I can trust.

I had to do my best to isolate the ad format only (collection and video), while keeping every other variable the same.

Given I’m dealing with multiple clients and products, across multiple audiences at any one time, this was a little challenging.

I studied my client base, and identified 13 campaigns to fit the bill:

1. All e-commerce stores
2. All had a video ad that was selling successfully (achieving a return on ad spend (ROAS) over their specific requirement, which varied between 3.5x and 6x)
3. All had mobile newsfeed traffic as one of the most profitable (if not the most profitable) driver of sales from Facebook
4. All had a sales price of the primary featured product in the video of between $179-$227 (less than $50 between the cheapest product and most expensive, to keep the sale ranges as close as possible)

Although many of my clients did use other ad formats (e.g., Facebook carousel ads with video), the common denominator for these 13 campaigns was that they all had a successful single video ad as a control for the product to be featured in this test.

As collection ads (upon first release) were designed to have only one feature video (or image), this produced the fairest test results.

The test was conducted for just over 2 months for most campaigns.

I could have used other clients and campaigns for even more data, but there were more variables that could have influenced results, so I didn’t feel they could contribute fairly to this Facebook ad test.

For this test, I made sure both the single video Facebook ad and the video Facebook collection ad were exactly the same, other than the ad format.

The ads were also tested in the same ad-set, so there was absolutely no difference between the ads other than the format.

This gave me true Collection vs. Video ad conditions:

Also, it’s worth noting that each video ad was a square video with captions as this generally produces much better results on mobile.

If you are not using this format – test it and you might just double your CTR’s overnight.

 

The Raw Data

After spending $177,843.34, and generating over $1,122,853.55 in sales, we were ready to dive into our analysis.

You’ll notice we discounted sales figures by about 25% (total sales for this experiment was actually $1,515,321.93 according to Facebooks reports).

We did this to be conservative.

Because we had ‘after-Facebook-click’ marketing (e.g.Google remarketing and email marketing), after extensive analysis, we attributed a maximum of 25% of sales to these ‘after-Facebook’click’ efforts.

At the end of the day, the percentages remain the same, however, I’m disclosing how we view these results with our clients.

As an example of this, you’ll see a snapshot of Facebook’s report for campaign one showing a spend of $15,904.81 and sales of $131,955.89.

In our calculations below, however, you’ll notice that I’ve kept the spend the same, but attributed only $97,779.31 (approx. 25% discount) in sales. Again, this is to be extra conservative on our performance to overall sales. Again, the percentages at the end of the day will remain exactly the same because we’re discounting ALL results by the same factor.

Example: Campaign 1, Raw Data Results:

 

To make up the $177,843.34 in ad spend, here’s the breakdown of spend, sales and overall Return on Ad Spend (ROAS) by campaign (sales are discounted by approximately 25%):

(Out of respect to the non-disclosure agreements I have with my clients, I’ve replaced the names of the generic campaign names)

Overall Data: Spend, Sales and ROAS Figures for Each Campaign

 

	Spent	Sales*	ROAS
Campaign 1	$15,904.81	$97,779.31	6.15 $97,779.31/ $15,904.81
Campaign 2	$4,688.56	$28,881.57	6.16  $28,881.57 / $4,688.56
Campaign 3	$19,117.55	$113,596.61	5.94 $113,596.61 / $19,117.55
Campaign 4	$8,512.10	$37,871.82	4.45 $37,871.82 / $8,512.10
Campaign 5	$4,095.15	$17,521.13	4.28 $17,521.13 / $4,095.15
Campaign 6	$2,663.22	$20,284.93	7.62 $20,284.93 / $2,663.22
Campaign 7	$3,808.06	$35,062.85	9.21 $35,062.85 / $3,808.06
Campaign 8	$25,727.04	$241,629.59	9.39 $241,629.59 / $25,727.04
Campaign 9	$35,392.41	$165,966.54	4.69 $165,966.54 / $35,392.41
Campaign 10	$2,598.94	$30,333.69	11.67 $30,333.69 / $2,598.94
Campaign 11	$4,391.47	$33,456.87	7.62 $33,456.87 / $4,391.47
Campaign 12	$3,130.33	$32,938.07	10.52 $32,938.07 / $3,130.33
Campaign 13	$47,813.70	$267,530.56	5.60 $267,530.56 / $47,813.70
Total	$177,843.34	$1,122,853.55	6.31

*Sales figures reduced by 25% to take into account remarketing and email marketing post-Facebook-click

 

---

 

Our Favourite Audiences For Ecommerce

Having managed millions in Facebook ad spend, I can say that without a doubt, audience selection can be the key difference between a losing campaign and a wildly profitable one that can scale businesses.

This is why, although we created up to 9 different audiences for each campaign, most of the budget were given to our favourite four audiences for e-commerce:

• Added to cart retargeting – visitors who viewed the primary product page (that we’re promoting in the Facebook video) AND visitors who added our product to cart (must satisfy both conditions)
• Product category retargeting (exc. added to cart) – Visitors who viewed our product and product category page but DID NOT add to cart
• Video view retargeting (love this one) – this is a retargeting list we created in Facebook of anyone who watched more than 3 seconds of a video we promoted on Facebook (we’ve done this test with 25%, 50%, AND other variations too, but for this test we kept the video view retargeting very loose to keep conditions as similar as possible)
• Product category buyers Lookalike audience (one of my favourite cold audiences) – This is a 1% Lookalike (LAL) audience of buyers who have bought at least one item from that product category, or products closely related to the promoted product

The additional 5 audiences used included:

• Specific interest targeting
• Product buyers (not product category as per the second audience above) – 1% Lookalike
• Lookalike of product category buyers not including product buyers of the main product being sold
• Product category buyers – 1-2% Lookalike
• Product category buyers – 2-3% Lookalike

---

 

Facebook Picking Winners Too Quickly
(More Problems I Had To Solve):

Facebook is notorious for picking a winning ad too quickly when split-testing inside an ad set.

In some cases, we ended up turning off the ‘winning’ ad. We did this if Facebook decided on the winning ad too early. This was designed to give both ads a fair chance of winning.

We’d then turn both ads on once we felt there was more of an even test.

However, when we felt there was a clear winner in an ad set, we ended up turning the other ad off – this is the reason why overall, collection ads ended up spending $31,908.84 more than single video ads.

Another notable point is that most of the products promoted in the below campaign appealed more to males. This naturally skewed the spend to males.

(That said, we were in for a huge surprise with gender analysis. This was a good reminder to never make assumptions, but rather, follow the data)

---

So… Which Format Won Overall
Collection Ads or Single Video Ads?

The answer surprised me.

Collection Ad vs. Single Video Ad – ROAS Analysis

Across the 13 campaigns, Facebook Collection Ads significantly beat Single Video Ads with a return on ad spend (ROAS) that was +22% higher:

	Spent	Sales	ROAS
Collection	$104,875.74	$715,137.28	6.82 $715,137.28/$104,875.74
Single Video	$72,967.60	$407,716.27	5.59 $407,716.27/$72,967.60
	$177,843.34	$1,122,853.55	+22% 6.82/5.59-1

This is why Facebook Collection Ads ended up getting more spend than the Facebook Single Video ad.

Once the test produced a clear winner, we let that ad type continue to bring in sales!

Overall Cost-Per-Sale (CPS) % analysis:

Across all campaigns, Facebook Collection Ads ended up costing -18.52% LESS to get each sale when compared to its equivalent single video counterpart:

	Cost	# Sales	Cost/# Sales
Collection	$104,875.74	3,184	32.94 $104,875.74/$3,184
Single Video	$72,967.60	1,805	40.43 $72,967.60/$1,805
	$177,843.34	4,989	-18.52% 32.94/40.43-1

 

This is highly significant!

And testimony to Facebook’s success in reaching their objective of reducing friction.

Overall Average Order Value (AOV) Analysis:

Across all campaigns, the average order value using Facebook collection ads was -0.57% less, almost negligible and too close to call.

	Sales	# Sales	Sales/# Sales
Collection	$715,137.28	3,184	$224.60 $715,137.28/3,184
Single Video	$407,716.27	1,805	$225.88 $407,716.27/1,805
	$1,122,853.55	4,989	-0.57% $224.60/$225.88

Look at this another way, if you look at the number of sales made per every dollar spent:

Collection

= 3,184 (# of sales)/104,875.74 (total ad spend)
= .03035974 sales/ dollar spent

Single Video

= 1,805 (# of sales)/72,967.60 (total ad spend)
= .024737 sales/ $ spent

The difference in ROAS looks to have been purely driven by the fact that a person is more likely to buy from a collection ad than a single Facebook single video ad!

After doing this analysis, I knew further segmentation was necessary

Assuming you have the right setup (product feed in Facebook), this is a great return for about 15 minutes work to switch over to Facebook collection ads!

---

 

How Did This Affect Profits?

I was getting more excited because I hadn’t even started drilling down yet.

Just this insight alone was significant enough for me to pay attention because…

The effect of a 22.04% increase in ROAS applied to tens of thousands of dollars in ad-spend per day would have been a dramatic increase in sales for my clients. The result being potentially millions of dollars in extra sales … all without any extra ad costs!

And the effect on profits would have been an even more dramatic 44% increase (I arrived at this figure through conversations with my clients – some indicative calculations below)!

For some clients, their individual results were better, which meant more than a doubling of profits from Facebook advertising.

I knew that clients from the 13 campaigns below, approximately averaged a 40% margin.

I also knew that they collectively averaged 5x ROAS.

So, a typical calculation of actual profits before and after collection ads would look like this:

	Spend	Sales	COGS Cost of Goods Sold	Profit B4 Ads	Profit Aft Ads
Before	$1	$5.00 $1 x 5	$3.00 $5.00 x 60%	$2.00 $5.00 – $3.00	$1.00 $2.00 – $1.00
After (22.04%+ increase in ROAS)	$1	$6.10 $5 x 122%	$3.66 $6.10 x 60%	$2.44 $6.10 – $3.66	$1.44 $2.44 – $1.00
				Profit Increase:	+44.08% 1.44/1.00-1

This was when I started getting really excited.

If I could somehow find a way to ramp up ROAS a little further to 30%, I could potentially deliver an extra 60% bump in profits.

This is because ad costs are staying exactly the same, but sales are growing:

Effect of a 30% increase in ROAS:

	Spend	Sale Value	CoGS	Profit B4 Ads	Profit Aft Ads
Before	$1.00	$5.00 $1 x 5	$3.00 $5.00 x 60%	$2.00 $5.00 – $3.00	$1.00 $2.00 – $1.00
After	$1.00	$6.50 $5 x 130%	$3.90 $6.50 x 60%	$2.60 $6.50 – $3.90	$1.60 $2.60 – $1.00
				Profit Increase:	+60.00% 1.60/1.00-1

Consider at this stage; I hadn’t even started segmenting.

If there’s one thing I know about managing ad campaigns, using Pareto’s principle (the 80/20 rule) to segment and find pockets of higher returns can skyrocket sales and profits.

That’s why ‘segmentation’ is the key to many of our most successful campaigns.

---

How Did Each Individual Campaign Do?

Comparing the Return On Ad Spend (ROAS) % change, Cost Per Sale (CPS) % change and Average Order Value (AOV) % change by individual campaign of the performance of collection ads vs the single video counterpart:

Collection Ad Performance vs Single Video Ad Performance by Campaign:

	% Increase in ROAS (Return on Ad Spend)	% CPS (Cost Per Sale) Change	% AOV (Avg Order Value) change
Campaign 1	48.07% 6.81 / 4.60	-22.35% $31.99 / $41.21	14.97% $217.76 / $189.40
Campaign 2	11.44% 6.51 / 5.84	-2.86% $36.08 / $37.14	8.25% $234.86 / $216.97
Campaign 3	-19.85% 5.41 / 6.75	8.12% $50.33 / $46.55	-13.34% $272.45 / $314.40
Campaign 4	-12.00% 4.07 / 4.63	10.31%  $55.88 / $50.65	-2.92% $227.55 / $234.40
Campaign 5	-22.02% 3.51 / 4.50	13.74% $69.74 / $61.32	-11.31% $244.61 / $275.79
Campaign 6	-24.08% 7.09 / 9.34	-0.99% $24.83 / $25.08	-24.83% $176.00 / $234.13
Campaign 7	28.11% 9.92 / 7.74	-8.38% $21.92 / $23.92	17.38% $217.38 / $185.19
Campaign 8	89.74% 12.64 / 6.66	-43.27% $19.48 / $34.35	7.64% $246.28 / $228.81
Campaign 9	27.56% 5.02 / 3.93	-15.45% $47.19 / $55.81	7.85% $236.75 / $219.51
Campaign 10	6.23% 11.79 / 11.10	2.17% $15.43 / $15.11	8.54% $181.95 / $167.63
Campaign 11	11.65% 7.81 / 6.99	-0.57% $21.08 / $21.21	11.01% $164.62 / $148.29
Campaign 12	34.52% 11.36 / 8.44	-8.20% $21.08 / $22.97	23.49% $239.41 / $193.87
Campaign 13	7.30% 5.76 / 5.37	-9.37% $36.61 / $40.39	-2.75% $210.94 / $216.91

 

Important Insights (so far):

These results were very interesting…

Return On Ad Spend (ROAS) Analysis:

• 69.23% (9 out of 13) clients increased their ROAS% simply by switching to collection ads
• Of those whose collection ads produced better results, the average increase on ROAS% spend was a clear 29.40%!
• If collection ads worked for a campaign, it produced a clear winner. One campaign (campaign 8) did 89.74% better than the single video counterpart! (to say our client was happy is a massive understatement)
• Overall ROAS% change was on average +22.04%, with a range of -24.08% to +89.74%

Cost Per Sale (CPS) analysis:

With an overwhelmingly positive response to collection ads in this test, I expected the CPS% to follow a similar pattern in the opposite direction for each campaign. And…

• Like the ROAS% results, 69.23% (9 out of 13) campaigns showed a CPS% decrease
• Three of the campaigns where the CPS% rose (campaigns 3, 4 and 5), were the same campaigns where the ROAS% was negative
• The highest performing campaign in terms of ROAS% change was driven more by a significant drop in CPS% than an increase in $$/Sale (campaign 8)
• Overall CPS was still significantly less at -18.52%, with a range between -43.27% and +13.74%

Average Order Value (AOV) analysis:

• 8 out of 13 campaigns (61.54%) showed a better outcome with collection ads
• Overall, average order value percentage change was negligible at -0.57%, with a range of between -24.83% to +23.49%

Overall, the results were extremely positive, with a high percentage of clients experiencing great results and an overall increase in ROAS%.

---

If You Haven’t Tested Collection Ads…
Why Not?

Who knows, you could end up like some of our clients who greatly benefited from the switch.

Clients 1, 7, 8, 9, and 12 (who are now expect to permanently testing Collections against other formats for mobile news feed ads) experienced an ROAS% increase of +48.07%, +28.11%, +89.74%, +27.56% and +34.52% respectively!

Results that, resulted in a huge increase in their bottom line profits.

Especially for Campaign 8, who by taking our same assumptions of a 40% margin, skyrocketed profits by an astounding +179.48%:

	Spend	Sale Value	CoGS	Profit B4 Ads	Profit Aft Ads
Before	$1.00	$5.00 $1 x 5	$3.00 $5.00 x 60%	$2.00 $5.00 – $3.00	$1.00 $2.00 – $1.00
After (ROAS % change 89.74%)	$1.00	$9.49 $5 x 189.74%	$5.69 $9.49 x 60%	$3.79 $9.49 – $5.69	$2.79 $3.79 – $1.00
				Profit Increase %	+179.48% 2.79/1.00-1

In conclusion to our test, we took away…

5 Highly Profitable Insights for Smart Facebook E-Commerce Advertisers

1. If collection ads work – it will quickly produce a clear winner… so start testing immediately! Even if your Facebook ads are marginally profitable, you could be leaving money on the table every single day that you’re not testing collection ads.

Of the campaigns where collection ads produced a better result, the average increase on ROAS was 29.40%, with one campaign doing close to 90% better. Across all campaigns (winning and losing), ROAS grew by an average of 22% on our collection ads , with results ranging from -24.08% and +89.74%.

Keep in mind, this is without any further segmentation. We used segmentation following this experiment to drive up returns and profits even further.

2. Even if a few of your collection ads fail, remember the probabilities are on your side…. so keep testing! 70% of our collection ads beat our single video ads by delivering a higher ROAS.

3. Look for a drop in cost per sale as the main driver to higher returns. On average, we experienced a CPS drop of 18.52%, with a range of 43.27% drop to a 13.74% increase. This was the main driver to higher returns on most of our campaigns.

4. Don’t expect collection ads to have much, if any impact on your average order value. In fact, your average order value may decrease (as it did for us across the board by 0.57%).

Still, the probability is again on your side, with over 60% of campaigns showing people purchasing more per sale with collection ads.

5. A big increase in ROAS will lead to an even bigger increase in profits. At an average margin of 40%, a 22% increase in ROAS lead to profits from our campaigns growing by over 44%. One of our clients ROAS grew close to 90%, which lead to an approximate growth in profits of 180%!

That is nearly three times the profit for the same spend! Don’t forget this and never stop testing to increase returns!

YOUR TURN:

Already made the switch to collection?
Hit the comments below. I’d love to know your results.