The Math of CRO – Whiteboard Friday

Posted by rickperreault

This week we are talking about math…. Don't worry it's not the tough kind that Isaac Newton used to do, but the math behind Conversion Rate Optimization (CRO). This week Rick Perreault from Unbounce joins us to show the value behind CRO and how you can show its value to your clients/bosses. After watching what Rick has to say, share a comment about your experience with CRO.

Video Transcription

Hi, I'm Rick Perreault. I am cofounder and CEO of Unbounce is a platform that allows marketers to create and A/B test landing pages without having to rely on IT. That is really a part of conversion rate optimization, which is today's subject.

When Rand asked me to come up here and do one of these Whiteboard Fridays and he asked me to do it on conversion rate optimization, you know, I run a software company. I am not a conversion rate optimization expert. However, I see hundreds of people do it every day, and I have experienced this through much of my career. So I am going to share with you what I've seen and how I've seen conversion rate optimization actually bring far more ROI to your online campaigns than not doing it.

So, let me begin. Let's imagine this as a period of a three-month campaign. In the old days, and to some degree still today, marketers are really concerned about what happens getting people to click an ad. Let's just say I'm using . . . these are all sample numbers. Just to keep the math simple. So, let's just say month one I've got $1,000 budget and I generate 1,000 clicks and I convert, my conversion rate 1%. I get 10 customers at a cost of acquiring that customer of about $100.

Now, so month two, I say, "Okay, that's pretty good." Now if I am going to get more sales, generate more customers, I just need to increase how much I am spending on advertising. So I increase that to $2,000 and I get 2,000 clicks. I'm still converting at 1%. The result of that, I get 20 customers at a cost of acquisition of still $100 per customer. That hasn't changed.

Month three, now I am going to increase that to $3,000. I get my 3,000 clicks. Again, convert at 1%, generates me 30 customers, again CPA stays at $100. Over a three-month period, a total spent of $6,000 generates me 60 sales. That's pretty good.

Now, as time has gone on, something has changes. Smart marketers realized they could actually get even more ROI from this online advertising by focusing on what happens after the ads are clicked and focusing on moving this number higher. This is what we call conversion rate optimization. That's A/B testing, using unique landing experiences, using analytics, and really understanding what happens after somebody clicks your ad.

So, in this example, I use the same thing. I spend my $1,000 to get my 1,000 clicks. But this time I am going to spend $200 on conversion rate optimization, and by using analytics and some A/B testing, quite quickly I am able to push my conversion rate up. So now, I push it up to 1.5%. What we see happen here, now I've generated 15 customers, but more importantly, my cost of acquisition has gone down to $80, a 20% improvement on the ROI.

So the next month, I continue and I spend my $1,000. I generate my 1,000 clicks. Again, I continue with my budget, my conversion rate optimization budget. Again, I do some more A/B testing, do some more analytics, create some more landing pages, improve it, test buttons, test messaging. I am able to push it up. Get 2% conversion rate. Now, look what happens here, again 20 customers at $60 to acquire them. So, again, even a better saving.

Then finally, okay, now I am really going for it. Month three, I am going to spend $3,000. I am going to get my 3,000 clicks, continue with my conversion rate optimization, my $200 here, maintain my 2% conversion rate, generate 60 customers, and the cost of acquiring them somewhere around $53, $53 something.

In this case, I spent $5,600 to generate 95 sales. Here I spent $6,000 to generate 60. The reason I was able to generate more sales on relatively the same ad spend is because I stopped worrying about just what was going on here and started focusing on what was going on here. This is the math of conversion rate optimization and this is why it is important.

So next time you are talking to your boss or a client and they're trying to understand the value of A/B testing or using a landing page or just spending any time on thinking of what happens after their ads are clicked, the landing experiences, walk them through this exercise.

I hope that was helpful. Thank you very much.

Video transcription by

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!

What Community Builders Can Learn From Research

Posted by thogenhaven

Two weeks ago, Tom Critchlow suggested that we work to close the gap between inbound marketing and content marketing communities. It's time to build bridges again, this time between inbound marketing and research. In this post, you'll find research on participation patterns, how to spot high-value users, seeding content in a new community, how to bring new life to old content, and a little bit of gamification.

Some research is already being shared with the inbound community. Bill Slawski from SEO By The Sea does a great job reading and condensing patents from the search industry. But there is so much more research waiting to be tapped.

I am currently in a PhD program and therefore attend academic conferences. They are different to MozCon, SearchLove, SMX, Blueglass and the other conferences we all usually go to. And different means different perspectives. Last week at CSCW, 160 researchers from private companies and universities presented a paper. Topics include social media analysis, collaboration, gamification, incentives, recommender algorithms and online communities. For better or worse, I did not attend 160 presentations. So this will be a very limited summary, focusing on online communities.

Why Should You Care?

Universities and private companies like IBM, Microsoft and Google do some legit research. Being familiar with this research is a competitive advantage and will help generate new ideas.

In this post I focus primarily on community building. At SearchLove last year, Rand had a slide stating a 34% growth in 4 months, primarily from Q+A, YouMoz, the blog and user profiles. Add to this that community members are some of the best link builders you'll ever find. Getting community right is a huge win.


Who Participates In Online Communities?

Previous research offers two perspectives on participation patterns in online communities:

  1. Some people contribute, and others do not. It is an inherent, personal trait like hair color.
  2. Lurking is a development stage toward being an active member. All people potentially contribute, after the learning/socialization phase: users lurk for a while before participating.

Michael Muller from IBM presented fascinating research on a study on 8,711 online communities covering diverse topics with 224,232 unique users. The insight of the research shows a completely different pattern than the conventional wisdom above: 84 % of those users who participate in one or more community, lurk in others. However, the majority of members' lifetime contributions are in the beginning on their membership. Thus, many users start off contributing like mad, then stop. This means retention is key.

(Graph is printed in Muller, 2012. See references in the bottom of this post).

Design implications: Do whatever you can to grasp new members. There are many ways to do this: Make sure they get encouraging feedback to their initial comments/contributions. Assign them a mentor. Send them nice emails. Reach out to them on social media.

Spotting Talent

Despite the overall participation trend identified by Michael Muller, some people are more likely to contribute more to new communities than others. In fact, only few people end up participating in the first place. Google+ VP Bradley Horowitz once wrote about 90-9-1 principle, describing how 1% of community members are creators, 9% are synthesizers, and the remaining 90% are users/lurkers who do not directly add anything to the community.

Rosta Farzan and colleague from Carnegie Mellon University and University of Minnesota developed an algorithm to identify potential high-contributing members. The algorithm uses the following metrics to spot a potential high value member.

  • Quality
  • Motivation (quantity, frequency, and commitment)
  • Ability (knowledge, trustworthiness, and politeness and clarits)

Those identified as potential high-contributing members participated 10 times more actively than those not classified.

Design implications: sometimes the gold is right in front of us, but without our knowing. Identifying high potential members early on can help us reach out and retain these creators.

Starting A New Community

In inbound marketing, one often hears the advice: go build a community. Yes, we'd all love to have flourishing communities, right? But how to get critical mass? One solution often used is seeding a site with (third party) content. This is supposed to show that the community is lively and thereby encourage users to contribute. Jacob Solomon and Rick Wash from Michigan State University tried this form of bootstrapping when starting a new wiki.

The results show that users contribute more when they are given a blank page, than they do when they see a seeded page. This makes sense, as there is more work to do on a blank page. However, contributions made on a blank page tend to be unstructured. If the users see a page with some content (e.g. headers, text chunks, objective content, opinionated content etc.), they tend to contribute content similar to the seeded content.

Design implication: If you want users to create a special kind of focused content (e.g. replies of a certain length or with a special focus), seeding can be good. The bad news: seeding content is not a shortcut to start a community as it might actually reduce contributions. Two weeks ago, Rand and Dharmesh launched Inbound. When the site was launched, it was already seeded with many good articles. According to this paper, this seeding reduced contributions, but made them more focused on the kind of articles Rand and Dharmesh want. Sounds plausible.

New Life To Old Content

This one might require a bit engineering power. But it is really neat. Aditya Pal and colleagues from University of Minnesota created an algorithm to detect expired content on a Q&A site. The algorithm uses metrics such as

  • TF/IDF
  • Reference to a specific time (e.g. date, month)
  • Fixed vs relative time reference (ago, after, before, today, tomorrow)
  • Reference a date in past
  • Tense of the question

Design implications: Such algorithms are not only useful on Q&A sites. On enterprise websites, it can be used to flag content that ought to be updated, removed, rel=canonicalized or 301 redirected to new content. This creates better and fresher content on websites, as well as help avoiding old and irrelevant pages rank in Google. It can also help scale some of Cyrus Shepard's advices on fresh content, and help you rank for QDF keywords.

(This illustration is made by Dawn Shepard for Cyrus' post mentioned above)

Gamification Over?

Gamification has been a hot topic in the last couple of years. For many websites, the question is no longer if gamification systems should be implemented, but if it should be kept. Jennifer Thom and collaborators from IBM studied the removal of gamification points from IBM's internal social network. The researchers found that removing the points system made users contribute significantly less than before.

Design implications: You might (also) be tired of hearing about gamification. But it kinda works… So you might want to take a look at these gamification slides from Richard Baxter:

Curious for more?

The ACM Library is very good. In fact, so good that Matt Cutts blogs about it. To access the articles, you might have to go to a library or a university. But many researchers are happy to share their research, and link to it directly to their own work from their personal websites (The authors have the rights to share their own articles for free). So a little Googling can usually provide the article.


Michael Muller (2012): Lurking as Personal Trait or Situational Disposition? Lurking and Contributing in Enterprise Social Media. Proceeding to CSCW 2012

Aditya Pal, James Margatan, Joseph Konstan (2012): Question Temporality: Identification and Uses. Proceeding to CSCW 2012

Jacob Solomon, Rick Wash (2012); Bootstrapping wikis: Developing critical mass in a fledgling community by seeding content. Proceeding to CSCW 2012

Rosta Farzan, Robert Kraut, Aditya Pal, Joseph Konstan (2012): Socializing volunteers in an online community: A field experiment. Proceeding to CSCW 2012

Jennifer Thom, David Millen, Joan DiMicco (2012): Removing Gamification from an Enterprise SNS. Proceeding to CSCW 2012

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!