How To Run Marketing Experiments: 10 Things You Should Know

7 min readJan 6, 2022

Marketing experiments are not something I had come across before and viewed marketing as a “one-time” thing.

I remember chatting with an experienced marketer, he said to me “test the shit out of everything. You will make assumptions that are wrong, and you have no idea what will actually work.”

I always viewed marketing as an art form. Being from a technical background, I didn’t think I could figure it out.

Once I started looking at marketing as like a big “science experiment”, it changed the game for me. Plus it made it more fun.

I also try thinking “what the one big domino I can knock over”. If I knock this one over either all the others will fall, and the ones that don’t, are irrelevant.

Why You Should Run Marketing Experiments

The whole point of marketing is to grow your business. To do this you need to experiment to find a better, faster, or more optimized way to promote your business’s products or services.

Unless you are running marketing experiments you are always going to fail. Unfortunately, I have first-hand experience with this.

A few years ago now, I was one of three founders that launched a mobile app-based business. We did the usual start-up thing of pivoting to find our niche. What we found was that our Facebook Page was our key marketing tool and could reach thousands of people with a single post.

We were on the verge of something great, or so we thought. Zuckerberg snapped his fingers and stopped Facebook Business Pages from being able to reach so many people. Overnight our entire business had dried up.

We hadn’t been running any marketing experiments, in fact, we hadn’t been testing anything. Needless to say, our start-up didn’t last long after that.

As I’ve mentioned you will have assumptions and other people will have their own opinions. But no one knows for sure.

Running marketing experiments are the best way to create new knowledge. And to learn more about your audience and your customers.

It makes you question your ideas, beliefs, and the so-called best practices. This is hard when the data doesn’t show you that your idea wasn’t a good one.

Why People Aren’t Running Marketing Experiments

For chatting to a lot of people, it’s not from the lack of want. From what I’ve learned there are 3 reasons why people aren’t running marketing experiments

The are working towards the wrong goals, i.e. getting more likes on Facebook
Lack of routine, they haven’t got into the habit of testing
Afraid of failure, I’ve been there, you don’t want to look stupid in front of your peers.

The struggle is real when it comes to setting goals that directly correlate with business performance. But once you set your mind on a goal, work towards them in an organized manner and start experimenting; things will happen naturally!

The fear of failure can be one of the biggest obstacles for marketers. One way to overcome this is by creating an environment where experimenting and coming up against new challenges would not lead directly to losing your job security

The other big issue could simply just come down from being afraid that you might end up getting fired or worse yet — labeled as “inexperienced” by management if things don’t go exactly how they were hoping so there isn’t much incentive then either

What can be done about this? Here are 3 suggestions

To get buy-in for experimentation, you need to appeal at the top of your organization. Get approval from investors or board members before implementing new ideas so they can be successful and worthwhile!
If you’re not a fan of the slow pace, start with small wins. You might be surprised at how much progress can come from them!
It’s not easy to get people on board when they don’t know what it takes but thankfully we’ve got the solution: Replace Your Team!

OK. Now we gave got the whys out of the way let’s start digging into the 10 things you should know about running marketing experiments.

Marketing Experiments: 3 Things Good Ones Have In Common With Each Other

All good marketing experiments should have the following things in common

1) Systematic In Their Approach and Measured With Data

Running marketing experiments is just like running a scientific experiment. You have to stick to a process and review the data to see how successful the experiment was.

All good marketing experiments are systematics and measured with data.

You need to follow a plan, or at least put a process in place. The plan is then validated using the data.

An example of a bad marketing experiment is when the goals have been set only after reviewing the data.

A solid process and a healthy relationship with data are what make or break a marketing experiment

2) Are They Big Enough

Now I’m not saying you should drop A/B testing altogether. I still A/B test things like headlines and button text. But this is more of an optimization process, not an experiment.

The other, perhaps a more controversial requirement for an experiment to be considered “good” is that it’s large enough. In other words; yes — forget about those button color A/B tests from decades ago!

You need to be bold, be willing to take a risk.

Run an experience that is a complete landing page redesigns instead of button colors.

Experiment with product pricing instead of call-to-action microcopy.

Play with different automated advertising strategies instead of tweaking single ads.

Experiment with budget allocation over different advertising platforms instead of micromanaging individual platforms

The problem with small tests is that even when they’re successful, they’ll yield small results.

As I’ve said, if you want to optimize existing assets that are working. Test small things to move the needle slightly.

But, if you want to go from zero to one, or make a leap from breaking even to experiencing growth year on year. You need to try something completely different.

Throw the existing landing page out the window and try something new instead. If the new version works better, then use that as your benchmark going forward.

The takeaway. Think big with you’re experimenting.

3) They Are Run As Split Tests

Any marketing expedient worth its weight in gold, needs to be run as a split test. So have something to compare too, that you’re testing one variable against another.

You split the audience (website visitors, advertising audience) into two or more groups.

Then you can offer different things to different groups. Make sure you keep a percentage of the audience separate, this will be the control group.

With this strategy, you can complete the effectiveness of the different assets.

You don’t need lots of website traffic or a large budget to run marketing experiments.

Yes, the results aren’t as scientific as with high traffic. Don’t let that stop you from running marketing experiments.

7 Step Process To Running Marketing Experiments

Now we have given you the 3 things any good marketing experiment should have. Here is a guide to run an experiment in 7 easy steps.

Step 1: Set Your Goal

The first thing you need to work out is what KPI you’re trying to change.

For example, we would like to increase our free trial take-up rate by 10% within the sign 3 months.

The key to this step is making the goal not open to more than one interpretation and giving it a distinct time frame.

Step 2: Review Historical Data

You have worked out what needle you are trying to move. The next step is to work out where you currently are, by looking at your existing data.

This gives you a chance to set a baseline and see what you have done in the past to get to this point.

Step 3: Brainstorm Ideas To Test

Once you have your baseline. You can start to come up with ideas or things that may help you to improve the KPI you want to look at.

I’d recommend coming up with at least 10 ideas to test, 20 would be even better. At this point, you just want a list of ideas, nothing else.

Step 4: Prioritize Your Ideas

Next, you need to prioritize the ideas that you have come up with. Everyone is different but you need to think about what resources are at your disposal. How easy it will be to implement. Probability of it working, and if it is scalable or not.

You should now have a clear idea of which marketing experiments to start testing. Pick the top 3 that you think will have the biggest impact. You don’t want to overwhelm yourself or your team.

Step 5: Run The Experiments

It’s time to run the marketing experiments. If you have multiple experiments you want to test. You need to make sure you can run multiple experiments at once.

If you can easily track, then by all means get the tests up and running. Or if you don’t care about which one works as long as you moved the needle, then by all means run multiple together.

Personally, I like running one experiment at a time. I’ve run multiple together in the past and had no clue as to which idea caused the upturn in success.

Step 6: Measure Success

It doesn’t matter if your marketing experiment has ended or if it’s still running. Start looking at the data, did the experiment get the results you were expecting?

Step 7: Rinse and Repeat

Depending on the scope and results of your experiment, you might want to start from scratch or simply go back through all steps again.

Conclusion

You might be wondering how to get started with running marketing experiments.

Lucky for you, we’ve compiled a list of 10 things that are critical when it comes to experimenting in the digital age.

This is not an exhaustive list by any means, but these tips should help you get on your way! Have you tried any of these? Did they work? Let us know in the comments — we want to hear from our readers and learn about their successes too!

Psst! This is where Elementary Analytics comes in handy. You can automate data and email alerts to cut down the time you would otherwise have to spend on manually collecting data from multiple platforms.

We make setting up analytics fun and easy so there’s no excuse not to experiment now. There’s nothing like watching