A/B Testing For Beginners: 70 Resources to Get You Started

ab testing

There are two ways to make more money from your website. The first is to drive more traffic, and the second is to convert more visitors into customers.

Perhaps you already know about traffic acquisition as there are thousands of resources online for it… But what about conversion optimization? How do you know what to test or how to run A/B tests?

Sure, there are a ton of resources that break down the art of A/B testing and conversion rate optimization. But very few of them walk you through the whole process from the very beginning (how to come up with test ideas) to the very end (how to improve your conversions).

Download a printable version of 70 resources on A/B testing for beginners.

In that spirit, I thought I would teach you everything you need to know about A/B testing:

What is A/B testing?

Before we get started, you should know what A/B testing is and why you should use it. You should also be familiar with some of the terms in the space as A/B testing is also known as conversion rate optimization.

  1. Introduction to conversion rate optimization – here, not only will you learn what conversion rate optimization is but also why it is so valuable.
  2. A/B testing glossary – this blog post will explain all of the technical terms you need to be familiar with.
  3. What is A/B testing? – a few visual diagrams that explain how A/B testing works.
  4. The definitive guide to landing page optimization – landing pages are a crucial part of A/B testing. They allow you to create custom experiences for each of your visitor types.

How to come up with test ideas

Now that you know what A/B testing is, here are a few articles that will teach you how to come up with test ideas.

  1. How to generate better A/B testing ideas – this article breaks down 3 quick ways in which you can generate ideas.
  2. How to come up with A/B tests using quantitative data – this video will teach you how to analyze your data to come up with testing ideas.
  3. How to use qualitative data to come up with testing ideas – you can’t just rely on numbers. Gaining your visitors’ opinions is a great way to figure out what you should test.
  4. Qualitative questions that lead to greater A/B tests – if you aren’t sure what questions to ask your visitors, read this.

How to prioritize and pick tests

Once you know what you want to test, you can’t just go out there and start testing. You need a plan. These resources will help you create one.

  1. How to build a strong A/B testing plan that gets results – A/B testing can also lose you money. If you keep running tests that fail, you will decrease your revenue. Make sure you have a plan before you start in order to reduce your chances of failure.
  2. Three frameworks to help you prioritize your tests – not all tests are created equal. Some will have a better chance of providing lifts. For this reason, you should prioritize your testing.
  3. How to run A/B tests that give you big wins – when you are starting off, you should focus on big wins. After a while, you’ll notice that it’s harder to get big gains, but at the beginning, big wins should be easily achievable.
  4. Should you run an A/B test or a multivariate test? – if you have a lot of changes to test, multivariate tests may be the right choice for you.

How to run tests

Let’s get you started…

  1. How to run your first A/B test – I wrote a whole chapter on running your first test. If you follow it, you will be off to the races.
  2. Split testing 101 – this guide will break down how to get a jump-start on your testing.
  3. How to get started with 5 simple steps – A/B testing doesn’t have to be rocket science. With this 5-step guide, you can get started quickly.
  4. How to set up an A/B test in 10 minutes using Google Analytics – Google Analytics has a free A/B testing feature. This tutorial will teach you how to use it.

Tools and resources

Although A/B testing is still a new concept, it has been around for years. Within that time, many tools and blogs have been created that are worth checking out.

  1. Which test won – this website has thousands of examples of A/B tests from other companies. They even share which test won and by how much.
  2. User Testing – it’s a great way to get feedback on your website and to figure out the issues that may be stopping people from purchasing. While browsing your website, users record themselves and give you live feedback.
  3. Crazy Egg – this tool shows you where users click and how they interact with your website. For example, it shows you how far down the page people scroll and if your calls to action are placed too low or too high. You will also see if users are clicking on elements that aren’t linked to anything.
  4. Google Experiments – if you want to run A/B tests for free, Google Analytics has that feature built in. It isn’t the easiest to use, but it is free.
  5. Optimizely – one of the best A/B testing tools out there. If you aren’t a developer, you can still modify your site through their WSIWYG editor and run tests.
  6. Unbounce – landing pages are a great way to boost your conversions. Unbounce makes it easy to create and test them on the fly.
  7. Bounce Exchange – if you have a lead generation site, you can easily boost your sales by 10%-20% through this service.
  8. Hellobar – you can now direct your users to the pages that drive the most sales with this free tool.
  9. ConversionXL – a great blog on everything conversion-related. It covers everything from tips and resources to recommendations on how you should go about your testing.
  10. Qualaroo – one of my favorite tools for qualitative analysis. It lets you ask users simple questions so you can figure out what bottlenecks you have on your site that are decreasing your sales.
  11. KISSmetrics – when running tests, you don’t want to just optimize for a conversion rate percentage. You want to optimize for revenue. KISSmetrics helps you do that.
  12. The definitive guide to conversion optimization – this guide gives you the nuts and bolts of A/B testing. It is over 30,000 words long.
  13. Bryan Eisenberg – if you want to read a good blog or books on testing, you can find them here.

How to analyze the results

Now that you have started your first test, it’s time to start analyzing the results.

  1. For how long should you run a test? – before you analyze the results, you need to make sure the test has been running long enough, or else the results won’t stick.
  2. How to analyze testing results through Google Analytics – if you are a Google Analytics user, you can get a lot of insights from the data it provides.
  3. Using primary and secondary conversion metrics – great article on what to look for when digging through your testing data.
  4. A/B testing calculator – a simple tool that helps you analyze which test has won.

How to write copy that converts

One of the best ways to increase your conversion rate is to adjust and test your copy. From headlines to the text within your calls to action, it should all be tested. Here are some resources that will teach you how to create high converting copy.

  1. The definitive guide to copywriting – if you want to be an expert in copy creation, this is a guide worth reading.
  2. Online copy that converts – 5 major lessons learned from running 250 copy-related split tests.
  3. The guaranteed way to radically improve your copywriting – Copyblogger has some excellent articles and tutorials on writing high converting copy such as this blog post.
  4. Minor changes and major lifts – how small copy tweaks can drastically impact your sales. This 57-minute video breaks down how one company increased its conversion rate by 45%.
  5. A framework for copywriting – when you are on a tight deadline and need some guidance, watch this video. It will give you a framework for copywriting.
  6. How small copy changes lead to big increases – some more examples of how you should adjust your copy in order to boost online sales.

Mobile A/B testing

With mobile devices being small, the browsing experience your site visitors get on them needs to be different from the browsing experience they get using a computer. Here are some tips on improving your users’ mobile experience.

  1. A/B testing tips for mobile marketers – before you start testing your mobile site or application, you should read this.
  2. 10 tips for mobile app optimization – in this video, Steven Collins will teach you how to optimize the conversion rates within your mobile application or games.
  3. Android VS iPhone – people have different types of phones, which means each user has a different experience. This article will break down A/B testing on the 2 most popular types of phones.
  4. A/B testing for mobile apps – testing of mobile sites is different than testing of mobile applications. This article will help you clear up any confusion you may have about mobile apps testing.

Ecommerce A/B testing

Tests vary depending on the type of sites you have. Here are some tips if you have an ecommerce site.

  1. A/B testing your way to more ecommerce revenue – a great ebook that will break down what you should do for your ecommerce site. It even covers the ways to fix any leaks your checkout process may have.
  2. 10 ecommerce A/B tests worth trying – one of the biggest ecommerce platforms, BigCommerce, breaks down 10 tests you have to run on your site.
  3. A/B testing for product images – the way you show the products on your site can affect whether people buy your products. This article offers tips for choosing your product images.
  4. Ecommerce optimization tips straight from the experts – 20 experts share what you should do in order to boost your ecommerce sales.
  5. A/B testing for ecommerce call to action buttons – learn how to maximize your call to action buttons, ranging from “buy now” to “add to cart.”

SaaS A/B testing

If you have a SaaS or subscription-based business, here are some tips and test ideas for you to follow.

  1. Where can SaaS companies use A/B testing to see direct results – a simple infographic that is worth checking out if you have a SaaS business and want to improve your conversion rate.
  2. How to build and optimize your onboarding funnel – with SaaS products, it isn’t just about optimizing your site’s front end. Testing your onboarding funnel is a great way to boost the lifetime value of your customers.
  3. Why you should never A/B test your pricing strategy – optimizing your pricing plans isn’t as simple as running an A/B test. In this scenario, you have to optimize for revenue and not conversions.
  4. Why A/B testing landing pages is critical for any SaaS business – if you have a SaaS business, you should be creating landing pages. This article will explain why and show you what to do.
  5. Qualaroo for SaaS – one of the best ways to boost your sales is through qualitative feedback. This article teaches you what to do if you are a SaaS-based business.
  6. What spending $252,000 on conversion rate optimization taught me – I break down the lessons I learned by hiring conversion consulting firms to help me A/B test my SaaS company.

Lead generation A/B testing

Collecting leads is different from trying to get someone to use their credit card on your site. Here are a few resources that will teach you about lead generation A/B testing.

  1. 30 things to test for lead generation – if you are looking for ideas, this is a great place to start if you have a lead generation business.
  2. How masthead changes increased leads by 232% – who would have ever thought that something as simple as a masthead image can change the number of leads you collect.
  3. How to A/B test your lead generation form – tweaking the number of form fields you have and their placement can affect the number of leads you collect.
  4. A/B testing case study on how one company increased its leads by 262% – one of the best ways to come up with ideas is to look at other people’s case studies.
  5. Lead generation lessons learned from A/B testing – before you start A/B testing your lead forms, you should learn from other people’s failures and successes.
  6. 4 A/B lead generation tests – a PowerPoint presentation that covers 4 tests for your lead generation site.
  7. 10 A/B tests that will skyrocket your sales – from the calls to action you use on your lead forms to the amount of content you place above your forms, this blog post will give you a few ideas to run with.
  8. Three usability tests that tripled lead conversion rates – by running usability tests, you’ll be able to generate more ideas on how to boost leads.

Email A/B testing

No matter what type of business you have, emails are a great way to boost your sales. From subject lines to calls to action and email text, you can A/B test all of email elements.

  1. Email campaigns that convert – if you are new to A/B testing, read this post before you start testing emails.
  2. How to split test your email campaigns – from tips on how to create the optimal subject line to even testing your body copy, this great article for beginners covers it all.
  3. A/B testing tips for email and landing pages – improving your conversion rate on your emails isn’t enough. You also need to optimize your email landing pages.
  4. Lessons learned from email testing subject lines – Coschedule’s team break down what they learned from 4 months of subject line testing.
  5. How to write email subject lines that increase your open rate by 203% – I broke down the lessons I learned from A/B testing email templates on Quick Sprout.

A/B testing mistakes

Last but not least, know that you will make mistakes. But if you learn from my – as well as other people’s – mistakes, you’ll increase your odds of success.

  1. 4 A/B tests that will kill your business – running the wrong tests can really hurt your business. Make sure you avoid these 4.
  2. 12 A/B testing mistakes that I see businesses making all the time – just because you see one business doing something doesn’t mean you should copy it.
  3. 7 A/B testing blunders that even the experts make – avoid these 7 blunders, and you’ll be better off.

Conclusion

Now that you know all about A/B testing, you are ready to start. You know everything from what A/B testing is to how to run and optimize tests.

If you haven’t started yet, you should consider running your first test in the next few weeks. It will help you reduce your acquisition cost per customer and give you a leg up on your competition.

What other resources do you recommend reading?

P.S. If you need help running A/B tests and optimizing your site for conversions click here.

If you want to break through to real profits online, you need some serious firepower. For a limited time I’m sharing some select tips and tricks Amazon, Microsoft, NBC & Hewlett Packard paid thousands of dollars per hour for, FREE.
  • The step by step guide to monster traffic generation
  • The how-to guide for increasing conversions on your website
  • 7 Cashflow killers your analytics tools are hiding from you
     
 
100% privacy, I will never spam you!

Comments

  1. Great post Neil. Just Instapapered it consume it fully at a later point.

    In my work, I often do outreach, and I find that the initial email matters a lot. So in order to incrementally improve the first contact I use templates name A/B/C – subject etc

    Later I can see which one has a better response and I can keep improving my approach.

    Good stuff!

    • Karan, glad I could help.

      Sounds like you are running a good strategy. The initial email is like an introduction. You need to make an impression to get the ball rolling.

  2. Brilliant post and I’m glad you included links to those A/B testing mistakes. The importance of A/B testing shouldn’t be underestimated but it’s really useful to know what to avoid when you carry it out.

  3. That’s a super list, Neil. It will take a good amount to go through some of the resources you have listed above.

    No wonder this page is going to rank very well soon for keywords like “ab testing for beginners” & “ab testing resources” – just like your other kissmetrics post 🙂

  4. Wow, information overload. Maybe I can read through this in the next year or so.

  5. Best A/B testing tool I’ve ever used for lead gen is LeadPages. You can make multiple pages in minutes without creating additional URLs. They also now have an exit intent option for pop up forms, which can also be split tested a lot.

    This is a great article because it’s amazing how split testing can help conversions. I’ve set up many lead gen pages and forms. I think I know which one will perform the best, yet that often performs the worst. Go figure.

    As usual you’ve put together a great list of resources.

    • Jon, glad you liked the list. It’s interesting how many people overlook split testing. It’s a valuable resource that should not be overlooked. I think LeadPages is an industry leader that people should really take time utilizing.

  6. Bounce exchange is seriously overpriced. Are there any alternatives minus the overdeveloped sense of worth?

  7. Thanks Neil for the ‘from A to Z’ list for beginners on A/B testing.

    In addition to the topic, there are such things as A/A testing (helps to avoid misinterpretations in A/B testing) and multivariate testing (you have mentioned it in p. 12 of your list).

    • Michael, glad you found it helpful.

      I definitely think that people should over overlook multivariate testing. Would love to hear more of your thoughts on A/A Testing.

      • Neil, I haven’t had a chance to put A/A testing into practice yet dealing with conversion optimization (But I’ve dealt with something like this analyzing time series regressions).

        So I know that A/A testing is run before A/B testing to support statistically the result of A/B testing by making sure that the two groups used for A/B testing are truly randomized (otherwise the results can not be considered statistically truthful).

        In other words, it makes sense to run A/A tests if you want a scientifically proved results.

  8. What an amazing list of resources! If applied, there’s more than enough resources here to get started and become an expert at A/B testing. But, practice is the best way to learn, so get testing!

    Neil, thanks so much for including our A/B Testing For eCommerce eBook in this list, much appreciated.

  9. Thanks for the vast resource list Neil. I love your Hello Bar plugin, and configure it on our blog to direct traffic to pages that generate the most leads (a combination of traffic and conversions determines which pages we direct the Hello Bar traffic to).

    Many people, myself included, think that A/B testing is complicated to implement and requires extensive technical knowledge. As a result, it is easy to put it off and never take action.

    I think the most important concept is to pick something to test and just get started experimenting. It doesn’t have to be perfect, as perfection doesn’t exist anyway. Once I finally got started, I realized that with all the tools available, split testing really isn’t all that difficult 🙂

    It’s all about taking imperfect action!

    • David, I agree. I think many people get scared off by the thought of testing and think it requires some technical no how, when in reality it really does not.

      It’s all about putting yourself out there and testing to see what works best.

  10. Thanks for writing this article in so detail as I am a beginner, so its better to learn the things like A/B testing as well. It will take much time to apply all resources, but I will try to implement one by one.

    Cheers!

  11. Thanks for the nice and detailed list Neil. With people like you online it’s now very easy to find your way through things if you are a beginner. I used the Hellobar for a while but to collect email addresses for the email list-instead of the pop-up bars, it worked well, but generally i felt that the pop-up bars gave me more convenience and a wide range of designs. I shifted back to the pop-ups but i believe for leading people to a particular landing page, Hello bar works very well.

    • Ian, glad I could help. I wish when I first started I had these types of list. That is why I make them for everyone else. I think a lot people are in the same boat as you in regards to HB — they like a subtle element to get people to sign up.

  12. great to have all concentrated in this post, although I would not call too beginner, there’s plenty of things to see and do, thanks as always.

  13. Neil, youve done it again. A very comprehensive blog with deeper explanation and links to your other posts. Youre a content behomoth.

    I will be actioning your lead generation and ecommerce A/B split test suggestions.

  14. Hey Neil,

    This is a post that a lot of serous beginners will definitely appreciate. You always are resource and this takes the cake when it comes to value! Thanks for the share Neil!

  15. Hey Neil,

    Another amazing juice filled post.

    On my daily to-do-list for my business I now always have at least one thing to do which has come from Quick Sprout.

    Thanks for adding to that list! No, really I mean it – I like being kept busy.

    Thanks

    • Naomi, thanks that means a lot. I try to provide as much valuable information as possible so I can help people out regardless of where they are with their marketing efforts.

  16. Thanks! You didnt mention Visual Website Optimizer. I think it’s way better than Optimizely.

    What do you think?

  17. How wonderful to AT LAST find out what A/B testing really is! I thought I had it figured out before, but was wrong in several respects. Now with this new understanding, the rest of this post makes infinitely more sense.

    Thank you, Neil. Now I must sign off and get to work….

  18. Hey,Good post Neil it is really worth to read Thank you so much for this..

  19. Wow Neil, what a great round-up of resources! And thanks so much for including one of my articles in the list too. As a long-time reader (ok, ok … a fan) I’m honoured to have ‘made your grade’. 🙂

  20. I have had a great struggle with A/B testing due to lack of good guides how to conduct. Neil I think I will need to use your article to get this started.

  21. Great post Neil…thank you for sharing

  22. Hi Neil,

    Your velocity of producing rich content is amazing. I wonder how you do it. Not many businesses produce content that is actually useful. Its almost impossible to not pay attention to your blog posts.

  23. I didn’t have much idea about A/B testing but this post had everything clear about A/B testing and I am gonna try it one of blogs now.

  24. thanks for share the list neil, very usefull articel

  25. This is a very informative post thanks for sharing .

  26. Great post again Patel, Keep up a good work as always!

  27. David Peterson :

    I completely agree! I would like to mention https://www.coreminer.com as another tool. It seems good and useful.

  28. Good post, Thanks for sharing A/B testing resource.

  29. It’s the most complete and detailed guideline for A/B testing
    It’s very interesting how much time did you spend in order to gather and structure all these resources in one place)

    Neil, if it’s not complex for you, can you add our A/B testing solution – Changeagain.me – to the list ‘Tools and resources’

    The main differentiators from other tools are:

    – Full and automatic Google Analytics integration. You just sing up to the service via Gmail account with websites linked to Google Analytics and in a few seconds we integrate our service with all properties, views and goals. After that you are suggested to create variation in visual editor and choose the goal from GA or create new goal in our service.

    Also this integration allows you to make deep analysis of A/B test results (segmentation, impact on all goals in the funnel). So if you are using GA all the time, it would be more convenient for you to create A/B tests in Changeagain

    – Completely new pricing model for A/B testing tool – Pay per experiment, not impressions. It starts from 50 $/month for 4 experiments/month. And cost of the tool doesn’t depend on website traffic. So for the projects with high-volume traffic it’s cost-effective.

    Also it has very intuitive interface without any unnecessary functions.

    It would be amazing if you checked out our service and added it to the list.
    Thanks in advance

  30. sure, ab testing is a very good method for optimizing, BUT…
    you can build 2 or 3… BAD CONTENT and compare those..
    a/b testing much come only after your content is legit..
    first step : optimize
    https://formtitan.com/Conversion-Rate-Optimization
    second : create few optimized versions and a/b test
    https://formtitan.com/AB-Testing
    last : choose what best work for you
    hope you will allow to share this.
    b.

  31. Actually not a bad list of resources. I’m particularly happy you mentioned A/B testing mistakes and testing analysis resources (though, it’s much important to jump in and do analysis than read about it).

    One thing I wish there was more of on this list, though, is UX resources. It’s a big part of developing better hypotheses and therefore better variations (more wins). We’ve been compiling as much original research as we can here (https://conversionxl.com/institute/research-studies/), but there’s always room for more.

Speak Your Mind

*