Visitors are shown 1 of 2 variations of homepage. One led to more registration than the other and is thus the winner.

How to quickly set up A/B testing for React websites

Start collecting important data in as little as 30 minutes

A/B testing, also called split testing, is an experiment where two (or more) variants of a webpage are shown to users at random, and is used as a means of determining which variant leads to a better performance. Once a variant wins, it is then shown to all users.

A/B testing is extremely important in the world of user acquisition. If your site is in React, setting up a simple test could take less than half an hour. All you need to do is:

1. Install an NPM package
2. Add a few lines of code to set up the test
3. Collect data

I. Setup

I used react-ab-test by pushtell when I implemented A/B testing for one of Pillow’s landing pages.

The code looks like this.

import { Experiment, Variant, emitter } from 'react-ab-test'
emitter.defineVariants('myExperiment', ['control', 'variant'], [50, 50])
class ImportantLandingPage extends Component {
  render() {
    return (
      <div>
        <Experiment name='myExperiment'>
          <Variant name='control'>
             // your control elements
          </Variant>
          <Variant name='variant'>
             // your variant elements
          </Variant>
        </Experiment>
      </div>
     )
  }
}

emitter.defineVariants() takes in three arguments. The first is the experiment name, the second an array of variant names, and the third an array of weights.

Now, when users visit your site, they will be randomly assigned either the “control” or the “variant”.

The variant assigned to a user is stored in your browser’slocalStorage . Thus, if the user visits the site again (assuming in non-incognito), he/she will see the same variant again.

II. Data Collection

The experiment wouldn’t be meaningful if we weren’t collecting data on it would it?

Let’s say you’re tracking the clicking of a CTA — a <button>element — on the landing page.

Pillow already uses Mixpanel for event tracking. However, you could easily send events to any form of storage unit, including your own database.

To collect data, add the following to your ImportantLandingPage component:

componentWillMount() {
  emitter.addPlayListener((experimentName, variantName) => {
    // what to do when experiment is displayed. E.g. log to Mixpanel or save to database (with redux-saga for instance)
  })
  
  emitter.addWinListener((experimentName, variantName) => {
    // what to do when a 'win' is emitted (by emitter.emitWin())
  })
}
ctaClicked() {
  emitter.emitWin('myExperiment')
}
render() {
   ...
     <button onClick={this.ctaClicked.bind(this)}></button>
   ...
}

Since Pillow uses Mixpanel, we were able to easily build a conversion funnel to track the performance of both variants.

Screenshot of our Test data on Mixpanel

III. Profit

The importance of A/B testing could not be overstated.

Jason Fried, founder and CEO of Basecamp, wrote an article on how a failure to A/B test Basecamp’s marketing page redesign cost the company millions.

If you’re trying to maximize signup or purchase, how do you know which variant would click with users more? You could ask your customers by sending out surveys, but as Steve Jobs once famously said, “It’s not the consumer’s job to know what they want.” The data from A/B testing could tell you what your consumers indeed want.

With all of the advantages provided by A/B testing, there are nevertheless caveats to watch out for.

Ideally, you should have a good hypothesis for the test. If you don’t, you’re taking a shot in the dark. If it wins, you’re lucky. If it loses, you won’t learn anything from it — a total loss of resource. There are of course exceptions. For example, in the case of a complete redesign, it would be difficult to test the effect of any particular element.

A/B testing works best in a high traffic site. As with any scientific experiments, sample size is critical to amassing statistically significant results. Optimizely developed a sample size calculator that calculates how many users you’d need to reach statistical significance.

Given time, resource, sample size, and hypothesis, there are countless examples of how A/B testing improved conversion for companies. Now you know how to set up an experiment in your React app in no time.

Further reading:

1. How we lost (and found) millions by not A/B testing
2. When not to A/B test

Pillow is hiring! See a list of our current openings, here.

I’m a software engineer in San Francisco. I graduated from MIT in 2013 with a BS in biology. I started my career in tech at the start of 2015 and have been loving it since. If you like my work, follow me on Medium to receive more stories from me.

If you learned something new from this article, mind clicking on the ❤ icon for me? It would mean so much to me!