This document is a work in progress. The Digital Experience team is the DRI for marketing's AB testing engineering efforts using the feature flag based tool LaunchDarkly.
An A/B test is when we release two versions of a page and compare how well they perform (comparing version A to version B). When running an experiment, we are testing a hypothesis using a control variant and a test variant, similar to how one typically employs the scientific method.
Below are some resources to learn more about feature flags. At a high level, a feature flag is an if-else wrapper around code that can be enabled, disabled, or served at a certain percentage to a certain group. This is controlled via a dashboard toggle, changing the production interface on-the-fly without having to wait for a release to change something.
Below are some links with information on the history of the decision:
We currently use LaunchDarkly to control whether or not a test is showing, at what percentage, and gather metrics about a test's performance.
This can be overridden by optional URL parameters as exhibited in the codepaths section below.