Experiments with Corbado Observe

Experiments let you compare how authentication performs between the variants of an A/B test, for example a new login layout against the current one. Corbado Observe does not assign variants. Your application or testing tool decides which variant a visitor sees and passes that assignment to Observe. Observe attributes every journey to the variant that was active and compares the variants’ results.

1. Experiments in Observe

An experiment assignment is a pair of an experiment key and a variant, for example login-layout with variant-a. Observe creates the experiment and its variants automatically when the first journeys carrying them are processed. No setup is needed before you send data. Experiments and tags both add context to journeys, but serve different purposes:

	Experiments	Tags
Purpose	Compare the variants of a test you run	Segment and filter analytics by business context
Example	`login-layout`: `variant-a`	`country`: `de`
Analysis	Experiment runs and Experiment Analysis	Filters and segments in dashboards

Experiments are stored separately from tags and do not count toward the tag limits.

2. Pass assignments from the browser

If the assignments are available in the browser, Autocapture can pick them up automatically, for example from the responses of your A/B testing tool. Corbado configures this together with the capture rules for your login, so your application does not need to change. Share which experiments you run and where your page receives their assignments. If Autocapture cannot read an assignment on its own, hand it over from your page in one of two ways. You can combine both.

2.1 Assignments known at page load

Add experiments to window.corbadoObserveConfig in an inline script before the Autocapture script tag:

<script>
  window.corbadoObserveConfig = {
    // Experiment assignments known when the page loads
    experiments: { "login-layout": "variant-a" },
  };
</script>
<script async src="<LoaderURL>"></script>

Observe reads the configuration once when it starts. See Configure the script tag for the other configuration options.

2.2 Assignments decided later

Call setExperiments() when your application knows the assignment, and again when it changes:

window.CorbadoObserve?.setExperiments?.({
  "login-layout": "variant-a",
});

Call it as early as possible, before the visitor starts to log in, so the whole journey is attributed to the variant.

3. Pass assignments from your backend

If your backend assigns the variant, send it with backend enrichment. Add experiments to a flow_enriched event and identify the browser session with sessionID, which your page reads with window.CorbadoObserve.getSessionId() and passes to your backend:

{
  "sessionID": "<ObserveSessionID>",
  "events": [{
    "id": "7d1f3a92-5c4e-4b8a-9f21-3e6d0c8b5a17",
    "type": "predefined",
    "name": "flow_enriched",
    "timestamp": 1761487201000,
    "experiments": { "login-layout": "variant-a" }
  }]
}

Send the event when your backend assigns the variant, ideally before the visitor starts to log in. If the browser also passes an assignment for the same experiment during a journey, the browser’s value applies to that journey. Use letters, digits, _ and - for experiment keys, with at most 128 characters. Variants are strings with at most 255 characters.

4. How journeys get their variant

Observe attributes each journey to the variant that is active while it runs:

Assignments stay active: once passed, an assignment applies to all following journeys of the session until you pass a new variant for the same experiment. Events without an assignment do not change it.
Changes during a journey: if the variant changes while a journey runs, the latest variant within that journey applies. Each journey counts for exactly one variant per experiment.
Nested journeys: a journey started inside another one, for example a sign-up inside a login, gets the same variant as the surrounding journey.
Sessions and devices: assignments apply within one browser session. Journeys on other devices are not linked to them, so assign the variant per visitor and pass it on every device the visitor uses.

5. Set up and analyze an experiment

5.1 Check the incoming data

Open Experiments → Manage. Experiments and their variants appear here once the first journeys carrying them have been processed. Use Explore exposure to confirm that each variant receives traffic and that the split is roughly as planned.

5.2 Create a run

A run is a named, time-boxed evaluation of one experiment. Select New run and fill in:

Field	What to enter
Experiment	The experiment key you pass from your application
Name and Description	How your team recognizes the run, for example the test’s ticket and hypothesis
Groups	One Control group and one or more Treatment groups. Give each group a Label and select its Variants.
Share %	Optional: the planned share of traffic per group, for example `50` and `50` or `90` and `10`. Set it for all groups or none; the shares must add up to 100.
Flow series	The authentication flows whose results you want to compare, for example login
Start date and End date	The test window. Leave the end date open while the test runs and set it when you stop the test.

After you create a run, Observe reprocesses the selected flows. Depending on the amount of data, the comparison can take up to 60 minutes to become available.

5.3 Analyze the results

Open Experiments → Analysis and select the run. The analysis compares the journeys of each variant during the run’s window against the control:

Completion rate: how many journeys ended successfully
Error rate: how many journeys ran into errors, and which errors differ between variants
Completion time: how long successful journeys took
Flow allocation: whether each variant received its planned share of journeys

The analysis counts journeys, not visitors. Treat observed differences as evidence to investigate alongside your experiment design.

6. Test your setup

Pass an assignment from a test page that runs Autocapture with a test project.
Complete a login with each variant.
Open Experiments → Manage and check that the experiment and its variants appear.
Use Explore exposure to check that your test logins are counted for the expected variants.

7. Next steps

Continue with Tags to segment journeys by business context.
Continue with Backend enrichment to send assignments and other facts from your backend.
See Analytics for the other analysis views.

Overview

Get started

Custom events

Developer tools

Monitoring & alerting

Passkey Intelligence

Data access

Security & compliance

Event model

Journey events

Context & enrichment

Experiments with Corbado Observe

1. Experiments in Observe

2. Pass assignments from the browser

2.1 Assignments known at page load

2.2 Assignments decided later

3. Pass assignments from your backend

4. How journeys get their variant

5. Set up and analyze an experiment

5.1 Check the incoming data

5.2 Create a run

5.3 Analyze the results

6. Test your setup

7. Next steps

​1. Experiments in Observe

​2. Pass assignments from the browser

​2.1 Assignments known at page load

​2.2 Assignments decided later

​3. Pass assignments from your backend

​4. How journeys get their variant

​5. Set up and analyze an experiment

​5.1 Check the incoming data

​5.2 Create a run

​5.3 Analyze the results

​6. Test your setup

​7. Next steps

1. Experiments in Observe

2. Pass assignments from the browser

2.1 Assignments known at page load

2.2 Assignments decided later

3. Pass assignments from your backend

4. How journeys get their variant

5. Set up and analyze an experiment

5.1 Check the incoming data

5.2 Create a run

5.3 Analyze the results

6. Test your setup

7. Next steps