Walkthroughs

The recordings were made on a local stack with demo data. Turn captions on in the player; the transcript is below each video.

Get started
The Quick Start's command, then a tour of the dashboard: a completed experiment's results and a flag's rollout schedule.
Narrated recording
Length: 1 minute 24 seconds
Transcript: Get started
On screen: Controlvar. Get started. A/B testing and feature flags you run yourself.
On screen: The Quick Start's command: shown here, not run. docker compose up -d --wait. It starts Postgres 16, Redis 7, the API and the dashboard, with the demo seed: 3 experiments, 2 flags. This recording starts the same stack off camera, in its own compose project on other ports; results are computed before they are opened, and waits for a page are cut.
On screen: One command starts the whole stack on your machine.
One command starts the whole stack on your machine: docker compose up.
On screen: Postgres, Redis, the API and the dashboard, with demo data.
Postgres, Redis, the API and the dashboard all come up. With demo data: 3 experiments and 2 flags.
On screen: Open the dashboard in your browser.
Open the dashboard in your browser, and choose Sign in.
On screen: Sign in with the demo admin account from the Quick Start.
Sign in as the demo admin. The email and password are in the Quick Start.
On screen: Your experiments, each with its status at a glance.
Here are your experiments, each with its status at a glance. Open the completed one.
On screen: Each one has its variants, its metric and a hypothesis.
It shows the variants, the metric it's judged on, and the hypothesis.
On screen: Its results, computed off camera first: lift, p-value, winner.
Its results show the lift, the p-value, and a recommended winner. Here, the new copy wins: a 15% lift in sign-ups.
On screen: Feature flags turn a feature on for some users, or all.
Feature flags turn a feature on for some users, or for everyone.
On screen: Flags roll out in stages, with a safety check alongside.
This flag rolls out in stages, with a safety check alongside.
On screen: Each stage raises the share of users who get the flag.
Each stage raises the share of users: 10%, then 25%, now 50%.
On screen: The safety check watches the error rate as it rolls out.
And the safety check watches the error rate as it rolls out. Right now, it's healthy.
On screen: Controlvar. Open source, Apache-2.0, runs on your own infrastructure. controlvar.com. github.com/Controlvar/controlvar. Recorded on Controlvar v0.36.0, a local stack with demo data.
Your first experiment
Create an experiment in the guided setup, work out how many users it needs, start it, and read its result once simulated shoppers have arrived.
Narrated recording
Length: 1 minute 27 seconds
Transcript: Your first experiment
On screen: Controlvar. Your first experiment. A/B testing and feature flags you run yourself.
On screen: Signed in off camera. Create an experiment in five guided steps.
Choose New Experiment: a guided setup in short steps.
On screen: Name it, and say what you expect to happen.
Name it, and the key your code uses fills itself in. Then say what you expect to happen.
On screen: Pick the event that counts as a win.
Pick the event that counts as a win: here, a purchase.
On screen: Two variants, split evenly between users.
Control and Sticky bar, split evenly between users.
On screen: Size it first: how many users do you need?
Size it first. Enter today's conversion rate, the lift worth catching, and your traffic.
On screen: 31,234 users per variant, about 13 days here.
That's 31,234 users per variant: about 13 days.
On screen: Review it, then create it.
Review it all on one page, then create it.
On screen: Start it, and it is ready for shoppers.
Choose Start, and it's ready for shoppers.
On screen: From the JavaScript SDK quick start.
```
const user = { userId: 'user-123', attributes: { country: 'US', plan: 'pro' } };

// Sticky experiment assignment (POST /api/v1/tracking/assign); attributes are sent as `context`
const assignment = await client.getAssignment('checkout_button_color', user);
assignment.variantName;     // 'control' | 'treatment'

// Track a conversion for one experiment (POST /api/v1/tracking/track)
await client.track('user-123', 'purchase', { value: 49.99, experimentKey: 'checkout_button_color' });
```
On screen: As the docs give it. Your app passes its own experiment's key; the shoppers in this video are simulated and sent the same way: 63,000 of them from a fixed seed, set to buy at 5% in Control and 5.6% with the Sticky bar.
On screen: Your app asks the SDK for a variant, then tracks purchases.
In your app, the JavaScript SDK asks for each shopper's variant. Then it tracks their purchases.
On screen: Open its results as simulated shoppers arrive.
Open its results as shoppers arrive, then the Live tab.
On screen: Each refresh shows more shoppers, split about evenly.
Each Refresh shows more shoppers, split about evenly.
On screen: Off camera. 63,000 shoppers later. About 2 weeks of traffic at 10,000 visitors a day with half of them in the test, sent by the tool in 13 minutes (measured), so the results page reads Duration 0d.
On screen: About two weeks of shoppers, simulated in minutes.
63,000 shoppers later: about 2 weeks of traffic, simulated in minutes.
On screen: Computed off camera first: Sticky bar wins, 12.1% lift, significant at 95%.
Sticky bar wins: a 12.1% lift in purchases, significant at 95% confidence.
On screen: Controlvar. Open source, Apache-2.0, runs on your own infrastructure. controlvar.com. github.com/Controlvar/controlvar. Recorded on Controlvar v0.36.0, a local stack with demo data.
Roll out a feature flag
Create a flag, target a segment, set a rollout percentage for everyone else, switch it on and off, and find each change in the audit log.
Narrated recording
Length: 1 minute 26 seconds
Transcript: Roll out a feature flag
On screen: Controlvar. Roll out a feature flag. A/B testing and feature flags you run yourself.
On screen: Signed in off camera. Wrap a new feature in a flag.
Wrap a new feature in a flag to decide who gets it. Choose New Flag.
On screen: Name it. Your code checks this key.
Name it, and the key your code uses fills itself in. Then describe what it controls.
On screen: Add a rule for the Beta testers segment, made off camera.
Under Targeting Rules, add a rule for users in a segment. Here, the Beta testers segment, made off camera.
On screen: They get it at any rollout percentage.
Beta testers get it at any rollout percentage.
On screen: Everyone else starts at 10%.
The slider sets the share for everyone else: here, 10%. Choose Create Feature Flag.
On screen: It starts off: not serving anyone yet.
It's created switched off, so no one gets it yet.
On screen: Switch it on when your code ships.
Switch it on when your code ships, and the page reads Serving.
On screen: Going well? Raise the rollout and save.
If it's going well, raise the rollout to 50%, and save.
On screen: Or schedule the stages through the API.
Or raise it in stages with a schedule, created through the API.
On screen: Problem? Switch it off in one click.
Switch it off in one click if there's a problem. It's off again.
On screen: Every change is logged: who, what, when.
Every change to the flag is in the audit log. Who made it, what it was, and when.
On screen: Controlvar. Open source, Apache-2.0, runs on your own infrastructure. controlvar.com. github.com/Controlvar/controlvar. Recorded on Controlvar v0.36.0, a local stack with demo data.
Roll out to devices, roll back by itself
A device rule, a kill switch, then the safety monitor rolls a staged rollout back by itself after simulated crashes. The demo uses the web SDK; iOS, Android and Flutter send only the user id to flags.
Narrated recording
Length: 1 minute 23 seconds
Transcript: Roll out to devices, roll back by itself
On screen: Controlvar. Roll out to devices, roll back by itself. A/B testing and feature flags you run yourself.
On screen: Seeded off camera: this iPhone on iOS 17.4 gets AI search.
In the StreamPulse demo, this iPhone on iOS 17.4 gets AI search.
On screen: A Pixel 7 on Android 14 gets keyword search.
A Pixel 7 on Android 14 gets keyword search.
On screen: Signed in off camera: the rule lives in the dashboard.
In the dashboard, the rule is iOS 17 or later, US, premium.
On screen: Kill switch: Recommendations v2 off, no app release.
A kill switch: turn Recommendations v2 off, with no app release.
On screen: In the app, the feed is back to Classic.
In the app, the home feed is back to Classic.
On screen: Off camera, the demo's script moved it to 25%.
In the dashboard, Player v2 is now at its 25% stage.
On screen: In the app, this Galaxy S10 on Android 12 gets Player v2.
In the app, this Galaxy S10 on Android 12 gets Player v2.
On screen: Off camera. Player v2 crashes on Android 12. Simulated by the tool: Android 12 devices ask for Player v2 200 times and report 40 crashes. The demo apps run from their own build in this recording.
On screen: Simulated: Player v2 crashes on Android 12.
Simulated: Android 12 devices crash in Player v2.
On screen: In the dashboard, the error rate passes its limit.
In the dashboard, the error rate passes its limit: Unhealthy.
On screen: Off camera. 1 minute later. Waited off camera; the time above is measured. The safety monitor checks every minute in this demo (default: every 5 minutes).
On screen: The safety monitor checks every minute here.
The safety monitor checks every minute here.
On screen: By itself, the monitor paused the rollout schedule.
By itself, the monitor paused the schedule.
On screen: And it rolled Player v2 back to 5%.
And rolled Player v2 back to 5%.
On screen: In the app, pick the Galaxy S10 again and refresh.
Back in the app: the Galaxy S10, then Refresh now.
On screen: Now it gets the classic player.
Now this Android 12 phone gets the classic player.
On screen: The audit log: the safety monitor did it, and why.
The dashboard's audit log shows the safety monitor did it, and why.
On screen: Controlvar. Open source, Apache-2.0, runs on your own infrastructure. controlvar.com. github.com/Controlvar/controlvar. Recorded on Controlvar v0.36.0, a local stack with demo data.
Serve a value with a feature flag
Create a flag that serves a string instead of on or off: its type, an on value and an off value, and a rule for pro plan users. Switch it on, then see the Go SDK lines that read the value.
Narrated recording
Length: 1 minute 13 seconds
Transcript: Serve a value with a feature flag
On screen: Controlvar. Serve a value with a feature flag. A/B testing and feature flags you run yourself.
On screen: Signed in off camera. A flag can serve a value.
A flag can serve a value instead of on or off. Choose New Flag.
On screen: Name it. The key fills in, and you can change it.
Name it, and the key your code uses fills itself in. You can change it: here, to the key the docs use.
On screen: Choose its type: String, Number or JSON.
Choose its type: a string, a number, or JSON. Here, a string: the checkout button label.
On screen: An on value, and an off value for everyone else.
Users it's on for get the on value: Complete purchase. Everyone else gets the off value: Buy now.
On screen: A rule: pro plan users get the on value.
Add a rule: users on the pro plan get the on value.
On screen: Rollout at 0%: only that rule's users get it.
With the rollout at 0%, no one else gets the on value. Choose Create Feature Flag.
On screen: It starts off: everyone gets the off value.
It's created switched off, so everyone gets the off value.
On screen: Switch it on when your code is ready.
Switch it on: rules and rollout decide who gets the on value.
On screen: From the Go SDK docs.
```
label, _ := client.EvaluateFlag(ctx, "checkout-button-label", user) // a string flag
fmt.Println(label.StringValue("Buy now"))
```
On screen: As the docs give it, for the flag made above; go get installs the Go SDK's latest release.
On screen: Your app reads the value, with a default.
In your app, the Go SDK reads the value for each user. Without an answer, it returns the default you give it.
On screen: Controlvar. Open source, Apache-2.0, runs on your own infrastructure. controlvar.com. github.com/Controlvar/controlvar. Recorded on Controlvar v0.44.1, a local stack with demo data.
Plan a test, bring your team
Work out a test's sample size in the power calculator, which needs no account, then sign in and invite teammates, each with a role.
Narrated recording
Length: 1 minute 22 seconds
Transcript: Plan a test, bring your team
On screen: Controlvar. Plan a test, bring your team. A/B testing and feature flags you run yourself.
On screen: Plan first: the calculator needs no account.
Plan a test first: the power calculator needs no account.
On screen: Start from today's rate and the lift worth catching.
Today 5% of visitors buy; you want to catch a 10% lift.
On screen: Add your traffic, and the share of it in the test.
Add 10,000 users a day, with 50% of them in the test.
On screen: 31,234 users per variant: about 12 days here.
That's 31,234 users per variant: about 12 days.
On screen: A third variant needs more users in each one.
A third variant needs 37,824 users in each.
On screen: Now bring in your team.
Now bring in your team.
On screen: Sign in as the demo admin from the Quick Start.
Sign in as the demo admin. The email and password are in the Quick Start.
On screen: Admin, then Users, then Invite User.
Open Admin, then Users, and choose Invite User.
On screen: Invite Priya as a Developer.
Invite Priya as a Developer: she can build experiments and flags.
On screen: Her password shows once; hidden for this video.
Her one-time password shows once; this video hides it.
On screen: Four roles: Admin, Developer, Analyst and Viewer.
Pick a role: Admin, Developer, Analyst or Viewer. Marcus joins as an Analyst.
On screen: Analysts see every experiment and flag, and edit neither.
An analyst sees every experiment and flag, and changes neither.
On screen: Controlvar. Open source, Apache-2.0, runs on your own infrastructure. controlvar.com. github.com/Controlvar/controlvar. Recorded on Controlvar v0.36.0, a local stack with demo data.
An A/B test, from setup to winner
Create a checkout test in the guided setup, see each version's checkout in the ShopLab demo storefront, and read the winner once simulated shoppers have arrived.
Narrated recording
Length: 1 minute 19 seconds
Transcript: An A/B test, from setup to winner
On screen: Controlvar. An A/B test, from setup to winner. A/B testing and feature flags you run yourself.
On screen: Signed in off camera. Set up a checkout test.
Set up a checkout test: choose New Experiment.
On screen: Name it, and the key fills itself in.
Name it, and the key your code uses fills itself in.
On screen: Pick the event that counts as a win.
Pick the event that counts as a win: here, a purchase.
On screen: Two variants: Control, and One page.
Call the other variant One page, with a configuration for the store.
On screen: Size it with today's rate and the lift worth catching.
Enter today's rate, and the smallest lift worth catching.
On screen: Review it: the 5% lift is saved with it.
Review it, with the 5% lift saved, then create it.
On screen: Start it, and it is ready for shoppers.
Choose Start, and it's ready for shoppers.
On screen: This shopper, set up off camera, gets 3 steps.
In ShopLab, a demo storefront, this shopper gets 3 steps.
On screen: Second shopper, set up and form filled off camera: one page.
A second shopper gets everything on one page.
On screen: Order placed. The purchase is tracked for the test.
They buy, and the purchase is tracked for the test.
On screen: Off camera. 8,000 shoppers later. Simulated by the tool: 8,000 shoppers at checkout, set to buy at 62% in Control and 68% with One page, sent in 3 minutes (measured), so the results page reads Duration 0d. The demo apps run from their own build in this recording.
On screen: 8,000 shoppers later, simulated in minutes.
8,000 shoppers later, simulated for this video.
On screen: Computed off camera first: One page wins, 9.4% lift, significant at 95%.
In the dashboard, One page wins: a 9.4% lift in purchases. Significant at 95% confidence.
On screen: Controlvar. Open source, Apache-2.0, runs on your own infrastructure. controlvar.com. github.com/Controlvar/controlvar. Recorded on Controlvar v0.36.0, a local stack with demo data.
Assign users from your warehouse
Your own SQL selects users in your warehouse, a script you write sends them to the batch assign API (beta), targeting admits some, and a deterministic hash splits them.
Animation without sound
Length: 1 minute 13 seconds
Transcript: Assign users from your warehouse
On screen: Your users already live in Snowflake or BigQuery, with attributes like country, age band, plan, device and app version.
On screen: Your own SQL query picks the eligible users and the attributes your targeting rules need.
On screen: A short sync script you write sends them in chunks of up to 1,000 users to POST /api/v1/tracking/assign/batch (beta), each user's attributes as context, with an API key.
On screen: 60 requests a minute per address. Over that, the API answers 429 with Retry-After; wait and resend. Resending is safe because assignments are sticky.
On screen: The experiment's targeting rules admit some users. The rest get assigned: false, reason: "targeting": they see the control and nothing is recorded.
On screen: Admitted users are split by a deterministic hash: the same user always lands in the same bucket, and an assigned user keeps their variant.
On screen: The response lists every user in request order, with counts for each reason. Enrolled users count in results from the moment they are assigned.
On screen: The experiment is in the dashboard. When a user arrives, the SDK's assign call returns the same variant and records the view.
Design an experiment in depth
Past the basics of guided setup: a second metric, a third version, who can join, the correction and sequential testing (beta); then a start date, a global holdout and an exclusion group.
Narrated recording
Length: 2 minutes 18 seconds
Transcript: Design an experiment in depth
On screen: Controlvar. Design an experiment in depth. A/B testing and feature flags you run yourself.
On screen: Signed in off camera. Past the basics of guided setup.
Choose New Experiment.
On screen: Name it, and the key fills itself in.
Name it, and the key your code uses fills itself in.
On screen: The primary metric: the one you decide on.
Make purchases the primary metric: the one you decide on.
On screen: Add a metric to track alongside it.
Add a second metric to track alongside it: add to cart.
On screen: Add a third version, and split the traffic.
Add a third version, and split the traffic so it reaches 100%.
On screen: Choose who can join, from what your app sends.
Choose who can join from what your app sends: US or Canada. Everyone else gets the control, and isn't enrolled.
On screen: Confidence and correction lock once it starts.
Next, a confidence level and a correction, which lock once it starts.
On screen: Several versions? Correct for lucky winners.
With several versions, one can look like a winner by luck. Benjamini-Hochberg, the default, keeps false winners to about 5% of all winners.
On screen: Optional: a Bayesian view of the primary metric.
Turn on Bayesian to also see each version's chance of being best.
On screen: Optional: sequential testing (beta).
Turn on sequential testing, in beta. Check results whenever you like, and stop once the evidence is strong.
On screen: Review your choices, then create it.
Review your choices, then create it.
On screen: Saved as a draft, with its versions, metrics and analysis.
It's saved as a draft, with its versions, metrics and analysis.
On screen: Schedule its start: it starts on its own.
Give it a start date, and it starts on its own. Until then, it stays a draft.
On screen: Global holdouts: one page, for every experiment.
A holdout covers all your experiments, so it has its own page.
On screen: Create one, and activate it now.
A global holdout keeps 5% of new users out of every experiment. That includes this one.
On screen: Active: its size is now fixed.
It's active now, and its size is fixed.
On screen: Checkout tests, made off camera with Checkout Button Color.
Experiments in the same exclusion group never share a user.
On screen: Add this draft to the group.
Add this draft to the group with Checkout Button Color.
On screen: Still a draft: sequential on, a start, a group, a holdout.
Read it back: it's still a draft, in its group. It has its start date, and sequential testing is on. And the holdout is active.
On screen: Controlvar. Open source, Apache-2.0, runs on your own infrastructure. controlvar.com. github.com/Controlvar/controlvar. Recorded on Controlvar v0.39.0, a local stack with demo data.

Walkthroughs

Get started

Your first experiment

Roll out a feature flag

Roll out to devices, roll back by itself

Serve a value with a feature flag

Plan a test, bring your team

An A/B test, from setup to winner

Assign users from your warehouse

Design an experiment in depth