Controlvar Documentation

Run the stack and create your first experiment.

• Quick Start Run the stack and create your first experiment
• Core Concepts Feature flags, experiments, variants, and metrics
• Architecture Overview How the components fit together, and how data flows between them
• FAQ Common questions answered
• Modules & Profiles What the core profile includes, what each module adds, and how to run the full profile

Create flags, target them with rules, and roll them out in stages.

• Creating Feature Flags Set up your first flag
• Targeting & Rules The operators a targeting rule can use, with examples
• Gradual Rollouts Staged rollout schedules
• Safety Monitoring Roll a flag back when its error rate or latency breaches a threshold

Create and start experiments, read their results, and the statistics behind them.

• Running Experiments End-to-end experiment lifecycle
• Dashboard and API Which capabilities have a dashboard screen, and which work through the API only for now
• Statistical Methods Sequential testing with mSPRT
• CUPED Variance reduction from each user's own pre-experiment events (API only)
• Multi-Armed Bandits Thompson Sampling, UCB1, Epsilon-Greedy (create a bandit through the API)
• Split URL Testing Full profile: give each variant its own URL. Your server or edge, or on AWS an optional CloudFront router, asks Controlvar where to send each visitor, and conversions count for the variant they saw.
• Mutual Exclusion Groups Keep users out of conflicting experiments, and a share of new users out of every experiment
• Balance Within Groups Give each group of users the same split as the whole experiment, set through the API (beta)
• Live Results Streaming Live experiment results over WebSocket: a snapshot on connect, then a fresh one every 30 seconds (by default), whatever the client sends.

Compare model variants on a task; the winner has the best business metric when every variant has one, and otherwise the best LLM-judge score.

• Overview Supported providers, task types, variant assignment, judge scoring, cost estimation and the analysis
• Quick Start Compare GPT-4o with Claude 3.5 Sonnet (steps 3–5 need provider API keys)

Client libraries for web, mobile and server. The JavaScript, React, React Native, Python, Java and .NET SDKs and the OpenFeature providers are on npm, PyPI, Maven Central or NuGet.

• JavaScript SDK Browser and Node.js
• Python SDK A synchronous Python client with no dependencies
• Java SDK Spring Boot 3 auto-configuration starter; thread-safe client with a TTL cache
• React SDK A context provider, hooks and a higher-order component
• Go SDK A Go client for services: standard library only, goroutine-safe, context-aware.
• iOS Swift SDK Native Swift SDK for iOS 14+ and macOS 11+, with an async/await API.
• Android Kotlin SDK Native Kotlin SDK for Android (minSdk 21). Coroutines, OkHttp, Compose examples.
• Flutter SDK Dart SDK for Flutter (iOS, Android, web and desktop). Results are decided by the server, with an offline fallback.
• React Native SDK useFlag and useExperiment hooks, Provider, AsyncStorage offline support.
• MCP server Connect Claude Code, Claude Desktop or Cursor to your instance: read flags and experiment results.
• OpenFeature Provider OpenFeature (a CNCF project) providers
• Edge SDK For Cloudflare Workers, Vercel Edge and Deno Deploy. Zero runtime dependencies; results cached in memory or KV.
• Ruby SDK Ruby gem with zero runtime dependencies, a thread-safe TTL cache and Net::HTTP.
• PHP SDK For PHP 8.1+, installed with Composer; it needs only ext-curl and ext-json.
• .NET SDK Targets net8.0 and netstandard2.1; System.Text.Json and HttpClient, no other dependencies.
• Elixir SDK For Elixir and Phoenix: OTP's :httpc and Jason, with a GenServer-backed ETS cache.

How to call the REST API: authentication, and the endpoints with example requests.

• Authentication API keys for SDKs, bearer tokens for people
• Experiments List experiments, get one, and get its results
• Feature Flags List flags, a user's flags, and archived flags
• Compliance & Audit Full profile: signed audit events and evidence-report export
• Integrations Jira, Salesforce, GitHub webhooks
• All Endpoints Endpoint reference, with example requests and error responses

The AWS services Controlvar runs on, the webhooks it receives, and the alerts it sends.

• AWS How the platform uses ECS Fargate, Aurora, ElastiCache and DynamoDB
• Jira Inbound webhooks, signed or with a shared secret
• Salesforce Inbound webhooks from a Flow or Apex callout
• GitHub Inbound webhooks, each checked by its HMAC-SHA256 signature
• Slack & Email Slack messages for scheduled experiment changes and rollout stage advances; email only from the test route

Audit records, compliance reports, roles, API keys and single sign-on.

• Compliance audit logging An append-only record of changes to experiments and flags; signing and export in the full profile
• Compliance evidence Full profile: SOC 2 and ISO 27001 reports and an export of the audit events
• Audit Logging Who changed what through the API; no API edits or deletes an entry
• RBAC Admin, Developer, Analyst and Viewer decide what a user can do; in the full profile, custom roles and direct grants add to them
• API Key Management Create, expire and delete keys; the sdk:ruleset scope for local evaluation
• SSO / SAML / OIDC Full profile: sign in through Okta, Google or GitHub (OIDC). SAML 2.0 sign-in is not available yet; use OIDC.
• HIPAA controls Full profile: PHI encryption (Fernet), a 6-year retention marker on PHI audit records, BAA records and the data-residency configuration.

Run Controlvar yourself: Docker Compose on one machine, Helm on Kubernetes, or a CDK app that deploys to AWS.

• AWS CDK Deployment CDK app for ECS Fargate, Aurora and ElastiCache
• Docker Compose The published images on one Docker host: deploy/compose/compose.yml, and a README that says how to run it and lists its settings
• Kubernetes (Helm) The Helm chart in charts/experimently: a one-time secret step, then one install command
• Environment Variables Configuration reference
• Database Migrations Alembic migration guide
• Monitoring CloudWatch dashboards, Prometheus metrics

Walkthroughs and method guides.

• Your First A/B Test End-to-end experiment walkthrough
• Bayesian Experimentation Beta-Binomial posteriors and stopping rules
• Guided Experiment Builder Guided setup in the dashboard, and the wizard API
• Interaction Detection Beta: find experiments that share users, and whether their effects interact

How power analysis, post-stratification and FDR correction work.

• Power Analysis & Sample Size Pre-experiment planning: MDE, power and runtime estimation
• Post-Stratification The method, explained; not available through the API yet
• FDR Correction Benjamini-Hochberg false discovery rate correction for multiple metrics

Group members into teams, projects or product areas with workspace roles. Workspaces do not limit access to experiments or flags; that is by platform role.

• Workspace Overview What workspaces do and do not do, workspace roles and invitations
• Quickstart Create a workspace, add your team to it, and connect an SDK