A/B Testing Your Business Logic: How to Run Experiments with Verbs

A/B testing is a cornerstone of modern product development. We obsess over button colors, headline copy, and checkout flows to eke out conversion gains. But what if the biggest levers for growth aren't on the surface? What if they're buried deep within your business logic?

Imagine A/B testing your core operational rules:

• Which risk assessment model leads to fewer defaults?
• Does a "freemium" or a "free trial" onboarding flow result in higher LTV?
• Which shipping provider algorithm minimizes costs without hurting delivery times?

Traditionally, running these kinds of experiments is a nightmare. Your codebase becomes a tangled web of if/else statements, feature flags, and custom tracking code. It's slow, risky, and difficult to analyze.

This is where treating your Actions as Code changes the game. With Verbs.do, you can A/B test your fundamental business logic as easily as you test a button color, creating a robust, data-driven approach to operational excellence.

The Challenge with Logic-Level Experiments

When you try to A/B test business logic in a conventional application, you immediately run into a few major roadblocks:

1. Code Complexity: Your primary signUp or processPayment function becomes bloated with conditional logic to handle different experimental variants. This increases cognitive load and the chance of introducing bugs.
2. Tracking & Attribution: How do you reliably track the downstream effects of a logic change? Connecting a specific execution variant to a user's behavior weeks later requires complex, custom analytics plumbing.
3. Deployment Risk: Rolling out a new logic variant often requires a full code deployment. Rolling it back is just as cumbersome. This slows down your iteration speed and increases risk.

The core problem is that the "action" (e.g., signing up a user) and the "implementation" (the specific code that runs) are tightly coupled. To experiment, you have to break this coupling.

A Better Way: Experimentation Through Verbs

Verbs.do is built on a simple but powerful idea: define every business operation as a standardized, observable "Verb." A Verb encapsulates the what and the why of an action, separating it from the how.

This separation is the key to clean, effective experimentation.

Let's walk through an example. Suppose we want to A/B test our user sign-up process. The control group will get a standard welcome email. The experiment group will receive a new welcome email that includes an onboarding video.

Step 1: Define the Core Verb

First, we define the SignUp Verb. This definition is consistent across our entire system and represents the single source of truth for what a "sign up" is. It specifies the actor (subject), the data it requires (properties), and what happens next (effects).

import { Verb } from 'verbs.do';

// Define the 'SignUp' action once
const signUp = new Verb({
  name: 'SignUp',
  description: 'A new user creates an account.',
  subject: { type: 'User' },
  properties: {
    email: { type: 'string', format: 'email' },
    password: { type: 'string', sensitive: true }
  },
  // We'll configure the effects dynamically for our A/B test
  effects: [] 
});

Notice the effects array is empty. This is because we're going to tell Verbs.do how to populate it based on our experiment rules.

Step 2: Configure the Experiment

Instead of writing if/else logic in our application code, we can configure an experiment directly within the Verb's definition or through the Verbs.do platform. This configuration dictates which side-effects should run for which user.

Here's a conceptual example of how you might configure a 50/50 split:

// This configuration lives outside your core application logic
signUp.addExperiment({
  name: 'OnboardingEmailTest-Q3',
  variants: [
    {
      name: 'Control',
      weight: 50,
      effects: [
        { name: 'CreateUserRecord' },
        { name: 'SendWelcomeEmail' } // The standard welcome email
      ]
    },
    {
      name: 'VideoOnboarding',
      weight: 50,
      effects: [
        { name: 'CreateUserRecord' },
        { name: 'SendWelcomeEmailWithVideo' } // The new variant
      ]
    }
  ]
});

The CreateUserRecord effect is common to both, but the email effect is now split. Your core business logic is completely unaware of this test.

Step 3: Execute the Verb (Without a Single if Statement)

Now, from anywhere in your application, you execute the SignUp Verb. The calling code is beautifully simple. It doesn't need to know an experiment is even running.

// This code is clean and doesn't change, regardless of the test
await signUp.execute({
  subject: { id: 'user_123' },
  properties: {
    email: 'hello@example.com',
    password: 'a-secure-password'
  }
});

Verbs.do handles the rest. It will check the experiment configuration, assign the user (user_123) to a variant, and trigger the appropriate chain of effects.

Step 4: Analyze the Results with Certainty

This is where the magic happens. Because every Verb execution is an immutable, structured event log, analysis becomes trivial. Every SignUp event is automatically tagged with:

• experiment_name: 'OnboardingEmailTest-Q3'
• experiment_variant: 'VideoOnboarding' (or 'Control')

You now have a perfect audit trail of your experiment. You can easily query your event stream to answer critical questions:

• "What is the 30-day retention rate for users in the VideoOnboarding variant vs. the Control?"
• "Show me all users who were part of this experiment and did not open their welcome email."
• "Compare the average number of key actions performed in the first week by users in each variant."

This powerful event sourcing approach means your data is clean, reliable, and requires no extra analytics instrumentation to track the experiment.

From A/B Tests to Dynamic Workflows

This pattern of separating action from implementation unlocks more than just A/B testing. It's a foundation for business-as-code, enabling practices like:

• Canary Releases: Roll out a new payment processing logic to 1% of users to monitor its performance and error rate in a controlled production environment before a full release.
• Feature Flags: Enable or disable entire workflows for specific user segments (e.g., "Pro" users get an advanced invoicing workflow) without a single code deployment.
• Dynamic Routing: Automatically switch to a backup service provider if the primary one is experiencing an outage, simply by updating a Verb's effect configuration.

By moving your business rules out of hard-coded logic and into a managed, observable Action Management layer, you build a system that is not only more robust but infinitely more adaptable.

Stop building brittle systems. Start running smarter experiments on the logic that truly matters.

Ready to transform your operations into code? Explore Verbs.do and discover the power of defining, executing, and logging every business action.