Developer Guide to Bandito

Before we go into details about all the API endpoints, it is important to understand some basics about how Bandito works, and how it can be implemented in different CMS/contexts.

Workflow

This diagram explains how tests are loaded:

Let’s go through the steps:

1. JavaScript identifies the tests on the page (from metadata provided by the CMS/PB).
2. JavaScript queries Bandito with those test IDs.
3. Bandito responds with current winning variant for each test (and additional metadata, see below for the test structure).
4. Based on the winner information, JavaScript updates the page to display the right variant.
5. JavaScript sends a “served” event for each test/variant (and a click event later, if a click occurs).

JavaScript

Bandito does NOT provide the JavaScript code in the example above (except for PageBuilder, which offers native integration). It is the responsibility of the developer.

Identifying Tests on the Page

The Content Management system needs to be aware of the content that is being tested, and should have available, for each page, the tests running on the page.

Updating the Page

The only thing Bandito does is calculating a winner (ie which variant to display at any point in time). Since Bandito does not modify the page, it is the responsibility of the calling script to do something with this information. The behavior can be simple (show/hide variants), or it can be more complicated (query a 3rd party with the test/variant IDs and then take action).

For reference, here is how the PageBuilder implementation works :

• The default variant is in the initial HTML of the page, hidden
• Once the JS receives the winner, it either unhides the default, or swaps it out for the winning variant

For PageBuilder, we also have a number of safeguards to make sure that variants display in a timely manner, and even if errors occur. If the call to Bandito fails (too long, or just fails), then we show the default.

Sending Impressions/Clicks

Once the correct variants have been displayed, the JavaScript also needs to send an “impression/display/served” event to the API. If one of the variant is clicked (or any other metric of success, such as a video start), then the JavaScript should send a “click” event as well. The “served” event is separate from the “request test winners” call because it’s possible displaying the winning variant will fail for one reason or another, in which case the JavaScript will probably want to show the default variant. We need to associate a potential click later with the correct variant.

Separation of Endpoints

The Bandito API can be pided in 3 parts :

• Creating, updating & deleting tests
• Querying existing tests (and winners)
• Sending test events (impressions, clicks)

Because Bandito is usually part of the “critical path” on a page, those 3 concerns are separated to provide greater resiliency and performance. Thus, the API is distributed across three distinct servers or endpoints:

• bandito-api.ORG.arcpublishing.com provides CRUD operations on tests
• bandito-tests.ORG.arcpublishing.com allows querying tests
• bandito-events.ORG.arcpublishing.com receives events

This ensures that:

• We can keep querying tests extremely fast (we use an Nginx module written in C, which talks directly to the DB)
• We can isolate recording events from the rest (events go to Kafka, which is extremely resilient, and gets processed later in the background)

General Concepts

Test IDs

Bandito is integrated with PageBuilder, but if you’re going to use it through the API, it helps to understand the role of IDs a bit. Because Bandito is a “Content Testing” tool, it will primarily be used within the context of a CMS or other provider of content. When the content is displayed, a query will be made to Bandito to see if there are any tests running, and if one is running, which variant is currently winning. The primary way to query tests is through their ID. Test IDs in Bandito are not automatically assigned (ie, no UUID). IDs are provided by the user. This means in particular, since you will query tests by ID, that the ID needs to make sense to you, and sense within the display context.

The most important thing is that:

• The ID is unique.
• The ID allows you to quickly identify the source of the test (if you use multiple systems, you might want to use the system name as a prefix).
• The ID can be generated from the frontend, where the test will actually run.

So for example, for Wordpress, you could use wp-1234 where 1234 is the unique Post ID within your Wordpress installation. If you have 2 different Wordpress backends, you might want to identify further: wp1- and wp2- for example. Or wpnews- and wpblog-.

Variant IDs

Variant IDs, like test IDs, are provided by the user. One of the variants IDs needs to be the same as the test ID. This is how Bandito finds out which variant is the default. Ideally, this should instead be a field (default = true), but because Bandito was born as an extension to PageBuilder, a number of assumptions and semantics from PageBuilder persist. The default variant ID = test ID is one of them.

Structure of a Test

Here is what a typical test looks like: Test are stored in Postgres DB SQL tables with Django as the framework

{
  "_id": "test1",
  "name": "test one",
  "location": "location",
  "location_id": "location_id",
  "creator_name": "John Doe",
  "status": "active",
  "winner": "v1",
  "variants": [
    {
      "_id": "test1",
      "name": "default",
      "status": "active",
      "rendering": "<p>...</p>",
      "served": 0,
      "clicked": 0,
      "ucb_value": 0.0,
      "click_through_rate": 0.0,
      "metadata": {
        "foo": "bar"
      },
      "created_date": "2016-11-10T13:18:34.278+0000",
      "modified_date": "2016-11-10T13:18:34.278+0000"
    },
    {
      "_id": "v2",
      "name": "variant 2",
      "status": "active",
      "rendering": "<p>...</p>",
      "served": 0,
      "clicked": 0,
      "ucb_value": 0.0,
      "click_through_rate": 0.0,
      "metadata": {
        "foo": "baz"
      },
      "created_date": "2016-11-10T13:18:34.280+0000",
      "modified_date": "2016-11-10T13:18:34.280+0000"
    }
  ],
  "creation_date": "2016-11-10T13:18:34.283+0000",
  "modified_date": "2016-11-10T13:18:34.283+0000"
}

Field Details

_id	The test ID. As explained above, this is NOT automatically generated, but provided by the user.
name	A name for the test
location_id	Where is this test running? For PageBuilder tests, this would be the ID of the page. For tests from other sources, this can be anything relevant to your use case.
location	Same as above, but name
creator_name	Who created this test? Mostly used for reporting (dashboard, etc).
status	Active or inactive
winner	The ID of the variant to display (currently winning)
variants	A list of variants

Variant Details

_id	The ID of the variant. The default variant needs to have ID = test ID.
name	The name of the variant
status	Is the variant active/inactive
rendering	Used in the dashboard/emails to show a preview of the variant
metadata	Fields provided by the user
served clicked ucb_value click_through_rate	Statistics