How Experiments work behind the scenes
This page describes how the Experiments feature works behind the scenes, what data is used to store them, and how do the modifications set up in the Experiment get applied on the webpage.
If you are not a frontend developer and you want to use our Experiments for basic changes, experimentation, or web personalization, you do not need to know how experiments work behind the scenes. This is an advanced topic for advanced users.
Let's start with the data model. Each experiment contains a configuration of AB-test variants. There is always at least one variant (i. e. you cannot have an experiment with no variants).
Each experiment variant contains an ordered list of modifications. The order is important and depends on how you set up the modifications in the Experiments editor. Modifications get applied in the order they are defined.
A modification is an object with a few properties. The most important and common for every modification are type and element. The type defines the type of modification, as you select it in the Experiments editor. You can see the list of possible types with their descriptions in the Experiments documentation. The element is also important – depending on the type of the modification, this is the CSS selector of the element to modify (or remove, or add content near).
Different types of modifications have other attributes as well. For example, the Run script modification contains the text of the script, as well as the selected option when to run the script.
There is one additional difference in how the SDK and the Modifications script handle the HTTP requests to fetch the experiments. The SDK performs one asynchronous request to fetch the list of experiment IDs (which gets filtered), and then fetches the experiments in the second request. This is all done asynchronously and takes some time, which results in the Flickering Effect.
The Modifications script does things differently – when the script itself is requested by the website (through its
<script src="..."> tag), the file is already returned with a list of static modifications for each experiment. Static modifications are those that do not contain any Jinja. Once those are loaded, the Modifications script performs an asynchronous request to fetch the dynamic personalized modifications, which are then applied asynchronously. Note that because the experiment modifications are always applied in the order they are defined, you should place your static modifications before any personalized ones. The Modifications script is loaded only with static modifications up to the first dynamic one, and must load the remaining modifications asynchronously (including any static modifications after the dynamic one).
Again, how the experiment modifications are applied depends on your integration setup (see Integrating and using Experiments for options).
When the SDK is querying elements based on a modification's CSS selector, it only does it once at the time it gets to apply the modification. If the element(s) exist(s), the modification gets applied. If it does not, the SDK does not wait for anything and just skips the modification.
Asynchronous / synchronous integration
If you are using the asynchronous or the synchronous solution, experiment modifications are applied by the Modifications script. The Modifications script receives the list of modifications (first static, then dynamic – read in the section above) and stores them in an internal list of modifications on the current page.
When the Modifications script loads, it sets up a MutationObserver on the whole webpage document. This means that as the page loads, or if any page content changes any time during the page view session, the Modifications script is notified and receives the content that has been added or changed. It looks into its internal list of active modifications, checks whether any of them match the new content, and if they do, it applies the modifications to the new/changed content before the user gets to see it. This is how we can modify even a dynamic webpage content without any flickering effects and without the end-user knowing about it.
This section roughly describes what changes are done to elements in different modification types.
Most visual changes made in this modification are done by either generating the equivalent CSS in a
<style> (Modifications script), or by changing the inline style of the matched element (JS SDK).
The text changes are done by setting
innerText of the matched element, and the HTML changes (done in the Code tab) are done by setting
innerHTML of the element.
A word of caution when changing the content of elements
When you are changing either the text or the HTML content of an element, you should pay attention to selecting the deepest child node possible for the content that you want to edit. For example, if you want to change a few texts in your image carousel, try selecting the smallest text elements in the carousel and editing those instead of selecting the whole carousel element and changing its HTML.
The reason for this is that when you change the carousel's HTML, the SDK simply sets the element's
Inserting new content is done using
document.createDocumentFragment to create elements for the HTML, and then
element.insertBefore to insert the content into the webpage.
Similar to inserting a new content, moving existing content is done using
element.insertBefore with the existing elements on the webpage. Note that both the container CSS selector and the element CSS selector must return the same number of elements.
Depending on the selected mode, the matched element is either hidden using CSS styling, or removed from the DOM using
- immediately, before the page content is loaded – the modification is executed as soon as possible and pretty much ignores its element CSS selector. It is executed regardless of whether the selector matches any element on the page.
- on document ready event with all matched elements – the SDK or the Modifications script waits until the document is loaded (the
DOMContentLoadedevent is fired), then it selects elements using the modification's CSS selector and runs the script. You can use
this.elementsin the script to access the list of matched elements. The modification is executed regardless of whether the selector matches any element on the page (in that case,
this.elements === ).
- once for each matched element – this is a useful mode, which ensures that your script runs exactly once for each matched element. You can access
this.elementto get a reference to the matched element and do any modifications to it. Additionally, the Modifications script ensures that your script gets executed for any dynamic content on the webpage (because it has set up the
MutationObserver), so you can also use this to modify any elements that appear later on the webpage. This approach is also non-flickering – your script runs before the element is actually displayed to the user, so you can do any visual changes to it without flickering.
For each applied modification, the SDK / Modifications script stores an inverse of the modification in a so-called "revert queue". This queue is used whenever the experiment had to be reverted. For example, when the Experiments editor needs to refresh the preview (it needs to revert the previous experiment preview), or when you change pages in a single page application.
Custom run script revert function
Because the "Run script" modification is entirely your own code, we do not know how to revert it! You have to return an object with a
revertfunction from the script for everything to function corretly. Read more here.
Updated over 1 year ago