top of page
Writer's pictureKelvin Luhr
Oct 256 min read

Setting Up Custom Segments for A/B Testing in Piwik PRO: A Step-by-Step Guide

Ever run an A/B test and wished you could easily track its impact in Piwik PRO? Although Piwik PRO does not provide native A/B testing features, its capabilities in handling custom dimensions and segments can be effectively used to monitor your experiments.

Note that Piwik PRO itself is not an A/B testing tool; to divide users into test groups and activate tests, especially when running server-side, you will need a dedicated A/B testing platform.

This guide outlines how to integrate these tools, from setting up custom dimensions in Piwik PRO to tracking results, all aimed at gathering data for a Bayesian A/B Testing dashboard. For detailed instructions on setting up such a dashboard, refer to our companion guide here.


 

Step 1: Select an A/B Testing Tool and Setup Type


Choose an A/B testing platform that aligns with your testing requirements:


  • Recommended Tools: Consider platforms like Optimizely, VWO, or AB Tasty, which support both client-side and server-side testing.


  • Integration: Ensure seamless integration with Piwik PRO for comprehensive data tracking.


  • Consideration of Testing Type: Server-side testing offers more control and often better performance, but if you opt for client-side testing, ensure that your tool's configuration is compatible with Piwik PRO's data tracking mechanisms.


 

Step 2: Define Your Experiment Details


When setting up your A/B test, align your experiment parameters with your chosen A/B testing platform:


  • Experiment Name: This should be a clear and descriptive label for your test, such as "Button Color Test".


  • Experiment Variations: Label the different versions being tested, like "Original" and "Variation-1".


  • Experiment ID (Optional): Use a unique identifier to manage multiple tests simultaneously, such as "Experiment 1".

    Important Note: If you choose to implement an Experiment ID, keep in mind that you’ll only be able to run one test at a time. Running a new test while another is active will cause the tracking to overwrite the previous test’s ID with the latest one, resulting in only the most recent test ID being stored.


Tip: Tailor these details to the naming conventions and configurations of your chosen A/B testing tool to ensure data consistency and accuracy.


 

Step 3: Setting Up Custom Dimensions in Piwik PRO


To capture detailed experiment information, set up custom dimensions in Piwik PRO:


  1. Log In to Piwik PRO: Access your account.


  2. Navigate to Settings: Locate this in the menu bar.


  3. Go to Custom Dimensions: Click on "Custom Dimensions" in the left sidebar.


  4. Select Dimension Type: Opt for "Session Dimension", which is best suited for tracking A/B test data.


  5. Create New Dimensions: Name your dimensions, such as "Experiment Name", "Experiment Variation", "Experiment ID".


  6. Note Dimension IDs: Each dimension will receive a unique ID, visible next to its name, necessary for later steps in data tracking.


 

Step 4: Setting Up Data Layer for Experiment Tracking with Piwik PRO Tag Manager


Configure the data layer to track crucial experiment details, suitable for both server-side and client-side setups:


  • Consult with Your Developer: Ensure that the dataLayer includes essential experiment details.


  • DataLayer Push Structure: Provide this structured example to your developer for implementation:

dataLayer.push({
  'event': 'An Event Name', // e.g., 'AB Test'
  'experimentName': 'Your Experiment Name Here', // e.g., 'Button Color Test'
  'experimentVariation': 'Current Variation Name', // e.g., 'Original'
  'experimentId': 'Your Experiment ID Here' // Optional, e.g., 'Experiment 1'
});

Tip: This script should be triggered when a user interacts with a test variation. Tailor the event and other variable names according to the specific needs of your A/B testing platform and the conventions used in your dataLayer setup. For client-side implementations, ensure these variable names match those specified in your A/B testing tool’s documentation to maintain accuracy and compatibility.


 

Step 5: Configure Tags and Triggers in Piwik PRO Tag Manager


This step involves setting up a tag that incorporates all the custom dimensions you've created and linking it with a trigger that activates based on a specific event in the data layer.

Follow these detailed steps:


  • Navigate to Tag Manager: Log into your Piwik PRO account and go to the Tag Manager under your site settings.


  • Create Variables for Tag:

    • Go to the Variables Section: Click on "Variables" and then "Create New Variable" for each of the following: experimentName, experimentVariation, and experimentId. Make sure these variable names are consistent with those specified in the data layer.


  • Create a Tag:

    • Add New Tag: Click on "Create New Tag".

    • Tag Configuration:

      • Name Your Tag: For clarity, name it something like "A/B Test Details Tag".

      • Select Tag Type: Choose "Custom Dimension".

      • Setup Custom Dimensions:

        1. Dimension for Experiment Name: Set the first custom dimension to capture experimentName, and link it to the variable experimentName.

        2. Dimension for Experiment Variation: Set the second custom dimension to capture experimentVariation, and link it to the variable experimentVariation.

        3. Dimension for Experiment ID: Set the third custom dimension to capture experimentId, and link it to the variable experimentId.


  • Configure Trigger:

    • Trigger Setup:

      • Add Trigger: Click "Add a Trigger".

      • Trigger Type: Choose "Data Layer Event".

      • Trigger Configuration:

        • Name Your Trigger: Label it descriptively, such as "AB Test Trigger".

        • Event Condition: Set the condition to listen for an event named "AB Test" or whatever you named the event in Step 4.

        • Condition Type: Set to "equals", and then specify the event name exactly as it appears in your data layer push code.


  • Review and Publish:

    • Check All Settings: Review the tag and trigger settings to ensure they correctly capture and react to the specified data layer events.

    • Publish: Once all settings are verified, save and publish the tag to activate it on your site.


Example:


  • Tag Name: "A/B Test Details Tag"


  • Custom Dimensions:

    • Dimension 1: Experiment Name linked to variable experimentName

    • Dimension 2: Experiment Variation linked to variable experimentVariation

    • Dimension 3: Experiment ID linked to variable experimentId


  • Trigger: "AB Test Trigger" configured to fire on data layer event "AB Test"


By configuring your tags and triggers in this manner, you ensure that every interaction that matches your A/B test configurations is captured and categorized accurately, providing reliable data for analysis in Piwik PRO.


 

Step 6: Create a Custom Dashboard and analyze Your A/B Test Results


To effectively monitor and analyze your A/B test results, setting up a custom dashboard in Piwik PRO allows you to visualize key metrics and dimensions at a glance. If you prefer a more streamlined approach using Bayesian methods, you might consider using our free template specifically designed for Bayesian A/B testing. Click here to access the post with the guide on how to download and set it up. Follow these steps to create and configure your dashboard in Piwik PRO:


  • Navigate to Dashboards:

    • Access Dashboards: Log into your Piwik PRO account and go to the "Dashboards" section from the main menu.

    • Add a New Dashboard: Click on the "Add a dashboard" button. This will prompt you to set up a new dashboard.


  • Configure Your Dashboard:

    • Name Your Dashboard: Give your dashboard a descriptive name that reflects its purpose, such as "A/B Test Analysis for Experiment 1".

    • Visibility Settings: Choose whether the dashboard should be visible to all users or just to you. This is important for managing access if the dashboard contains sensitive data or if it’s for personal use.


  • Add Widgets:

    • Select Dashboard: Make sure your newly created dashboard is selected in the left menu.

    • Add a Widget: Click the "Add a widget" button.

    • Choose Widget Type: Select "Flat Table" as the widget type. This format is ideal for displaying detailed data rows.

    • Configure Metrics and Dimensions:

      • Add Dimensions: Type in the names of your custom dimensions such as 'Experiment Name', 'Experiment Variation', and 'Experiment ID'. Drag these into the metric/dimension box on the widget configuration screen.

      • Include Metrics: Add quantitative metrics like 'Visitors' and 'Orders' to track the number of visitors and conversions associated with each test variation.


  • Set Filters:

    • Apply Filters: Under the "Filter dimension" section of your widget setup, choose a dimension to filter by, such as 'Experiment ID'.

    • Configure Filter Criteria: Set the filter to "equals" and type in the unique ID for your test, like "Experiment 1". This ensures that the dashboard only displays data relevant to a specific A/B test.


  • Save and Review:

    • Save Widget: After setting up your widgets and filters, save the changes to your dashboard.

    • Review Dashboard: Navigate through your dashboard to ensure all widgets are displaying the intended data accurately and that filters are correctly applied.


This customized dashboard setup enables you to dynamically track and analyze the performance of your A/B tests directly within Piwik PRO, offering immediate insights into test effectiveness and user engagement.


 

By following these steps, you’ll be able to track and analyze your A/B tests effectively within Piwik PRO, even without native A/B testing functionality. If you want to go further and set up a Bayesian AB Testing dashboard with this data, be sure to check out our other guide here for a detailed walkthrough. Enjoy your testing and data-driven decision-making!

bottom of page