Welcome!

Blog Feed Post

A/B Testing and Web Performance

A/B testing (or split testing) is a method of comparing two or more versions of a webpage or app against each other to determine which one has better response or conversion rate.

A/B tests allow you to test one version of images, colors, headings, forms etc… against one another. Layout choices such as where you place the elements, or functionality choices like when and how you display messages can also be tested.

It can also be used to improve drastic changes like redesigning a page by only serving it to a portion of your visitors. Companies use this strategy to test major interface changes by only rolling out the new version to a segment of their visitors and measuring how that group reacts.

http://blog.catchpoint.com/wp-content/uploads/2017/05/ab1-206x300.png 206w" sizes="(max-width: 381px) 100vw, 381px" />

Performance Impact

While A/B testing helps in achieving better conversion rates and making good decisions in site redesign, on the other hand, it may also impact the web performance.

Most A/B testing software creates an additional step in loading and rendering a web page. Many of these use a technology based on JavaScript that is executed on the visitor’s browser. The client-side JavaScript creates a variation of your test page by doing manipulations on your browser. The script modifies the source code as the page is loaded and all the variations are rendered by the browser; the page variation cannot be implemented until the elements that need to be modified are first loaded. During this process you may notice elements on the page changing as loading progresses.

Another issue resulting from A/B testing is flickering. This happens when the website visitor sees the original page content for a fraction of time before the page variation loads or before the testing script changes the targeted elements causing a delay. The presence of such scripts can block certain sections of the page from rendering and slow it down even more.

The image below shows data collected from an experiment that tested the page performance with and without the A/B testing tag. We can clearly see form the result that the page without the testing JS is faster.

http://blog.catchpoint.com/wp-content/uploads/2017/05/ab2-300x34.png 300w, http://blog.catchpoint.com/wp-content/uploads/2017/05/ab2-768x87.png 768w" sizes="(max-width: 816px) 100vw, 816px" />

When we drill down further to the request level, we can see the full impact of the testing tag. The JS request itself contributes an average of almost 600 ms to the overall response time of the page.

http://blog.catchpoint.com/wp-content/uploads/2017/05/ab3-300x140.png 300w, http://blog.catchpoint.com/wp-content/uploads/2017/05/ab3-768x358.png 768w" sizes="(max-width: 925px) 100vw, 925px" />

Breaking down the different components of the network that makes up the response time of the JS tag gives us the following details.

http://blog.catchpoint.com/wp-content/uploads/2017/05/ab4-300x32.png 300w, http://blog.catchpoint.com/wp-content/uploads/2017/05/ab4-768x83.png 768w" sizes="(max-width: 834px) 100vw, 834px" />

The page load time is directly impacted by the size of the variation code used on the page. The JS content would require some additional time to parse, execute and make changes to the original content thus adding to the delay in rendering the page variations on the visitor’s browser.

Type of Testing Tools

A/B testing can be performed by choosing either server-side or client-side tools.

1. Server-side:

Server-side tools do not perform any changes in the browser, instead, a randomly picked version of the test page is served. It requires the developers to get involved in the testing process for creating different versions of the page. Server-side testing requires more investment and work. However, the server-side tools offer greater flexibility and control. They protect your end user privacy and security better.

2. Client-side:

This is the most common type of testing tool. As we have seen earlier, client-side tools use JavaScript to perform the manipulation/changes at the visitor’s browser. Client-side tools are easier to use; you can get the tests set up and running faster without burning many resources.

Client side JavaScript can be made to load either synchronously or asynchronously.

http://blog.catchpoint.com/wp-content/uploads/2017/05/ab5-300x116.png 300w" sizes="(max-width: 650px) 100vw, 650px" />

Synchronous: Synchronous code allows the JS to load sequentially one after the other starting from the <head> tag. Loading synchronously reduces the risk of flickering but it adds an overhead to the page load time as the page doesn’t render anything else while the JavaScript is loading. The users must wait until the JavaScript is loaded and parsed before the rest of the page renders.

Asynchronous: Asynchronous loading allows multiple scripts to load simultaneously reducing the page load time. This technique is usually preferred as page load time is an important factor. However, asynchronous loading doesn’t guarantee that the variation script will load before the targeted elements loads. This could cause flickering but it is possible to implement certain tweaks to minimize it.

3. Proxy server method:

Another type of tool acts as a proxy server between the visitor’s browser and your server and it controls the site content that is presented to the visitor. It eliminates most of the problems that occur with client-side tools, while retaining the robustness of server-side tools.

The following best practices can help you reap the benefits of A/B testing while also mitigating the risks it presents to web performance:

  • Serve the testing tag from CDN/multi-CDN to ensure it is delivered as fast as possible.
  • Make sure that everything loads fast by moving the site to a fast web host or by employing performance-tuning techniques such as compression and/or image optimization.
  • Limit the changes to a few HTML elements. Adding variations to numerous elements will make flickering more likely.
  • Ensure any asynchronous code in the page is configured to hide the original page until the testing script is fully loaded and executed.
  • The script can be implemented with a timeout value. This is useful in cases where the users have a slow internet connection. If the script doesn’t get loaded within the timeout threshold, the users are provided the original content instead of performing slow changes.
  • It is not wise to integrate the A/B testing tool script within the tag manager as this might cause delays in loading the tags.
  • Make sure that the order of execution of the various JS matches the order of elements on the original page. If the starting line of your script tries to make changes to the HTML element at the bottom, it will not happen until all the page is loaded and cause a delay.
  • Structure the variation code in such a way the changes are implemented from top to bottom.

The post A/B Testing and Web Performance appeared first on Catchpoint's Blog - Web Performance Monitoring.

Read the original blog entry...

More Stories By Mehdi Daoudi

Catchpoint radically transforms the way businesses manage, monitor, and test the performance of online applications. Truly understand and improve user experience with clear visibility into complex, distributed online systems.

Founded in 2008 by four DoubleClick / Google executives with a passion for speed, reliability and overall better online experiences, Catchpoint has now become the most innovative provider of web performance testing and monitoring solutions. We are a team with expertise in designing, building, operating, scaling and monitoring highly transactional Internet services used by thousands of companies and impacting the experience of millions of users. Catchpoint is funded by top-tier venture capital firm, Battery Ventures, which has invested in category leaders such as Akamai, Omniture (Adobe Systems), Optimizely, Tealium, BazaarVoice, Marketo and many more.

Latest Stories
With major technology companies and startups seriously embracing Cloud strategies, now is the perfect time to attend 21st Cloud Expo October 31 - November 2, 2017, at the Santa Clara Convention Center, CA, and June 12-14, 2018, at the Javits Center in New York City, NY, and learn what is going on, contribute to the discussions, and ensure that your enterprise is on the right path to Digital Transformation.
SYS-CON Events announced today that MobiDev, a client-oriented software development company, will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. MobiDev is a software company that develops and delivers turn-key mobile apps, websites, web services, and complex software systems for startups and enterprises. Since 2009 it has grown from a small group of passionate engineers and business...
What's the role of an IT self-service portal when you get to continuous delivery and Infrastructure as Code? This general session showed how to create the continuous delivery culture and eight accelerators for leading the change. Don Demcsak is a DevOps and Cloud Native Modernization Principal for Dell EMC based out of New Jersey. He is a former, long time, Microsoft Most Valuable Professional, specializing in building and architecting Application Delivery Pipelines for hybrid legacy, and cloud ...
The Internet giants are fully embracing AI. All the services they offer to their customers are aimed at drawing a map of the world with the data they get. The AIs from these companies are used to build disruptive approaches that cannot be used by established enterprises, which are threatened by these disruptions. However, most leaders underestimate the effect this will have on their businesses. In his session at 21st Cloud Expo, Rene Buest, Director Market Research & Technology Evangelism at Ar...
SYS-CON Events announced today that GrapeUp, the leading provider of rapid product development at the speed of business, will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Grape Up is a software company, specialized in cloud native application development and professional services related to Cloud Foundry PaaS. With five expert teams that operate in various sectors of the market acr...
SYS-CON Events announced today that Ayehu will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on October 31 - November 2, 2017 at the Santa Clara Convention Center in Santa Clara California. Ayehu provides IT Process Automation & Orchestration solutions for IT and Security professionals to identify and resolve critical incidents and enable rapid containment, eradication, and recovery from cyber security breaches. Ayehu provides customers greater control over IT infras...
21st International Cloud Expo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy. Me...
DevOps at Cloud Expo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 21st Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to w...
We build IoT infrastructure products - when you have to integrate different devices, different systems and cloud you have to build an application to do that but we eliminate the need to build an application. Our products can integrate any device, any system, any cloud regardless of protocol," explained Peter Jung, Chief Product Officer at Pulzze Systems, in this SYS-CON.tv interview at @ThingsExpo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA
"Loom is applying artificial intelligence and machine learning into the entire log analysis process, from start to finish and at the end you will get a human touch,” explained Sabo Taylor Diab, Vice President, Marketing at Loom Systems, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
With major technology companies and startups seriously embracing Cloud strategies, now is the perfect time to attend 21st Cloud Expo October 31 - November 2, 2017, at the Santa Clara Convention Center, CA, and June 12-14, 2018, at the Javits Center in New York City, NY, and learn what is going on, contribute to the discussions, and ensure that your enterprise is on the right path to Digital Transformation.
With the introduction of IoT and Smart Living in every aspect of our lives, one question has become relevant: What are the security implications? To answer this, first we have to look and explore the security models of the technologies that IoT is founded upon. In his session at @ThingsExpo, Nevi Kaja, a Research Engineer at Ford Motor Company, discussed some of the security challenges of the IoT infrastructure and related how these aspects impact Smart Living. The material was delivered interac...
Artificial intelligence, machine learning, neural networks. We’re in the midst of a wave of excitement around AI such as hasn’t been seen for a few decades. But those previous periods of inflated expectations led to troughs of disappointment. Will this time be different? Most likely. Applications of AI such as predictive analytics are already decreasing costs and improving reliability of industrial machinery. Furthermore, the funding and research going into AI now comes from a wide range of com...
The financial services market is one of the most data-driven industries in the world, yet it’s bogged down by legacy CPU technologies that simply can’t keep up with the task of querying and visualizing billions of records. In his session at 20th Cloud Expo, Karthik Lalithraj, a Principal Solutions Architect at Kinetica, discussed how the advent of advanced in-database analytics on the GPU makes it possible to run sophisticated data science workloads on the same database that is housing the rich...
"When we talk about cloud without compromise what we're talking about is that when people think about 'I need the flexibility of the cloud' - it's the ability to create applications and run them in a cloud environment that's far more flexible,” explained Matthew Finnie, CTO of Interoute, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.