Blog Feed Post

A/B Testing and Web Performance

A/B testing (or split testing) is a method of comparing two or more versions of a webpage or app against each other to determine which one has better response or conversion rate.

A/B tests allow you to test one version of images, colors, headings, forms etc… against one another. Layout choices such as where you place the elements, or functionality choices like when and how you display messages can also be tested.

It can also be used to improve drastic changes like redesigning a page by only serving it to a portion of your visitors. Companies use this strategy to test major interface changes by only rolling out the new version to a segment of their visitors and measuring how that group reacts.

http://blog.catchpoint.com/wp-content/uploads/2017/05/ab1-206x300.png 206w" sizes="(max-width: 381px) 100vw, 381px" />

Performance Impact

While A/B testing helps in achieving better conversion rates and making good decisions in site redesign, on the other hand, it may also impact the web performance.

Most A/B testing software creates an additional step in loading and rendering a web page. Many of these use a technology based on JavaScript that is executed on the visitor’s browser. The client-side JavaScript creates a variation of your test page by doing manipulations on your browser. The script modifies the source code as the page is loaded and all the variations are rendered by the browser; the page variation cannot be implemented until the elements that need to be modified are first loaded. During this process you may notice elements on the page changing as loading progresses.

Another issue resulting from A/B testing is flickering. This happens when the website visitor sees the original page content for a fraction of time before the page variation loads or before the testing script changes the targeted elements causing a delay. The presence of such scripts can block certain sections of the page from rendering and slow it down even more.

The image below shows data collected from an experiment that tested the page performance with and without the A/B testing tag. We can clearly see form the result that the page without the testing JS is faster.

http://blog.catchpoint.com/wp-content/uploads/2017/05/ab2-300x34.png 300w, http://blog.catchpoint.com/wp-content/uploads/2017/05/ab2-768x87.png 768w" sizes="(max-width: 816px) 100vw, 816px" />

When we drill down further to the request level, we can see the full impact of the testing tag. The JS request itself contributes an average of almost 600 ms to the overall response time of the page.

http://blog.catchpoint.com/wp-content/uploads/2017/05/ab3-300x140.png 300w, http://blog.catchpoint.com/wp-content/uploads/2017/05/ab3-768x358.png 768w" sizes="(max-width: 925px) 100vw, 925px" />

Breaking down the different components of the network that makes up the response time of the JS tag gives us the following details.

http://blog.catchpoint.com/wp-content/uploads/2017/05/ab4-300x32.png 300w, http://blog.catchpoint.com/wp-content/uploads/2017/05/ab4-768x83.png 768w" sizes="(max-width: 834px) 100vw, 834px" />

The page load time is directly impacted by the size of the variation code used on the page. The JS content would require some additional time to parse, execute and make changes to the original content thus adding to the delay in rendering the page variations on the visitor’s browser.

Type of Testing Tools

A/B testing can be performed by choosing either server-side or client-side tools.

1. Server-side:

Server-side tools do not perform any changes in the browser, instead, a randomly picked version of the test page is served. It requires the developers to get involved in the testing process for creating different versions of the page. Server-side testing requires more investment and work. However, the server-side tools offer greater flexibility and control. They protect your end user privacy and security better.

2. Client-side:

This is the most common type of testing tool. As we have seen earlier, client-side tools use JavaScript to perform the manipulation/changes at the visitor’s browser. Client-side tools are easier to use; you can get the tests set up and running faster without burning many resources.

Client side JavaScript can be made to load either synchronously or asynchronously.

http://blog.catchpoint.com/wp-content/uploads/2017/05/ab5-300x116.png 300w" sizes="(max-width: 650px) 100vw, 650px" />

Synchronous: Synchronous code allows the JS to load sequentially one after the other starting from the <head> tag. Loading synchronously reduces the risk of flickering but it adds an overhead to the page load time as the page doesn’t render anything else while the JavaScript is loading. The users must wait until the JavaScript is loaded and parsed before the rest of the page renders.

Asynchronous: Asynchronous loading allows multiple scripts to load simultaneously reducing the page load time. This technique is usually preferred as page load time is an important factor. However, asynchronous loading doesn’t guarantee that the variation script will load before the targeted elements loads. This could cause flickering but it is possible to implement certain tweaks to minimize it.

3. Proxy server method:

Another type of tool acts as a proxy server between the visitor’s browser and your server and it controls the site content that is presented to the visitor. It eliminates most of the problems that occur with client-side tools, while retaining the robustness of server-side tools.

The following best practices can help you reap the benefits of A/B testing while also mitigating the risks it presents to web performance:

  • Serve the testing tag from CDN/multi-CDN to ensure it is delivered as fast as possible.
  • Make sure that everything loads fast by moving the site to a fast web host or by employing performance-tuning techniques such as compression and/or image optimization.
  • Limit the changes to a few HTML elements. Adding variations to numerous elements will make flickering more likely.
  • Ensure any asynchronous code in the page is configured to hide the original page until the testing script is fully loaded and executed.
  • The script can be implemented with a timeout value. This is useful in cases where the users have a slow internet connection. If the script doesn’t get loaded within the timeout threshold, the users are provided the original content instead of performing slow changes.
  • It is not wise to integrate the A/B testing tool script within the tag manager as this might cause delays in loading the tags.
  • Make sure that the order of execution of the various JS matches the order of elements on the original page. If the starting line of your script tries to make changes to the HTML element at the bottom, it will not happen until all the page is loaded and cause a delay.
  • Structure the variation code in such a way the changes are implemented from top to bottom.

The post A/B Testing and Web Performance appeared first on Catchpoint's Blog - Web Performance Monitoring.

Read the original blog entry...

More Stories By Mehdi Daoudi

Catchpoint radically transforms the way businesses manage, monitor, and test the performance of online applications. Truly understand and improve user experience with clear visibility into complex, distributed online systems.

Founded in 2008 by four DoubleClick / Google executives with a passion for speed, reliability and overall better online experiences, Catchpoint has now become the most innovative provider of web performance testing and monitoring solutions. We are a team with expertise in designing, building, operating, scaling and monitoring highly transactional Internet services used by thousands of companies and impacting the experience of millions of users. Catchpoint is funded by top-tier venture capital firm, Battery Ventures, which has invested in category leaders such as Akamai, Omniture (Adobe Systems), Optimizely, Tealium, BazaarVoice, Marketo and many more.

Latest Stories
Enterprises have taken advantage of IoT to achieve important revenue and cost advantages. What is less apparent is how incumbent enterprises operating at scale have, following success with IoT, built analytic, operations management and software development capabilities – ranging from autonomous vehicles to manageable robotics installations. They have embraced these capabilities as if they were Silicon Valley startups. As a result, many firms employ new business models that place enormous impor...
SYS-CON Events announced today that TidalScale will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. TidalScale is the leading provider of Software-Defined Servers that bring flexibility to modern data centers by right-sizing servers on the fly to fit any data set or workload. TidalScale’s award-winning inverse hypervisor technology combines multiple commodity servers (including their ass...
As popularity of the smart home is growing and continues to go mainstream, technological factors play a greater role. The IoT protocol houses the interoperability battery consumption, security, and configuration of a smart home device, and it can be difficult for companies to choose the right kind for their product. For both DIY and professionally installed smart homes, developers need to consider each of these elements for their product to be successful in the market and current smart homes.
SYS-CON Events announced today that MIRAI Inc. will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. MIRAI Inc. are IT consultants from the public sector whose mission is to solve social issues by technology and innovation and to create a meaningful future for people.
In his general session at 21st Cloud Expo, Greg Dumas, Calligo’s Vice President and G.M. of US operations, will go over the new Global Data Protection Regulation and how Calligo can help business stay compliant in digitally globalized world. Greg Dumas is Calligo's Vice President and G.M. of US operations. Calligo is an established service provider that provides an innovative platform for trusted cloud solutions. Calligo’s customers are typically most concerned about GDPR compliance, applicatio...
Companies are harnessing data in ways we once associated with science fiction. Analysts have access to a plethora of visualization and reporting tools, but considering the vast amount of data businesses collect and limitations of CPUs, end users are forced to design their structures and systems with limitations. Until now. As the cloud toolkit to analyze data has evolved, GPUs have stepped in to massively parallel SQL, visualization and machine learning.
In his Opening Keynote at 21st Cloud Expo, John Considine, General Manager of IBM Cloud Infrastructure, will lead you through the exciting evolution of the cloud. He'll look at this major disruption from the perspective of technology, business models, and what this means for enterprises of all sizes. John Considine is General Manager of Cloud Infrastructure Services at IBM. In that role he is responsible for leading IBM’s public cloud infrastructure including strategy, development, and offering ...
As hybrid cloud becomes the de-facto standard mode of operation for most enterprises, new challenges arise on how to efficiently and economically share data across environments. In his session at 21st Cloud Expo, Dr. Allon Cohen, VP of Product at Elastifile, will explore new techniques and best practices that help enterprise IT benefit from the advantages of hybrid cloud environments by enabling data availability for both legacy enterprise and cloud-native mission critical applications. By rev...
SYS-CON Events announced today that Dasher Technologies will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Dasher Technologies, Inc. ® is a premier IT solution provider that delivers expert technical resources along with trusted account executives to architect and deliver complete IT solutions and services to help our clients execute their goals, plans and objectives. Since 1999, we'v...
SYS-CON Events announced today that NetApp has been named “Bronze Sponsor” of SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. NetApp is the data authority for hybrid cloud. NetApp provides a full range of hybrid cloud data services that simplify management of applications and data across cloud and on-premises environments to accelerate digital transformation. Together with their partners, NetApp emp...
SYS-CON Events announced today that TidalScale, a leading provider of systems and services, will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. TidalScale has been involved in shaping the computing landscape. They've designed, developed and deployed some of the most important and successful systems and services in the history of the computing industry - internet, Ethernet, operating s...
Join IBM November 1 at 21st Cloud Expo at the Santa Clara Convention Center in Santa Clara, CA, and learn how IBM Watson can bring cognitive services and AI to intelligent, unmanned systems. Cognitive analysis impacts today’s systems with unparalleled ability that were previously available only to manned, back-end operations. Thanks to cloud processing, IBM Watson can bring cognitive services and AI to intelligent, unmanned systems. Imagine a robot vacuum that becomes your personal assistant tha...
SYS-CON Events announced today that Massive Networks, that helps your business operate seamlessly with fast, reliable, and secure internet and network solutions, has been named "Exhibitor" of SYS-CON's 21st International Cloud Expo ®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. As a premier telecommunications provider, Massive Networks is headquartered out of Louisville, Colorado. With years of experience under their belt, their team of...
Widespread fragmentation is stalling the growth of the IIoT and making it difficult for partners to work together. The number of software platforms, apps, hardware and connectivity standards is creating paralysis among businesses that are afraid of being locked into a solution. EdgeX Foundry is unifying the community around a common IoT edge framework and an ecosystem of interoperable components.
Infoblox delivers Actionable Network Intelligence to enterprise, government, and service provider customers around the world. They are the industry leader in DNS, DHCP, and IP address management, the category known as DDI. We empower thousands of organizations to control and secure their networks from the core-enabling them to increase efficiency and visibility, improve customer service, and meet compliance requirements.