Blog Feed Post

DevOps tool chain: Shift-Left with Bamboo, SoapUI, JMeter and Dynatrace

Dynatrace is a true player in the DevOps tool chain. We built Dynatrace to provide fast feedback on code quality all the way from Dev, via Continuous Integration/Continuous Delivery, all the way into production. The same Dynatrace solution for every pipeline stage to provide fast metrics-based feedback loops on code changes that you are pushing from left (Dev) to right (Ops).

Shift-Left therefore really means that we want to identify bad code quality early on. We also re-define code quality as not just meeting functional specification. We look into performance, scalability and architecture quality. Many of our users execute Unit-, Web/REST API-, Functional- or Load Tests automatically through Bamboo, Jenkins, TFS or other build servers. Dynatrace automatically levels-up these tests by identifying performance, architectural and scalability regressions across builds introduced by the recent code changes. We do this by automatically analyzing and baselining key application metrics captured while you execute your tests against your application as part of your automated build and pipeline execution.

If a regression is detected, e.g: a code or configuration change results in an increase of SQL Queries, and increase in Microservice calls or changed resource consumption behavior then Dynatrace tells the build server (Jenkins, Bamboo, TFS, …) to stop the build early (left) in the lifecycle. Giving the developers faster feedback on critical quality metrics:

Dynatrace enables real Shift-Left by detecting performance-, scalability and architectural regressions when code changes are pushed through your pipeline.

Shift-Left with Bamboo and Dynatrace

While we often talk about users that integrate Dynatrace into Jenkins we see more and more that choose other solutions. A very prominent one is Bamboo, the CI/CD solution from Atlassian.

The Dynatrace AppMon Plugin for Bamboo has been available for a while now – thanks to Jan Swaelens from Sofico who built the integration based on the open Dynatrace AppMon Test Automation REST API. You can watch this Shift-Left YouTube Tutorial to see how Jan is using Bamboo with Selenium and Dynatrace AppMon in their environment.

Lester Hilton, one of our power users out of South Africa, sent me an email explaining how they use Bamboo to kick of SoapUI and JMeter scripts and how they now implemented Shift-Left with Dynatrace AppMon leveraging the plugin from Jan.

Integration Scenario with SoapUI

Lester uses SoapUI to execute Web API tests against their application. In order to integrate SoapUI tests with Dynatrace AppMon Test Automation a couple of steps are necessary. Here is a high-level overview:

Workflow of integrating Dynatrace AppMon Test Automation with SoapUI executed via Bamboo. Key step is to pass the Test Run ID from Dynatrace to the SoapUI test.

The Dynatrace Bamboo plugin will automatically register a new Test Run ID through the Register Test Session task. Because we are executing Web API Tests we chose the category Web API. The Test Run ID is stored in a variable which can be accessed via ${bamboo.dynatrace.testsession.id}. Now simply pass that variable to the script that launches SoapUI via the -P command line option. Following is a screenshot that shows these two tasks configured in a Bamboo Job:

The Dynatrace Test Run ID is passed to the SoapUI via the command line. It can then be referenced in the SoapUI script.

As the screenshot is a bit cut off – here is the full command line script to launch SoapUI:

SoapUI Script Changes

In SoapUI, Lester created a Groovy script that gets called from SoapUI while executing the tests. The script will add the X-Dynatrace HTTP Header that allows Dynatrace to identify the incoming Web Request from SoapUI. The value of that X-Dynatrace HTTP Header has to follow our Tagged Web Request specification. We require at least two values:

  • NA=Test Name -> this should be the logical name of your Test Case
  • TR=Test Run ID -> this is the Test Run ID that we received from the Bamboo Plugin

Here is the Groovy Script:

Make sure that your Load Test Script executes the Groovy script. If that is done you are all set. You can verify by doing a quick test run:

Validating that SoapUI executes all scripts correctly

That’s it. Save the Groovy project. You are all set to now execute your Bamboo Jobs. Here is what will happen

#1: The Dynatrace Plugin will register a new Test Run Id of Category Web API Test

#2: The Test Run ID is passed to the SoapUI shell that executes your script

#3: The Groovy script as part of the SoapUI project will add the X-Dynatrace header

#4: Dynatrace will capture PurePaths and understands that they come from your SoapUI script and are part of your current Bamboo Job execution

Integration Scenario with JMeter

The second integration scenario that Lester told me about was with JMeter. In his opinion JMeter gives them a bit more flexibility over SoapUI when it comes to data driven testing which is why he also integrated JMeter with Bamboo and Dynatrace AppMon.

Similar to the SoapUI scenario the first step is register a Test Run ID through the Dynatrace Bamboo Pluin. This Test Run ID now has to be passed to JMeter via the command line. JMeter provides the command line option -JNAME=VALUE to pass in a property. Here is a sample command line which executes foo.jmx, writes JMeter results into foo.jtl and passes the Dynatrace Test Run ID as property TRID:

The JMeter test result file can be fed into Bamboo’s JMeter Result Aggregator:

JMeter Script Changes

The only thing that has to be changed in your JMeter scripts is to parse the passed Test Run ID from the JMeter property. Then use this value when you create the X-dynatrace HTTP Header that you pass to every HTTP Request executed by JMeter. The NA value is going to be the Test Name and TR is going to be the Test Run ID. The easiest way to accomplish this is by using a BeanShell PreProcessor with the following code:

As you can see from the code above, the setProperty call is used to make the command line property TRID available as a regular property with the name “testrunId”.

If you want to see a live demo on how to integrate JMeter scripts with Dynatrace AppMon Test check out my YouTube Performance Clinic Tutorial on “JMeter + Dynatrace Pro Active Performance Engineering

The Final Results in Dynatrace AppMon and Bamboo

If you follow these steps you will achieve just what Lester achieved. When kicking off a build in Bamboo the Dynatrace AppMon Bamboo Plugin will first register a Test Run with Dynatrace. That Test Run ID is then passed to either the SoapUI or JMeter scripts. The scripts use this Test Run ID and pass it with the executed HTTP Requests as part of the X-Dynatrace header. This allows Dynatrace AppMon to analyze the incoming PurePaths and associate them with the correct test and Bamboo Build.

In case a code change results in a changed dynamic code behavior, e.g: a spike in SQL Executions, spike in internal API or Web Service Calls or spike in internal Exceptions Dynatrace will automatically mark these tests as degraded. With that you gain an extra level of quality metrics for your code which allows you to give faster and more accurate feedback to your developers about the impact their code changes have.

In Dynatrace AppMon you will see the quality status of all tests executed per Bamboo Build like shown here in the screenshot provided by Lester:

The Dynatrace AppMon Bamboo plugin from Jan additionally pulls this data back into Bamboo so that you can see this information in directly in Bamboo – next to all your other test results:

Thanks – And – Try it yourself

Thanks Lester for sharing your best practices when implementing real Shift-Left Quality with Bamboo, JMeter, SoapUI and Dynatrace AppMon.

If anyone wants to give this a try download the Bamboo Plugin, follow the practies in this blog, and watch some of my YouTube Tutorials where I cover other aspects of Shift-Left and Metrics-Driven Delivery Pipelines.

The post DevOps tool chain: Shift-Left with Bamboo, SoapUI, JMeter and Dynatrace appeared first on Dynatrace blog – monitoring redefined.

Read the original blog entry...

More Stories By Dynatrace Blog

Building a revolutionary approach to software performance monitoring takes an extraordinary team. With decades of combined experience and an impressive history of disruptive innovation, that’s exactly what we ruxit has.

Get to know ruxit, and get to know the future of data analytics.

Latest Stories
"With Digital Experience Monitoring what used to be a simple visit to a web page has exploded into app on phones, data from social media feeds, competitive benchmarking - these are all components that are only available because of some type of digital asset," explained Leo Vasiliou, Director of Web Performance Engineering at Catchpoint Systems, in this SYS-CON.tv interview at DevOps Summit at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
"Outscale was founded in 2010, is based in France, is a strategic partner to Dassault Systémes and has done quite a bit of work with divisions of Dassault," explained Jackie Funk, Digital Marketing exec at Outscale, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
"We were founded in 2003 and the way we were founded was about good backup and good disaster recovery for our clients, and for the last 20 years we've been pretty consistent with that," noted Marc Malafronte, Territory Manager at StorageCraft, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
"DivvyCloud as a company set out to help customers automate solutions to the most common cloud problems," noted Jeremy Snyder, VP of Business Development at DivvyCloud, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
Kubernetes is an open source system for automating deployment, scaling, and management of containerized applications. Kubernetes was originally built by Google, leveraging years of experience with managing container workloads, and is now a Cloud Native Compute Foundation (CNCF) project. Kubernetes has been widely adopted by the community, supported on all major public and private cloud providers, and is gaining rapid adoption in enterprises. However, Kubernetes may seem intimidating and complex ...
While the focus and objectives of IoT initiatives are many and diverse, they all share a few common attributes, and one of those is the network. Commonly, that network includes the Internet, over which there isn't any real control for performance and availability. Or is there? The current state of the art for Big Data analytics, as applied to network telemetry, offers new opportunities for improving and assuring operational integrity. In his session at @ThingsExpo, Jim Frey, Vice President of S...
DX World EXPO, LLC., a Lighthouse Point, Florida-based startup trade show producer and the creator of "DXWorldEXPO® - Digital Transformation Conference & Expo" has announced its executive management team. The team is headed by Levent Selamoglu, who has been named CEO. "Now is the time for a truly global DX event, to bring together the leading minds from the technology world in a conversation about Digital Transformation," he said in making the announcement.
"We focus on SAP workloads because they are among the most powerful but somewhat challenging workloads out there to take into public cloud," explained Swen Conrad, CEO of Ocean9, Inc., in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
"As we've gone out into the public cloud we've seen that over time we may have lost a few things - we've lost control, we've given up cost to a certain extent, and then security, flexibility," explained Steve Conner, VP of Sales at Cloudistics,in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
"We provide IoT solutions. We provide the most compatible solutions for many applications. Our solutions are industry agnostic and also protocol agnostic," explained Richard Han, Head of Sales and Marketing and Engineering at Systena America, in this SYS-CON.tv interview at @ThingsExpo, held June 6-8, 2017, at the Javits Center in New York City, NY.
"We are focused on SAP running in the clouds, to make this super easy because we believe in the tremendous value of those powerful worlds - SAP and the cloud," explained Frank Stienhans, CTO of Ocean9, Inc., in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
"DX encompasses the continuing technology revolution, and is addressing society's most important issues throughout the entire $78 trillion 21st-century global economy," said Roger Strukhoff, Conference Chair. "DX World Expo has organized these issues along 10 tracks with more than 150 of the world's top speakers coming to Istanbul to help change the world."
"Peak 10 is a hybrid infrastructure provider across the nation. We are in the thick of things when it comes to hybrid IT," explained , Chief Technology Officer at Peak 10, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
"I think DevOps is now a rambunctious teenager – it’s starting to get a mind of its own, wanting to get its own things but it still needs some adult supervision," explained Thomas Hooker, VP of marketing at CollabNet, in this SYS-CON.tv interview at DevOps Summit at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
"We are still a relatively small software house and we are focusing on certain industries like FinTech, med tech, energy and utilities. We help our customers with their digital transformation," noted Piotr Stawinski, Founder and CEO of EARP Integration, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.