Welcome!

Blog Feed Post

DevOps tool chain: Shift-Left with Bamboo, SoapUI, JMeter and Dynatrace

Dynatrace is a true player in the DevOps tool chain. We built Dynatrace to provide fast feedback on code quality all the way from Dev, via Continuous Integration/Continuous Delivery, all the way into production. The same Dynatrace solution for every pipeline stage to provide fast metrics-based feedback loops on code changes that you are pushing from left (Dev) to right (Ops).

Shift-Left therefore really means that we want to identify bad code quality early on. We also re-define code quality as not just meeting functional specification. We look into performance, scalability and architecture quality. Many of our users execute Unit-, Web/REST API-, Functional- or Load Tests automatically through Bamboo, Jenkins, TFS or other build servers. Dynatrace automatically levels-up these tests by identifying performance, architectural and scalability regressions across builds introduced by the recent code changes. We do this by automatically analyzing and baselining key application metrics captured while you execute your tests against your application as part of your automated build and pipeline execution.

If a regression is detected, e.g: a code or configuration change results in an increase of SQL Queries, and increase in Microservice calls or changed resource consumption behavior then Dynatrace tells the build server (Jenkins, Bamboo, TFS, …) to stop the build early (left) in the lifecycle. Giving the developers faster feedback on critical quality metrics:

Dynatrace enables real Shift-Left by detecting performance-, scalability and architectural regressions when code changes are pushed through your pipeline.

Shift-Left with Bamboo and Dynatrace

While we often talk about users that integrate Dynatrace into Jenkins we see more and more that choose other solutions. A very prominent one is Bamboo, the CI/CD solution from Atlassian.

The Dynatrace AppMon Plugin for Bamboo has been available for a while now – thanks to Jan Swaelens from Sofico who built the integration based on the open Dynatrace AppMon Test Automation REST API. You can watch this Shift-Left YouTube Tutorial to see how Jan is using Bamboo with Selenium and Dynatrace AppMon in their environment.

Lester Hilton, one of our power users out of South Africa, sent me an email explaining how they use Bamboo to kick of SoapUI and JMeter scripts and how they now implemented Shift-Left with Dynatrace AppMon leveraging the plugin from Jan.

Integration Scenario with SoapUI

Lester uses SoapUI to execute Web API tests against their application. In order to integrate SoapUI tests with Dynatrace AppMon Test Automation a couple of steps are necessary. Here is a high-level overview:

Workflow of integrating Dynatrace AppMon Test Automation with SoapUI executed via Bamboo. Key step is to pass the Test Run ID from Dynatrace to the SoapUI test.

The Dynatrace Bamboo plugin will automatically register a new Test Run ID through the Register Test Session task. Because we are executing Web API Tests we chose the category Web API. The Test Run ID is stored in a variable which can be accessed via ${bamboo.dynatrace.testsession.id}. Now simply pass that variable to the script that launches SoapUI via the -P command line option. Following is a screenshot that shows these two tasks configured in a Bamboo Job:

The Dynatrace Test Run ID is passed to the SoapUI via the command line. It can then be referenced in the SoapUI script.

As the screenshot is a bit cut off – here is the full command line script to launch SoapUI:

SoapUI Script Changes

In SoapUI, Lester created a Groovy script that gets called from SoapUI while executing the tests. The script will add the X-Dynatrace HTTP Header that allows Dynatrace to identify the incoming Web Request from SoapUI. The value of that X-Dynatrace HTTP Header has to follow our Tagged Web Request specification. We require at least two values:

  • NA=Test Name -> this should be the logical name of your Test Case
  • TR=Test Run ID -> this is the Test Run ID that we received from the Bamboo Plugin

Here is the Groovy Script:

Make sure that your Load Test Script executes the Groovy script. If that is done you are all set. You can verify by doing a quick test run:

Validating that SoapUI executes all scripts correctly

That’s it. Save the Groovy project. You are all set to now execute your Bamboo Jobs. Here is what will happen

#1: The Dynatrace Plugin will register a new Test Run Id of Category Web API Test

#2: The Test Run ID is passed to the SoapUI shell that executes your script

#3: The Groovy script as part of the SoapUI project will add the X-Dynatrace header

#4: Dynatrace will capture PurePaths and understands that they come from your SoapUI script and are part of your current Bamboo Job execution

Integration Scenario with JMeter

The second integration scenario that Lester told me about was with JMeter. In his opinion JMeter gives them a bit more flexibility over SoapUI when it comes to data driven testing which is why he also integrated JMeter with Bamboo and Dynatrace AppMon.

Similar to the SoapUI scenario the first step is register a Test Run ID through the Dynatrace Bamboo Pluin. This Test Run ID now has to be passed to JMeter via the command line. JMeter provides the command line option -JNAME=VALUE to pass in a property. Here is a sample command line which executes foo.jmx, writes JMeter results into foo.jtl and passes the Dynatrace Test Run ID as property TRID:

The JMeter test result file can be fed into Bamboo’s JMeter Result Aggregator:

JMeter Script Changes

The only thing that has to be changed in your JMeter scripts is to parse the passed Test Run ID from the JMeter property. Then use this value when you create the X-dynatrace HTTP Header that you pass to every HTTP Request executed by JMeter. The NA value is going to be the Test Name and TR is going to be the Test Run ID. The easiest way to accomplish this is by using a BeanShell PreProcessor with the following code:

As you can see from the code above, the setProperty call is used to make the command line property TRID available as a regular property with the name “testrunId”.

If you want to see a live demo on how to integrate JMeter scripts with Dynatrace AppMon Test check out my YouTube Performance Clinic Tutorial on “JMeter + Dynatrace Pro Active Performance Engineering

The Final Results in Dynatrace AppMon and Bamboo

If you follow these steps you will achieve just what Lester achieved. When kicking off a build in Bamboo the Dynatrace AppMon Bamboo Plugin will first register a Test Run with Dynatrace. That Test Run ID is then passed to either the SoapUI or JMeter scripts. The scripts use this Test Run ID and pass it with the executed HTTP Requests as part of the X-Dynatrace header. This allows Dynatrace AppMon to analyze the incoming PurePaths and associate them with the correct test and Bamboo Build.

In case a code change results in a changed dynamic code behavior, e.g: a spike in SQL Executions, spike in internal API or Web Service Calls or spike in internal Exceptions Dynatrace will automatically mark these tests as degraded. With that you gain an extra level of quality metrics for your code which allows you to give faster and more accurate feedback to your developers about the impact their code changes have.

In Dynatrace AppMon you will see the quality status of all tests executed per Bamboo Build like shown here in the screenshot provided by Lester:

The Dynatrace AppMon Bamboo plugin from Jan additionally pulls this data back into Bamboo so that you can see this information in directly in Bamboo – next to all your other test results:

Thanks – And – Try it yourself

Thanks Lester for sharing your best practices when implementing real Shift-Left Quality with Bamboo, JMeter, SoapUI and Dynatrace AppMon.

If anyone wants to give this a try download the Bamboo Plugin, follow the practies in this blog, and watch some of my YouTube Tutorials where I cover other aspects of Shift-Left and Metrics-Driven Delivery Pipelines.

The post DevOps tool chain: Shift-Left with Bamboo, SoapUI, JMeter and Dynatrace appeared first on Dynatrace blog – monitoring redefined.

Read the original blog entry...

More Stories By Dynatrace Blog

Building a revolutionary approach to software performance monitoring takes an extraordinary team. With decades of combined experience and an impressive history of disruptive innovation, that’s exactly what we ruxit has.

Get to know ruxit, and get to know the future of data analytics.

Latest Stories
Five years ago development was seen as a dead-end career, now it’s anything but – with an explosion in mobile and IoT initiatives increasing the demand for skilled engineers. But apart from having a ready supply of great coders, what constitutes true ‘DevOps Royalty’? It’ll be the ability to craft resilient architectures, supportability, security everywhere across the software lifecycle. In his keynote at @DevOpsSummit at 20th Cloud Expo, Jeffrey Scheaffer, GM and SVP, Continuous Delivery Busine...
With major technology companies and startups seriously embracing IoT strategies, now is the perfect time to attend @ThingsExpo 2016 in New York. Learn what is going on, contribute to the discussions, and ensure that your enterprise is as "IoT-Ready" as it can be! Internet of @ThingsExpo, taking place June 6-8, 2017, at the Javits Center in New York City, New York, is co-located with 20th Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry p...
SYS-CON Events announced today that Progress, a global leader in application development, has been named “Bronze Sponsor” of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Enterprises today are rapidly adopting the cloud, while continuing to retain business-critical/sensitive data inside the firewall. This is creating two separate data silos – one inside the firewall and the other outside the firewall. Cloud ISVs oft...
SYS-CON Events announced today that Juniper Networks (NYSE: JNPR), an industry leader in automated, scalable and secure networks, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Juniper Networks challenges the status quo with products, solutions and services that transform the economics of networking. The company co-innovates with customers and partners to deliver automated, scalable and secure network...
In recent years, containers have taken the world by storm. Companies of all sizes and industries have realized the massive benefits of containers, such as unprecedented mobility, higher hardware utilization, and increased flexibility and agility; however, many containers today are non-persistent. Containers without persistence miss out on many benefits, and in many cases simply pass the responsibility of persistence onto other infrastructure, adding additional complexity.
SYS-CON Events announced today that CA Technologies has been named "Platinum Sponsor" of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, New York, and 21st International Cloud Expo, which will take place in November in Silicon Valley, California.
New competitors, disruptive technologies, and growing expectations are pushing every business to both adopt and deliver new digital services. This ‘Digital Transformation’ demands rapid delivery and continuous iteration of new competitive services via multiple channels, which in turn demands new service delivery techniques – including DevOps. In this power panel at @DevOpsSummit 20th Cloud Expo, moderated by DevOps Conference Co-Chair Andi Mann, panelists will examine how DevOps helps to meet th...
DevOps is being widely accepted (if not fully adopted) as essential in enterprise IT. But as Enterprise DevOps gains maturity, expands scope, and increases velocity, the need for data-driven decisions across teams becomes more acute. DevOps teams in any modern business must wrangle the ‘digital exhaust’ from the delivery toolchain, "pervasive" and "cognitive" computing, APIs and services, mobile devices and applications, the Internet of Things, and now even blockchain.
Did you know that you can develop for mainframes in Java? Or that the testing and deployment can be automated across mobile to mainframe? In his session at @DevOpsSummit at 20th Cloud Expo, Vaughn Marshall, Sr. Principal Product Owner at CA Technologies, will discuss and demo how increasingly teams are developing with agile methodologies using modern development environments and automating testing and deployments, mobile to mainframe.
SYS-CON Events announced today that Hitachi, the leading provider the Internet of Things and Digital Transformation, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Hitachi Data Systems, a wholly owned subsidiary of Hitachi, Ltd., offers an integrated portfolio of services and solutions that enable digital transformation through enhanced data management, governance, mobility and analytics. We help globa...
@DevOpsSummit has been named the ‘Top DevOps Influencer' by iTrend. iTred processes millions of conversations, tweets, interactions, news articles, press releases, blog posts - and extract meaning form them and analyzes mobile and desktop software platforms used to communicate, various metadata (such as geo location), and automation tools. In overall placement, @DevOpsSummit ranked as the number one ‘DevOps Influencer' followed by @CloudExpo at third, and @MicroservicesE at 24th.
@GonzalezCarmen has been ranked the Number One Influencer and @ThingsExpo has been named the Number One Brand in the “M2M 2016: Top 100 Influencers and Brands” by Analytic. Onalytica analyzed tweets over the last 6 months mentioning the keywords M2M OR “Machine to Machine.” They then identified the top 100 most influential brands and individuals leading the discussion on Twitter.
SYS-CON Events announced today that Hitachi, the leading provider the Internet of Things and Digital Transformation, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Hitachi Data Systems, a wholly owned subsidiary of Hitachi, Ltd., offers an integrated portfolio of services and solutions that enable digital transformation through enhanced data management, governance, mobility and analytics. We help globa...
Blockchain is a shared, secure record of exchange that establishes trust, accountability and transparency across supply chain networks. Supported by the Linux Foundation's open source, open-standards based Hyperledger Project, Blockchain has the potential to improve regulatory compliance, reduce cost and time for product recall as well as advance trade. Are you curious about Blockchain and how it can provide you with new opportunities for innovation and growth? In her session at 20th Cloud Exp...
In his keynote at 19th Cloud Expo, Sheng Liang, co-founder and CEO of Rancher Labs, discussed the technological advances and new business opportunities created by the rapid adoption of containers. With the success of Amazon Web Services (AWS) and various open source technologies used to build private clouds, cloud computing has become an essential component of IT strategy. However, users continue to face challenges in implementing clouds, as older technologies evolve and newer ones like Docker c...