Welcome!

Blog Feed Post

How Feedback Loops Make It Safe to Deploy to Production

Almost every organization is being pressured to respond more quickly to changes in the market. As development and operations professionals, supporting this business imperative requires that we get new code into production quickly or risk falling behind our competitors. But whether we work in Dev or Ops, pushing the deploy button can be intimidating. After all, who wants to be the person responsible for bringing down production?

Feedback loops promote a better working relationship between Dev and Ops by reinforcing shared goals, responsibilities and empathy.https://i0.wp.com/blog.xebialabs.com/wp-content/uploads/2017/07/Two-men-... 300w, https://i0.wp.com/blog.xebialabs.com/wp-content/uploads/2017/07/Two-men-... 768w" sizes="(max-width: 542px) 100vw, 542px" data-recalc-dims="1" />

Fear of deploying code by both Dev and Ops is not unusual. In fact, Mike Bland described deploying code at Google in 2005 like this:

Fear became the mind-killer. Fear stopped new team members from changing things because they didn’t understand the system. But fear also stopped experienced people from changing things because they understood it all too well.

By providing faster and more frequent feedback to engineers performing deployments, and by reducing the batch size of their work, we can create a safe system of work, integrating the deployment of changes into production as a part of our daily work and elevating everyone’s productivity. Doing this also promotes a better working relationship between Dev and Ops by reinforcing shared goals, responsibilities and empathy.

Let’s look at what we can do to create the feedback mechanisms needed to take the fear out of deploying to production and keep our companies at the top of the market.

Use Telemetry to Make Deployments Safer

Feedback loops depend on understanding how our systems behave as a whole. For that, we need telemetry—the collecting of measurements and other data within our applications and environments (both in production and pre-production) and in our deployment pipeline.

Telemetry provides the intelligence we need to make fact-based decisions about how to improve the health of the value stream at every stage of the service life cycle, ensuring that our services are “production ready,” even at the earliest stages of the project. The information that telemetry yields empowers us to integrate what we learn from each release and production problem into our future work, resulting in better safety and productivity for everyone.

Using telemetry, we can actively monitor the metrics associated with features during deployment. This enables whoever is doing the deployment, whether Dev or Ops, to catch errors in our deployment pipeline before our features reach production, to quickly determine whether features are operating as designed once they get there, and to quickly restore service in the event of errors that we did not detect.

DevOps by the Numbers On Demand Webinarhttps://i0.wp.com/blog.xebialabs.com/wp-content/uploads/2017/04/DevOps-b... 150w, https://i0.wp.com/blog.xebialabs.com/wp-content/uploads/2017/04/DevOps-b... 300w, https://i0.wp.com/blog.xebialabs.com/wp-content/uploads/2017/04/DevOps-b... 120w" sizes="(max-width: 232px) 100vw, 232px" data-recalc-dims="1" />

ON-DEMAND WEBINAR

DevOps by the Numbers

How to Approach the Measurement and Metrics of Your Continuous Delivery Transformation

Watch this on-demand webinar to learn ways to better measure the processes and output of your DevOps and Continuous Delivery transformation.

Have Dev Share Pager Rotation Duties with Ops    

Our production deployment and release went flawlessly, but we still experienced some unexpected problems. Left unfixed, they can cause recurring problems and suffering for Ops engineers downstream. But even if issues are assigned to a feature team, they may be considered low priority, which can cause chaos and disruption in Operations and degrade performance for the entire value stream.

To prevent this upheaval, we can put developers, development managers, and architects on pager rotation so that everyone in the value stream shares the downstream responsibilities of handling operational incidents. In this way, Operations no longer struggles alone with code-related production issues. Instead, everyone works together to find the proper balance between fixing production defects and developing new functionality.

Have Developers Follow Work Downstream

Observing customers using an application in their natural environment often uncovers startling ways that they struggle with the application. For developers who choose to participate in the observation, it can be a difficult thing to watch, but it almost always results in significant learning and a fervent desire to improve the situation for the customer.

We can use this same technique to observe how our work affects internal customers. Developers follow their work downstream so they can see how downstream work centers must interact with their product to get it running into production. UX observation enables the creation of quality at the source and helps developers make more informed decisions in their daily work. It also results in far greater empathy for fellow team members in the value stream, which is important for creating a strong DevOps work culture.¶

Have Developers Initially Self-Manage Their Production Service

Even when developers write and run their code in production-like environments in their daily work, Operations may still experience disastrous production releases. That’s because it is the first time we see how our code behaves under true production conditions. Operational learnings often occur too late in the software life cycle, which can be an outcome of not having enough Ops engineers to support all the product teams and the services already in production.

As a countermeasure, Development can self-manage their services in production before they go to a centralized Ops group to manage. By making developers responsible for deployment and production support, we are far more likely to see a smooth transition to Operations.

Defining launch requirements can help prevent the possibility of problematic, self-managed services going into production and creating organizational risk. Services would need to meet these requirements before interacting with real customers or being exposed to real production traffic. Launch guidance allows every product team to benefit from the cumulative and collective experience of the entire organization, especially Operations.

Ops engineers, acting as consultants, can help the feature team resolve issues or even re-engineer a service if necessary, so that it can be easily deployed and managed in production. For services already in production, creating a “handback” mechanism helps ensure that Operations can return production support responsibility back to Development when a service becomes sufficiently fragile.

To Learn More

Creating fast and continuous feedback from Operations to Development is part of the “Second Way,” which is the second of the three major principles underpinning DevOps. To learn more about the Second Way and the other DevOps principles, see The DevOps Handbook and The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win.

(Portions of this article were excerpted with permission from The DevOps Handbook.)

The post How Feedback Loops Make It Safe to Deploy to Production appeared first on XebiaLabs Blog.

Read the original blog entry...

More Stories By XebiaLabs Blog

XebiaLabs is the technology leader for automation software for DevOps and Continuous Delivery. It focuses on helping companies accelerate the delivery of new software in the most efficient manner. Its products are simple to use, quick to implement, and provide robust enterprise technology.

Latest Stories
"As we've gone out into the public cloud we've seen that over time we may have lost a few things - we've lost control, we've given up cost to a certain extent, and then security, flexibility," explained Steve Conner, VP of Sales at Cloudistics,in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
Internet of @ThingsExpo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 21st Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The Internet of Things (IoT) is the most profound change in personal and enterprise IT since the creation of the Worldwide Web more than 20 years ago. All major researchers estimate there will be tens of billions devic...
"The Striim platform is a full end-to-end streaming integration and analytics platform that is middleware that covers a lot of different use cases," explained Steve Wilkes, Founder and CTO at Striim, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
"We want to show that our solution is far less expensive with a much better total cost of ownership so we announced several key features. One is called geo-distributed erasure coding, another is support for KVM and we introduced a new capability called Multi-Part," explained Tim Desai, Senior Product Marketing Manager at Hitachi Data Systems, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
"We provide IoT solutions. We provide the most compatible solutions for many applications. Our solutions are industry agnostic and also protocol agnostic," explained Richard Han, Head of Sales and Marketing and Engineering at Systena America, in this SYS-CON.tv interview at @ThingsExpo, held June 6-8, 2017, at the Javits Center in New York City, NY.
"With Digital Experience Monitoring what used to be a simple visit to a web page has exploded into app on phones, data from social media feeds, competitive benchmarking - these are all components that are only available because of some type of digital asset," explained Leo Vasiliou, Director of Web Performance Engineering at Catchpoint Systems, in this SYS-CON.tv interview at DevOps Summit at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
21st International Cloud Expo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy. Me...
SYS-CON Events announced today that DXWorldExpo has been named “Global Sponsor” of SYS-CON's 21st International Cloud Expo, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Digital Transformation is the key issue driving the global enterprise IT business. Digital Transformation is most prominent among Global 2000 enterprises and government institutions.
SYS-CON Events announced today that Datera, that offers a radically new data management architecture, has been named "Exhibitor" of SYS-CON's 21st International Cloud Expo ®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Datera is transforming the traditional datacenter model through modern cloud simplicity. The technology industry is at another major inflection point. The rise of mobile, the Internet of Things, data storage and Big...
Kubernetes is an open source system for automating deployment, scaling, and management of containerized applications. Kubernetes was originally built by Google, leveraging years of experience with managing container workloads, and is now a Cloud Native Compute Foundation (CNCF) project. Kubernetes has been widely adopted by the community, supported on all major public and private cloud providers, and is gaining rapid adoption in enterprises. However, Kubernetes may seem intimidating and complex ...
SYS-CON Events announced today that Calligo, an innovative cloud service provider offering mid-sized companies the highest levels of data privacy and security, has been named "Bronze Sponsor" of SYS-CON's 21st International Cloud Expo ®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Calligo offers unparalleled application performance guarantees, commercial flexibility and a personalised support service from its globally located cloud plat...
"We focus on SAP workloads because they are among the most powerful but somewhat challenging workloads out there to take into public cloud," explained Swen Conrad, CEO of Ocean9, Inc., in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
"Outscale was founded in 2010, is based in France, is a strategic partner to Dassault Systémes and has done quite a bit of work with divisions of Dassault," explained Jackie Funk, Digital Marketing exec at Outscale, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
"We are still a relatively small software house and we are focusing on certain industries like FinTech, med tech, energy and utilities. We help our customers with their digital transformation," noted Piotr Stawinski, Founder and CEO of EARP Integration, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
"I think DevOps is now a rambunctious teenager – it’s starting to get a mind of its own, wanting to get its own things but it still needs some adult supervision," explained Thomas Hooker, VP of marketing at CollabNet, in this SYS-CON.tv interview at DevOps Summit at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.