Blog Feed Post

DevOps and Security: From the Trenches to Command Centers

Shared values, if you’re doing it right, doom and gloom if you’re not.

Previously, system administrators frequently were responsible for security: applying patches, building firewalls, enforcing security practices. Work often involved a lot of exciting manual labor, unique tooling, and esoteric techniques.

Rare, esoteric techniques

Here Come The DevOps

As time passed, both security operations and maintenance operations have evolved. The DevOps movement emerged as an attempt to build the bridge between people who write code, people who maintain the infrastructure to run it, and people who make the business decisions. These changes have put emphasis on the new set of techniques and values. These techniques and values can either be beneficial or problematic for the security posture.

New exciting techniques

Through the years, in one way or another, I’ve been responsible for building, running, and enhancing product infrastructures for companies I worked in, or for their clients. Somehow, many things emerging into the landscape from DevOps seem pretty familiar if looked at from the right angle. Working for a cryptographic engineering company Cossack Labs, where demands towards the development security are slightly higher than the regular “hey, we’ve got payment processing data and potentially millions in regulatory fines”, I’ve had an opportunity to rethink many of ideas behind building secure processes in a modern way. Somehow, many DevOps concepts perfectly fit into the picture.

This post is an inquiry into mental models that make DevOps beneficial for security, instead of being detrimental.

Looking at DevOps through its values, DevOps is all about business performance through technical innovation:

  1. Improving value delivery to the customer: delivering continuously, reliably, predictably, via automated scenarios.
  2. Improving change management and value creation: using continuous integration and testing everything, programmatic infrastructure, testable user experience.
  3. Improving failure recovery: preventing incidents, managing incidents, understanding root causes, repeat.

In the end, business is able to charge clients faster and more frequently: just because more value gets delivered faster, with increasing consistency. All of this is achieved by using certain processes and tools. When used blindly, it brings a lot of damage.

Driving Blindly

Security is an invisible showstopper for all the aforementioned values if done wrong:

  1. Real-world impact: Customer data leaks and service disruptions ruin any value you aim to deliver faster. Lawsuits and regulator fines will not just affect the developers, in many cases they will rule the affected companies out of business.
  2. More causes for damaging impact: Quicker changes, done while impairing security and reliability, will introduce more leaks, more service disruptions, and indirect damage as well: your systems will be used to damage others.
  3. Even fixing things quickly is a threat: Failure recovery, which puts “time to get back in production” first, without security considerations, is a frequent path to introduce security inconsistencies into a system.

At the same time, security-conscious DevOps could significantly increase the security posture:

  1. Improve value reliability: Optimize for reliability, repeatability, and predictability of value delivery, and the speed will organically follow. Use programmatic and consistent security controls.
  2. Improve value verification: Optimize change management for the deterministic state of any code where state includes code’s security status verified automatically.
  3. Prepare for the bad times: Aggregate metrics for incident response and backup controls for threat mitigation.

Whether your company has a security policy and security-aware developers and operators, or you’re just looking forward to getting started, DevOps tooling can bring a lot of process/infrastructure safety from day zero, then help evolve it into consistent security posture.

https://i1.wp.com/blog.xebialabs.com/wp-content/uploads/2017/05/Screen-S... 682w" sizes="(max-width: 251px) 100vw, 251px" data-recalc-dims="1" />

Case Study


How One Major Bank Reduced Software Time-to-Market From Weeks to Days

Learn how release pipeline orchestration solutions help even the most audited of enterprises efficiently manage and optimize their software release pipelines.

Transforming DevOps Practices for Security

Even though tooling is just a part of DevOps phenomena, it’s a rather important part of it.

From viewpoint of traditional secure development practices, many DevOps approaches are not just “compatible” — they’re perfectly fitting. Let’s take a look.

Even though a bit chaotic, DevOps methodologies can be put to a good use!

Test everything!

Functional testing as Dynamic Application Security testing: First of all, you want to know that security controls within your application actually work.

Testing security functions in functional tests and running specific non-functional tests against known vulnerabilities allows ensuring that security controls do work as expected on every iteration of continuous integration.

Things to try:

  • BDD-security suite as the testing framework for functional security testing, infrastructure security testing, and application security testing. Integrates with Jenkins!
  • Gauntlt, a number of Ruby hooks to security tools to integrate into your CI infrastructure.
  • OWASP Zap and OWASP Zapper (Jenkins plugin): automate attack proxy to test some of the attacks.
  • Mittn, F-Secure’s security testing tooling for CI.

Source code analysis as Static Application Security testing: it’s worth investing effort into making sure that detectable vulnerabilities get detected automatically.

Reducing vulnerability risk by scanning sources with static code analysis tools / static application security tests for possibly vulnerable code during CI iterations.

Things to try: OWASP list of some SAST tools, a list of classic tools, many modern ones emerging daily.

Blocking or parallel? Obviously, when developing for security, security tests should be blocking, period.

Collection of audit and operational data

Metrics DevOps’ use to monitor and tune development and production environments are an important context for audit logs relevant to security. Being gathered together, in a secure way, can make intrusion detection and incident response stellar.

Automatic configurations 101

Over the course of years, I’ve had several occasions to have automatic configurations in building firewalls and access control, and while some of the stuff was taxing to operations, it’s the simplest way to have safe defaults, consistent firewalls, and some confidence in your access control system.

Moreover, positive security for something like application firewall means enumerating all possible scenarios of moving around HTTP endpoint graphs with parameters and stuff… easy to make manual mistakes, easy for insiders to hide a channel to leak data out.

Generating firewall rules programmatically, if done right, is a great nerve savior.

Orchestrated automatic configurations

Orchestrating multiple automatic configuration tools is where the magic starts: imagine you know the role and address of every node in the system. Now imagine you can whitelist access based on role model, and basically have positively modeled iptables configuration instantly drawing out a significant portion of threats.

Know thy limits

Investing in automation shouldn’t be religious. Preventing attacks and managing ongoing incidents is still a decision-making process of a human being under stress, pretty much like the OODA loop in the military methodology.

Things that can be automated, should be. The rest should be orchestrated in a way that humans, who should actually make important decisions, will not only have all the relevant metrics and audit logs but means to act quickly, efficiently and across vast infrastructures of today as well.

Getting Serious and Boring

Well-orchestrated DevOps environment

After your infrastructural tools have been adjusted, you’re already far ahead of many development and production environments. However, whatever amount of labor you’ve put into it, it’s worthless if there still is a simple way to mount an attack against your infrastructure. Security is an asymmetric game: you have to defend against every threat, while attackers have to find just one weakness to succeed.

And so, security policy and procedures are still necessary. Having built parts of the necessary infrastructure, next steps will be simpler: understand risk model and security objectives, audit current state, define gaps in processes, techniques, then come up with a development plan and then, finally, go back to engineering, but with an all-encompassing construction plan.


If DevOps practitioner sees that DevOps is about enhancing business value and business value includes business continuity and security — suddenly, tooling falls into place and helps adjacent operational needs.

If DevOps practitioner is just excited by tools and fancy lingo — it’s a wreck in the making. And if you add security into the equation, the wreck might be more than just inefficient development and maintenance, but all the typical scary tales coming true.

The post DevOps and Security: From the Trenches to Command Centers appeared first on XebiaLabs.

Read the original blog entry...

More Stories By XebiaLabs Blog

XebiaLabs is the technology leader for automation software for DevOps and Continuous Delivery. It focuses on helping companies accelerate the delivery of new software in the most efficient manner. Its products are simple to use, quick to implement, and provide robust enterprise technology.

Latest Stories
SYS-CON Events announced today that Fusion, a leading provider of cloud services, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Fusion, a leading provider of integrated cloud solutions to small, medium and large businesses, is the industry’s single source for the cloud. Fusion’s advanced, proprietary cloud service platform enables the integration of leading edge solutions in the cloud, including cloud...
Five years ago development was seen as a dead-end career, now it’s anything but – with an explosion in mobile and IoT initiatives increasing the demand for skilled engineers. But apart from having a ready supply of great coders, what constitutes true ‘DevOps Royalty’? It’ll be the ability to craft resilient architectures, supportability, security everywhere across the software lifecycle. In his keynote at @DevOpsSummit at 20th Cloud Expo, Jeffrey Scheaffer, GM and SVP, Continuous Delivery Busine...
SYS-CON Events announced today that Cloud Academy will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Cloud Academy is the industry’s most innovative, vendor-neutral cloud technology training platform. Cloud Academy provides continuous learning solutions for individuals and enterprise teams for Amazon Web Services, Microsoft Azure, Google Cloud Platform, and the most popular cloud computing technologies. Ge...
While some vendors scramble to create and sell you a fancy solution for monitoring your spanking new Amazon Lambdas, hear how you can do it on the cheap using just built-in Java APIs yourself. By exploiting a little-known fact that Lambdas aren’t exactly single threaded, you can effectively identify hot spots in your serverless code. In his session at 20th Cloud Expo, David Martin, Principal Product Owner at CA Technologies, will give a live demonstration and code walkthrough, showing how to ov...
SYS-CON Events announced today that Interoute has been named “Bronze Sponsor” of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Interoute is the owner operator of Europe's largest network and a global cloud services platform, which encompasses over 70,000 km of lit fiber, 15 data centers, 17 virtual data centers and 33 colocation centers, with connections to 195 additional partner data centers. Our full-service Unifie...
Cloud promises the agility required by today’s digital businesses. As organizations adopt cloud based infrastructures and services, their IT resources become increasingly dynamic and hybrid in nature. Managing these require modern IT operations and tools. In his session at 20th Cloud Expo, Raj Sundaram, Senior Principal Product Manager at CA Technologies, will discuss how to modernize your IT operations in order to proactively manage your hybrid cloud and IT environments. He will be sharing bes...
SYS-CON Events announced today that Ocean9will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Ocean9 provides cloud services for Backup, Disaster Recovery (DRaaS) and instant Innovation, and redefines enterprise infrastructure with its cloud native subscription offerings for mission critical SAP workloads.
SYS-CON Events announced today that Systena America will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Systena Group has been in business for various software development and verification in Japan, US, ASEAN, and China by utilizing the knowledge we gained from all types of device development for various industries including smartphones (Android/iOS), wireless communication, security technology and IoT serv...
Internet of @ThingsExpo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with the 21st International Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. @ThingsExpo Silicon Valley Call for Papers is now open.
This talk centers around how to automate best practices in a multi-/hybrid-cloud world based on our work with customers like GE, Discovery Communications and Fannie Mae. Today’s enterprises are reaping the benefits of cloud computing, but also discovering many risks and challenges. In the age of DevOps and the decentralization of IT, it’s easy to over-provision resources, forget that instances are running, or unintentionally expose vulnerabilities.
SYS-CON Events announced today that Twistlock, the leading provider of cloud container security solutions, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Twistlock is the industry's first enterprise security suite for container security. Twistlock's technology addresses risks on the host and within the application of the container, enabling enterprises to consistently enforce security policies, monitor...
SYS-CON Events announced today that Enzu will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY, and the 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Enzu’s mission is to be the leading provider of enterprise cloud solutions worldwide. Enzu enables online businesses to use its IT infrastructure to their competitive ad...
Everywhere we turn in our industry we can find strong opinions about the direction, type and nature of cloud’s impact on computing and business. Another word that is used in every context in our industry is “hybrid.” In his session at 20th Cloud Expo, Alvaro Gonzalez, Director of Technical, Partner and Field Marketing at Peak 10, will use a combination of a few conceptual props and some research recently commissioned by Peak 10 to offer a real-world consideration of how the various categories of...
DevOps is often described as a combination of technology and culture. Without both, DevOps isn't complete. However, applying the culture to outdated technology is a recipe for disaster; as response times grow and connections between teams are delayed by technology, the culture will die. A Nutanix Enterprise Cloud has many benefits that provide the needed base for a true DevOps paradigm. In his Day 3 Keynote at 20th Cloud Expo, Chris Brown, a Solutions Marketing Manager at Nutanix, will explore t...
SYS-CON Events announced today that SoftLayer, an IBM Company, has been named “Gold Sponsor” of SYS-CON's 18th Cloud Expo, which will take place on June 7-9, 2016, at the Javits Center in New York, New York. SoftLayer, an IBM Company, provides cloud infrastructure as a service from a growing number of data centers and network points of presence around the world. SoftLayer’s customers range from Web startups to global enterprises.