Blog Feed Post

DevOps and Security: From the Trenches to Command Centers

Shared values, if you’re doing it right, doom and gloom if you’re not.

Previously, system administrators frequently were responsible for security: applying patches, building firewalls, enforcing security practices. Work often involved a lot of exciting manual labor, unique tooling, and esoteric techniques.

Rare, esoteric techniques

Here Come The DevOps

As time passed, both security operations and maintenance operations have evolved. The DevOps movement emerged as an attempt to build the bridge between people who write code, people who maintain the infrastructure to run it, and people who make the business decisions. These changes have put emphasis on the new set of techniques and values. These techniques and values can either be beneficial or problematic for the security posture.

New exciting techniques

Through the years, in one way or another, I’ve been responsible for building, running, and enhancing product infrastructures for companies I worked in, or for their clients. Somehow, many things emerging into the landscape from DevOps seem pretty familiar if looked at from the right angle. Working for a cryptographic engineering company Cossack Labs, where demands towards the development security are slightly higher than the regular “hey, we’ve got payment processing data and potentially millions in regulatory fines”, I’ve had an opportunity to rethink many of ideas behind building secure processes in a modern way. Somehow, many DevOps concepts perfectly fit into the picture.

This post is an inquiry into mental models that make DevOps beneficial for security, instead of being detrimental.

Looking at DevOps through its values, DevOps is all about business performance through technical innovation:

  1. Improving value delivery to the customer: delivering continuously, reliably, predictably, via automated scenarios.
  2. Improving change management and value creation: using continuous integration and testing everything, programmatic infrastructure, testable user experience.
  3. Improving failure recovery: preventing incidents, managing incidents, understanding root causes, repeat.

In the end, business is able to charge clients faster and more frequently: just because more value gets delivered faster, with increasing consistency. All of this is achieved by using certain processes and tools. When used blindly, it brings a lot of damage.

Driving Blindly

Security is an invisible showstopper for all the aforementioned values if done wrong:

  1. Real-world impact: Customer data leaks and service disruptions ruin any value you aim to deliver faster. Lawsuits and regulator fines will not just affect the developers, in many cases they will rule the affected companies out of business.
  2. More causes for damaging impact: Quicker changes, done while impairing security and reliability, will introduce more leaks, more service disruptions, and indirect damage as well: your systems will be used to damage others.
  3. Even fixing things quickly is a threat: Failure recovery, which puts “time to get back in production” first, without security considerations, is a frequent path to introduce security inconsistencies into a system.

At the same time, security-conscious DevOps could significantly increase the security posture:

  1. Improve value reliability: Optimize for reliability, repeatability, and predictability of value delivery, and the speed will organically follow. Use programmatic and consistent security controls.
  2. Improve value verification: Optimize change management for the deterministic state of any code where state includes code’s security status verified automatically.
  3. Prepare for the bad times: Aggregate metrics for incident response and backup controls for threat mitigation.

Whether your company has a security policy and security-aware developers and operators, or you’re just looking forward to getting started, DevOps tooling can bring a lot of process/infrastructure safety from day zero, then help evolve it into consistent security posture.

https://i1.wp.com/blog.xebialabs.com/wp-content/uploads/2017/05/Screen-S... 682w" sizes="(max-width: 251px) 100vw, 251px" data-recalc-dims="1" />

Case Study


How One Major Bank Reduced Software Time-to-Market From Weeks to Days

Learn how release pipeline orchestration solutions help even the most audited of enterprises efficiently manage and optimize their software release pipelines.

Transforming DevOps Practices for Security

Even though tooling is just a part of DevOps phenomena, it’s a rather important part of it.

From viewpoint of traditional secure development practices, many DevOps approaches are not just “compatible” — they’re perfectly fitting. Let’s take a look.

Even though a bit chaotic, DevOps methodologies can be put to a good use!

Test everything!

Functional testing as Dynamic Application Security testing: First of all, you want to know that security controls within your application actually work.

Testing security functions in functional tests and running specific non-functional tests against known vulnerabilities allows ensuring that security controls do work as expected on every iteration of continuous integration.

Things to try:

  • BDD-security suite as the testing framework for functional security testing, infrastructure security testing, and application security testing. Integrates with Jenkins!
  • Gauntlt, a number of Ruby hooks to security tools to integrate into your CI infrastructure.
  • OWASP Zap and OWASP Zapper (Jenkins plugin): automate attack proxy to test some of the attacks.
  • Mittn, F-Secure’s security testing tooling for CI.

Source code analysis as Static Application Security testing: it’s worth investing effort into making sure that detectable vulnerabilities get detected automatically.

Reducing vulnerability risk by scanning sources with static code analysis tools / static application security tests for possibly vulnerable code during CI iterations.

Things to try: OWASP list of some SAST tools, a list of classic tools, many modern ones emerging daily.

Blocking or parallel? Obviously, when developing for security, security tests should be blocking, period.

Collection of audit and operational data

Metrics DevOps’ use to monitor and tune development and production environments are an important context for audit logs relevant to security. Being gathered together, in a secure way, can make intrusion detection and incident response stellar.

Automatic configurations 101

Over the course of years, I’ve had several occasions to have automatic configurations in building firewalls and access control, and while some of the stuff was taxing to operations, it’s the simplest way to have safe defaults, consistent firewalls, and some confidence in your access control system.

Moreover, positive security for something like application firewall means enumerating all possible scenarios of moving around HTTP endpoint graphs with parameters and stuff… easy to make manual mistakes, easy for insiders to hide a channel to leak data out.

Generating firewall rules programmatically, if done right, is a great nerve savior.

Orchestrated automatic configurations

Orchestrating multiple automatic configuration tools is where the magic starts: imagine you know the role and address of every node in the system. Now imagine you can whitelist access based on role model, and basically have positively modeled iptables configuration instantly drawing out a significant portion of threats.

Know thy limits

Investing in automation shouldn’t be religious. Preventing attacks and managing ongoing incidents is still a decision-making process of a human being under stress, pretty much like the OODA loop in the military methodology.

Things that can be automated, should be. The rest should be orchestrated in a way that humans, who should actually make important decisions, will not only have all the relevant metrics and audit logs but means to act quickly, efficiently and across vast infrastructures of today as well.

Getting Serious and Boring

Well-orchestrated DevOps environment

After your infrastructural tools have been adjusted, you’re already far ahead of many development and production environments. However, whatever amount of labor you’ve put into it, it’s worthless if there still is a simple way to mount an attack against your infrastructure. Security is an asymmetric game: you have to defend against every threat, while attackers have to find just one weakness to succeed.

And so, security policy and procedures are still necessary. Having built parts of the necessary infrastructure, next steps will be simpler: understand risk model and security objectives, audit current state, define gaps in processes, techniques, then come up with a development plan and then, finally, go back to engineering, but with an all-encompassing construction plan.


If DevOps practitioner sees that DevOps is about enhancing business value and business value includes business continuity and security — suddenly, tooling falls into place and helps adjacent operational needs.

If DevOps practitioner is just excited by tools and fancy lingo — it’s a wreck in the making. And if you add security into the equation, the wreck might be more than just inefficient development and maintenance, but all the typical scary tales coming true.

The post DevOps and Security: From the Trenches to Command Centers appeared first on XebiaLabs.

Read the original blog entry...

More Stories By XebiaLabs Blog

XebiaLabs is the technology leader for automation software for DevOps and Continuous Delivery. It focuses on helping companies accelerate the delivery of new software in the most efficient manner. Its products are simple to use, quick to implement, and provide robust enterprise technology.

Latest Stories
In the world of DevOps there are ‘known good practices’ – aka ‘patterns’ – and ‘known bad practices’ – aka ‘anti-patterns.' Many of these patterns and anti-patterns have been developed from real world experience, especially by the early adopters of DevOps theory; but many are more feasible in theory than in practice, especially for more recent entrants to the DevOps scene. In this power panel at @DevOpsSummit at 18th Cloud Expo, moderated by DevOps Conference Chair Andi Mann, panelists discussed...
Cloud-based disaster recovery is critical to any production environment and is a high priority for many enterprise organizations today. Nearly 40% of organizations have had to execute their BCDR plan due to a service disruption in the past two years. Zerto on IBM Cloud offer VMware and Microsoft customers simple, automated recovery of on-premise VMware and Microsoft workloads to IBM Cloud data centers.
Elon Musk is among the notable industry figures who worries about the power of AI to destroy rather than help society. Mark Zuckerberg, on the other hand, embraces all that is going on. AI is most powerful when deployed across the vast networks being built for Internets of Things in the manufacturing, transportation and logistics, retail, healthcare, government and other sectors. Is AI transforming IoT for the good or the bad? Do we need to worry about its potential destructive power? Or will we...
Many organizations adopt DevOps to reduce cycle times and deliver software faster; some take on DevOps to drive higher quality and better end-user experience; others look to DevOps for a clearer line-of-sight to customers to drive better business impacts. In truth, these three foundations go together. In this power panel at @DevOpsSummit 21st Cloud Expo, moderated by DevOps Conference Co-Chair Andi Mann, industry experts will discuss how leading organizations build application success from all...
SYS-CON Events announced today that SIGMA Corporation will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. uLaser flow inspection device from the Japanese top share to Global Standard! Then, make the best use of data to flip to next page. For more information, visit http://www.sigma-k.co.jp/en/.
SYS-CON Events announced today that Daiya Industry will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Daiya Industry specializes in orthotic support systems and assistive devices with pneumatic artificial muscles in order to contribute to an extended healthy life expectancy. For more information, please visit https://www.daiyak...
The last two years has seen discussions about cloud computing evolve from the public / private / hybrid split to the reality that most enterprises will be creating a complex, multi-cloud strategy. Companies are wary of committing all of their resources to a single cloud, and instead are choosing to spread the risk – and the benefits – of cloud computing across multiple providers and internal infrastructures, as they follow their business needs. Will this approach be successful? How large is the ...
Your clients expect transactions to never fail, cloud access to be fast and always on, and their data to be protected - no exceptions. Hear about how Secure Service Container (SSC), an IBM-exclusive open technology, enables secure building and hosting of next-generation applications, both cloud and on-premises. SSC protects the full stack from external and insider threats, allows automatic encryption of data in-flight and at-rest, and is tamper-resistant during installation and runtime – with no...
With major technology companies and startups seriously embracing Cloud strategies, now is the perfect time to attend 21st Cloud Expo October 31 - November 2, 2017, at the Santa Clara Convention Center, CA, and June 12-14, 2018, at the Javits Center in New York City, NY, and learn what is going on, contribute to the discussions, and ensure that your enterprise is on the right path to Digital Transformation.
SYS-CON Events announced today that B2Cloud will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. B2Cloud specializes in IoT devices for preventive and predictive maintenance in any kind of equipment retrieving data like Energy consumption, working time, temperature, humidity, pressure, etc.
SYS-CON Events announced today that NetApp has been named “Bronze Sponsor” of SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. NetApp is the data authority for hybrid cloud. NetApp provides a full range of hybrid cloud data services that simplify management of applications and data across cloud and on-premises environments to accelerate digital transformation. Together with their partners, NetApp em...
SYS-CON Events announced today that Interface Corporation will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Interface Corporation is a company developing, manufacturing and marketing high quality and wide variety of industrial computers and interface modules such as PCIs and PCI express. For more information, visit http://www.i...
SYS-CON Events announced today that Massive Networks, that helps your business operate seamlessly with fast, reliable, and secure internet and network solutions, has been named "Exhibitor" of SYS-CON's 21st International Cloud Expo ®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. As a premier telecommunications provider, Massive Networks is headquartered out of Louisville, Colorado. With years of experience under their belt, their team of...
SYS-CON Events announced today that Mobile Create USA will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Mobile Create USA Inc. is an MVNO-based business model that uses portable communication devices and cellular-based infrastructure in the development, sales, operation and mobile communications systems incorporating GPS capabi...
SYS-CON Events announced today that Nihon Micron will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Nihon Micron Co., Ltd. strives for technological innovation to establish high-density, high-precision processing technology for providing printed circuit board and metal mount RFID tags used for communication devices. For more inf...