Welcome!

Article

Applying DevOps to Deliver Quality at Speed

Applying DevOps to Deliver Quality at Speed

Fannie Mae operates in the already heavily regulated financial services/mortgage markets. Add to that it is a government sponsored entity, which means it is laden with many of the policies and regulations of federal government agencies. Plus, they have 6,000 employees and annual revenue of $22 billion. So, they could be a poster child for, "You can't implement DevOps there."

Barry Snyder, DevOps Manager, Developer Platforms at Fannie Mae, begs to differ, which is what he did in his 2016 All Day DevOps talk, How Fannie Mae Applied DevOps to Deliver Quality at Speed. He looks at:

  • Agile and DevOps transformation
  • Doing more with less with greater quality
  • Lessons learned

Fannie Mae started implementing DevOps in 2014, and Barry was in the middle of it from the start. He gave his talk at the end of 2016, and, as he said, they still aren't fully adopted (are you ever), but he gave a status and a comparison between where they were and where they are now.

2012

2016

10 projects using modified-Agile. Essentially waterfall with terminology/concepts from Agile

155 using Scrum/SAFe and growing

9 to 18 months to release to production

Projects releasing every month

9 governance gates/inspect 100% before release to production

1 working group, 1 CAB, inspect 10% after release to production

2-4 months to provision dev and test servers

Minutes to provision dev and test servers

300-400 builds per month; 900-1200 deployments per month

3-4K builds per month; 12-15K deployments per month

Testing took 50% the time it took to construct the solution

80% savings in testing time and executed within the sprint

No means to measure and evaluate the productivity from release to release

Productivity increasing by an average of 30%-40% with costs going down by 30%

barry1.png

According Barry, these results are being sustained - they are not just a blip. To get there, they had to get an appetite for risk. Overcoming that hurdle required them to realize when you do 100% risk mitigation, you are going to spend more time avoiding risk than it would take to mitigate something.

They are also executing Application Lifecycle Management (ALM), a step up from CI/CD. They look at all of the tools and processes along the lifecycyle, peer reviews, automated building, testing, etc. They are implementing automated processes with a variety of tools to achieve this.

As an example, at Fannie Mae, it previously took 5-6 weeks to request a library, now it is immediate because it is automated through Sonatype's Nexus.

barry2.png

As we hear from all practitioners, it is a journey, not a sprint (pun intended). Fannie Mae started at the end of 2014, and they are still in progress. They went from 9-18 months per deployment to a month on a consistent basis. They are close to always deploying during sprints, and are working towards having an idea, writing code, and releasing it into production.

Barry presented lessons learned:

Leverage what they had while shifting left:

    • Release Management Infrastructure (evolved into our CI/CD)
    • VM provisioning infrastructure (evolved into our RapidIT)
    • Test Automation
    • Code Quality
  • Recognize it is a culture shift in how work is performed:
    • Agile Adoption "You do this for the people, not do the people.""We started on the IT side, then the business side saw the value add."
    • Automate Oversight/Transparency - You can't do this manually. With 120 apps, there were 8,000 libraries not accounted for.
    • Portfolio Analysis (Risk Reduction, Quality, ROI, Cost Reduction)
    • Application Quality is a Supply Chain
    • Trust developers and get out of the way
  • Measure Results
    • Shifts in quality
    • Adoption rate
    • Accelerated delivery
    • Benchmark productivity and associated productivity cost

You can watch Barry's entire talk online here. If you missed any of the other 30-minute long presentations from All Day DevOps, they are easy to find and available free-of-charge here.  Finally, be sure to register you and the rest of your team for the 2017 All Day DevOps conference here.  This year's event will offer 96 practitioner-led sessions (no vendor pitches allowed).  It's all free and online on October 24th.

More Stories By Derek Weeks

In 2015, Derek Weeks led the largest and most comprehensive analysis of software supply chain practices to date across 160,000 development organizations. He is a huge advocate of applying proven supply chain management principles into DevOps practices to improve efficiencies, reduce costs, and sustain long-lasting competitive advantages.

As a 20+ year veteran of the software industry, he has advised leading businesses on IT performance improvement practices covering continuous delivery, business process management, systems and network operations, service management, capacity planning and storage management. As the VP and DevOps Advocate for Sonatype, he is passionate about changing the way people think about software supply chains and improving public safety through improved software integrity. Follow him here @weekstweets, find me here www.linkedin.com/in/derekeweeks, and read me here http://blog.sonatype.com/author/weeks/.

Latest Stories
Enterprise architects are increasingly adopting multi-cloud strategies as they seek to utilize existing data center assets, leverage the advantages of cloud computing and avoid cloud vendor lock-in. This requires a globally aware traffic management strategy that can monitor infrastructure health across data centers and end-user experience globally, while responding to control changes and system specification at the speed of today’s DevOps teams. In his session at 20th Cloud Expo, Josh Gray, Chie...
FinTechs use the cloud to operate at the speed and scale of digital financial activity, but are often hindered by the complexity of managing security and compliance in the cloud. In his session at 20th Cloud Expo, Sesh Murthy, co-founder and CTO of Cloud Raxak, showed how proactive and automated cloud security enables FinTechs to leverage the cloud to achieve their business goals. Through business-driven cloud security, FinTechs can speed time-to-market, diminish risk and costs, maintain continu...
Docker containers have brought great opportunities to shorten the deployment process through continuous integration and the delivery of applications and microservices. This applies equally to enterprise data centers as well as the cloud. In his session at 20th Cloud Expo, Jari Kolehmainen, founder and CTO of Kontena, discussed solutions and benefits of a deeply integrated deployment pipeline using technologies such as container management platforms, Docker containers, and the drone.io Cl tool. H...
Kubernetes is a new and revolutionary open-sourced system for managing containers across multiple hosts in a cluster. Ansible is a simple IT automation tool for just about any requirement for reproducible environments. In his session at @DevOpsSummit at 18th Cloud Expo, Patrick Galbraith, a principal engineer at HPE, discussed how to build a fully functional Kubernetes cluster on a number of virtual machines or bare-metal hosts. Also included will be a brief demonstration of running a Galera MyS...
Leading companies, from the Global Fortune 500 to the smallest companies, are adopting hybrid cloud as the path to business advantage. Hybrid cloud depends on cloud services and on-premises infrastructure working in unison. Successful implementations require new levels of data mobility, enabled by an automated and seamless flow across on-premises and cloud resources. In his general session at 21st Cloud Expo, Greg Tevis, an IBM Storage Software Technical Strategist and Customer Solution Architec...
Amazon started as an online bookseller 20 years ago. Since then, it has evolved into a technology juggernaut that has disrupted multiple markets and industries and touches many aspects of our lives. It is a relentless technology and business model innovator driving disruption throughout numerous ecosystems. Amazon’s AWS revenues alone are approaching $16B a year making it one of the largest IT companies in the world. With dominant offerings in Cloud, IoT, eCommerce, Big Data, AI, Digital Assista...
"We work around really protecting the confidentiality of information, and by doing so we've developed implementations of encryption through a patented process that is known as superencipherment," explained Richard Blech, CEO of Secure Channels Inc., in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
In his session at Cloud Expo, Alan Winters, U.S. Head of Business Development at MobiDev, presented a success story of an entrepreneur who has both suffered through and benefited from offshore development across multiple businesses: The smart choice, or how to select the right offshore development partner Warning signs, or how to minimize chances of making the wrong choice Collaboration, or how to establish the most effective work processes Budget control, or how to maximize project result...
The Founder of NostaLab and a member of the Google Health Advisory Board, John is a unique combination of strategic thinker, marketer and entrepreneur. His career was built on the "science of advertising" combining strategy, creativity and marketing for industry-leading results. Combined with his ability to communicate complicated scientific concepts in a way that consumers and scientists alike can appreciate, John is a sought-after speaker for conferences on the forefront of healthcare science,...
In his keynote at 19th Cloud Expo, Sheng Liang, co-founder and CEO of Rancher Labs, discussed the technological advances and new business opportunities created by the rapid adoption of containers. With the success of Amazon Web Services (AWS) and various open source technologies used to build private clouds, cloud computing has become an essential component of IT strategy. However, users continue to face challenges in implementing clouds, as older technologies evolve and newer ones like Docker c...
When shopping for a new data processing platform for IoT solutions, many development teams want to be able to test-drive options before making a choice. Yet when evaluating an IoT solution, it’s simply not feasible to do so at scale with physical devices. Building a sensor simulator is the next best choice; however, generating a realistic simulation at very high TPS with ease of configurability is a formidable challenge. When dealing with multiple application or transport protocols, you would be...
As organizations shift towards IT-as-a-service models, the need for managing and protecting data residing across physical, virtual, and now cloud environments grows with it. Commvault can ensure protection, access and E-Discovery of your data – whether in a private cloud, a Service Provider delivered public cloud, or a hybrid cloud environment – across the heterogeneous enterprise. In his general session at 18th Cloud Expo, Randy De Meno, Chief Technologist - Windows Products and Microsoft Part...
Personalization has long been the holy grail of marketing. Simply stated, communicate the most relevant offer to the right person and you will increase sales. To achieve this, you must understand the individual. Consequently, digital marketers developed many ways to gather and leverage customer information to deliver targeted experiences. In his session at @ThingsExpo, Lou Casal, Founder and Principal Consultant at Practicala, discussed how the Internet of Things (IoT) has accelerated our abilit...
Data is the fuel that drives the machine learning algorithmic engines and ultimately provides the business value. In his session at Cloud Expo, Ed Featherston, a director and senior enterprise architect at Collaborative Consulting, discussed the key considerations around quality, volume, timeliness, and pedigree that must be dealt with in order to properly fuel that engine.
In IT, we sometimes coin terms for things before we know exactly what they are and how they’ll be used. The resulting terms may capture a common set of aspirations and goals – as “cloud” did broadly for on-demand, self-service, and flexible computing. But such a term can also lump together diverse and even competing practices, technologies, and priorities to the point where important distinctions are glossed over and lost.