Blog Feed Post

Process Is King

How a proper software development process helps you deal with unforeseen problems.

How a proper software development process helps you deal with unforeseen problems

Software development is often unpredictable. Changes in the infrastructure or libraries we depend on happen. Bugs come up. Milestones slip by. To be able to deal with all of those changes, we need a proper process in place that will help us deal with unforeseen problems and give us the confidence to upgrade any part of our system at any time.

Often, we only perceive dependencies on a library level as they are the ones we deal with daily. Not updating those means taking on technical debt. By setting up proper automated testing and continuous delivery where the system tells us when it’s ready, we can get to a point where upgrading any library is simple. But this workflow often doesn’t extend to more basic parts of the infrastructure, like the Operating System we use on our machines. We often take them for granted as their release cycles are slow and they are typically supported for a long time. But depending on that level of support, without a well-thought-out plan to update these parts, it just pushes the pain of dealing with it to a later point, where it becomes chronic and much more difficult to manage.

So even the best development process with automated testing and continuous delivery can’t help us if we don’t consider the whole system while building workflows.

Managing Large Changes

We’re currently in the process of upgrading the Ubuntu version we use on our servers. Those changes took quite a while to implement. It wasn’t just a technical challenge to provide the same consistent high-level of quality we expect at Codeship, but also to have a process that lets us slowly release those changes and find any problems we have before the general release.

We can now roll out different versions of our infrastructure based on different operating systems and decide on which of those to run our builds. This allows us to test the whole infrastructure much more thoroughly and ensures that we don’t negatively interrupt our customers’ deployments.

We had to dive deep into the process necessary to roll out these changes and decide on how our technology needs to support those processes. It’s often hard to decide early what these specific update paths are and how to deal with them in the future. But it’s critical to set up your team and development to see change as the natural order. We fully stand behind this and it’s something that we implement across our company and technology. We need to Build for Change.

While you might not need the technology or the process to replace every part of your infrastructure from the beginning, you need to be aware of the dependencies you build and how those might affect you in the future.


In software projects, we often build up dependencies that aren’t visible on first sight, but painful once we need to deal with them. So deal with them early and often, even if it’s just as an experiment on the side that isn’t intended to go into production. At least you’ll have the data and experience to work with, and a process to deal with that changes, once it becomes inevitable.

You will have to handle unforeseen changes, upgrades to your dependencies and general problems in your infrastructure. Sometimes those might turn out as minor or sometimes they grow while investigating them, similar to what happened with our Trusty upgrade.

If you fear those changes and ignore them today, they will come back even stronger tomorrow. A minor change, a small process improvement or a minimal technology upgrade becomes an unbeatable beast that will suck up your time, energy and productivity.

Prepare your team, your process and your technology for change. This will allow you to face it without any fear. Fear is what holds us back, fear is what decreases our performance and innovation. By preparing and reducing this fear, we can build products that are truly innovative and truly loved by our users.

Did you have to make changes you were afraid of? How did you handle them? Let us know in the comments.

Read the original blog entry...

More Stories By Manuel Weiss

I am the cofounder of Codeship – a hosted Continuous Integration and Deployment platform for web applications. On the Codeship blog we love to write about Software Testing, Continuos Integration and Deployment. Also check out our weekly screencast series 'Testing Tuesday'!

Latest Stories
"Space Monkey by Vivent Smart Home is a product that is a distributed cloud-based edge storage network. Vivent Smart Home, our parent company, is a smart home provider that places a lot of hard drives across homes in North America," explained JT Olds, Director of Engineering, and Brandon Crowfeather, Product Manager, at Vivint Smart Home, in this SYS-CON.tv interview at @ThingsExpo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
SYS-CON Events announced today that Conference Guru has been named “Media Sponsor” of the 22nd International Cloud Expo, which will take place on June 5-7, 2018, at the Javits Center in New York, NY. A valuable conference experience generates new contacts, sales leads, potential strategic partners and potential investors; helps gather competitive intelligence and even provides inspiration for new products and services. Conference Guru works with conference organizers to pass great deals to gre...
DevOps is under attack because developers don’t want to mess with infrastructure. They will happily own their code into production, but want to use platforms instead of raw automation. That’s changing the landscape that we understand as DevOps with both architecture concepts (CloudNative) and process redefinition (SRE). Rob Hirschfeld’s recent work in Kubernetes operations has led to the conclusion that containers and related platforms have changed the way we should be thinking about DevOps and...
The Internet of Things will challenge the status quo of how IT and development organizations operate. Or will it? Certainly the fog layer of IoT requires special insights about data ontology, security and transactional integrity. But the developmental challenges are the same: People, Process and Platform. In his session at @ThingsExpo, Craig Sproule, CEO of Metavine, demonstrated how to move beyond today's coding paradigm and shared the must-have mindsets for removing complexity from the develop...
In his Opening Keynote at 21st Cloud Expo, John Considine, General Manager of IBM Cloud Infrastructure, led attendees through the exciting evolution of the cloud. He looked at this major disruption from the perspective of technology, business models, and what this means for enterprises of all sizes. John Considine is General Manager of Cloud Infrastructure Services at IBM. In that role he is responsible for leading IBM’s public cloud infrastructure including strategy, development, and offering m...
The next XaaS is CICDaaS. Why? Because CICD saves developers a huge amount of time. CD is an especially great option for projects that require multiple and frequent contributions to be integrated. But… securing CICD best practices is an emerging, essential, yet little understood practice for DevOps teams and their Cloud Service Providers. The only way to get CICD to work in a highly secure environment takes collaboration, patience and persistence. Building CICD in the cloud requires rigorous ar...
Companies are harnessing data in ways we once associated with science fiction. Analysts have access to a plethora of visualization and reporting tools, but considering the vast amount of data businesses collect and limitations of CPUs, end users are forced to design their structures and systems with limitations. Until now. As the cloud toolkit to analyze data has evolved, GPUs have stepped in to massively parallel SQL, visualization and machine learning.
"Evatronix provides design services to companies that need to integrate the IoT technology in their products but they don't necessarily have the expertise, knowledge and design team to do so," explained Adam Morawiec, VP of Business Development at Evatronix, in this SYS-CON.tv interview at @ThingsExpo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
To get the most out of their data, successful companies are not focusing on queries and data lakes, they are actively integrating analytics into their operations with a data-first application development approach. Real-time adjustments to improve revenues, reduce costs, or mitigate risk rely on applications that minimize latency on a variety of data sources. In his session at @BigDataExpo, Jack Norris, Senior Vice President, Data and Applications at MapR Technologies, reviewed best practices to ...
Widespread fragmentation is stalling the growth of the IIoT and making it difficult for partners to work together. The number of software platforms, apps, hardware and connectivity standards is creating paralysis among businesses that are afraid of being locked into a solution. EdgeX Foundry is unifying the community around a common IoT edge framework and an ecosystem of interoperable components.
"ZeroStack is a startup in Silicon Valley. We're solving a very interesting problem around bringing public cloud convenience with private cloud control for enterprises and mid-size companies," explained Kamesh Pemmaraju, VP of Product Management at ZeroStack, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
Large industrial manufacturing organizations are adopting the agile principles of cloud software companies. The industrial manufacturing development process has not scaled over time. Now that design CAD teams are geographically distributed, centralizing their work is key. With large multi-gigabyte projects, outdated tools have stifled industrial team agility, time-to-market milestones, and impacted P&L stakeholders.
"Akvelon is a software development company and we also provide consultancy services to folks who are looking to scale or accelerate their engineering roadmaps," explained Jeremiah Mothersell, Marketing Manager at Akvelon, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
Enterprises are adopting Kubernetes to accelerate the development and the delivery of cloud-native applications. However, sharing a Kubernetes cluster between members of the same team can be challenging. And, sharing clusters across multiple teams is even harder. Kubernetes offers several constructs to help implement segmentation and isolation. However, these primitives can be complex to understand and apply. As a result, it’s becoming common for enterprises to end up with several clusters. Thi...
"Infoblox does DNS, DHCP and IP address management for not only enterprise networks but cloud networks as well. Customers are looking for a single platform that can extend not only in their private enterprise environment but private cloud, public cloud, tracking all the IP space and everything that is going on in that environment," explained Steve Salo, Principal Systems Engineer at Infoblox, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Conventio...