Welcome!

Blog Feed Post

Automation through workflow state

The benefits of automation are well understood: more agile service provisioning, faster time to insight when there are issues, and a reduction in human error as manual interaction is reduced. Much of the premise behind long-term SDN architectural advantages is steeped in the hope that SDN will help enable and ultimately promote automation. But while centralizing control has significant operational advantages, by itself, it doesn’t actually address the most important requirement for automation.

If automation is going to be more than just reducing keystrokes, there will have to be a rise of workflow state.

Referential space

Successfully managing a network is an exercise in constant iteration through network state. Whenever something needs to be done, the architect or operator examines her current frame of reference to figure out the starting point. That frame of reference usually starts with some implicit understanding of how the network is designed. From there, she takes some action. Maybe she pings an endpoint, checks the state of a BGP neighbor, or examines some interface statistics. Whatever the first step, the point is that she knows when she starts that there is work after the first step.

The information gleaned from the first step yields additional understanding. Her frame of reference changes as she now knows more than before. With her new position in referential space, she takes the next step. And the next, and the next after that. Each step yields a different piece of information, and the process of iterating through a constantly changing referential space ultimately yields some outcome or resolution.

Byproducts of iterative workflows

There are two major byproducts of this iterative approach to workflow. The first is that the starting point is rarely based on an absolute understanding of fact. Rather it is an interpretation that the individual operator or architect creates based on a number of somewhat soft conditions – knowledge, experience, intuition, whatever. This means that for each task, the workflow is somewhat unique, depending on the operator and the environment.

The impact here is important. If workflows are unique based on the operator and the conditions (i.e., the referential space or frame of reference), then the outcomes driven by those workflows are difficult to repeat. Part of why networking is so hard is that so much of it borders on arcane dark art. Science demands repeatability, but the very nature of workflow management in networking makes that challenging.

The second byproduct of networking’s iterative nature is that workflows frequently depend on a set of chained tasks, each of which has a dependency on the preceding task. To make things worse, that dependency is actually rarely known at the start of a a workflow. It’s not that tasks cannot be predictably chained – first, you look at the physical layer, and then you move up stack perhaps. But each subsequent task is executed based on not just the previous task but also the output of the previous task. This creates a complex set of if/then statements in most workflows.

Part of the challenge in automation is providing the logic to navigate the conditional nature of networking workflows.

“Network engineers need to think like programmers”

With the rise of movements like DevOps, “network engineers thinking like programmers” has become a popular phrase. This is a very important change in how we handle network architecture and operations. But there are subtleties here that get lost in the cliche.

First, when people toss the phrase around, they often mean that network engineers need to pick up a scripting language (Python, Ruby, even Perl). Thinking like a software developer has very little to do with programming languages. Languages are a way of expressing intent, but it’s entirely possible to know Python and think nothing like a developer.

Second, when people refer to programming in the context of DevOps, they generally mean that network operators need to think about configuration less as a collection of commands and more like code. Once you make that shift, then you can think about things like source code management, automated testing, and rapid deployment.

But networking needs to do more than just treat configuration as code.  DevOps has more to do with deploying and validating changes. It doesn’t fundamentally change how workflows are executed, and it barely touches more operational tasks like troubleshooting network conditions.

Before anyone picks a religious battle over DevOps here, my point is not that DevOps is bad. It’s just that DevOps by itself is not sufficient. And there are things that ought to be done that are separate from DevOps.

Tiny feedback loops

So if thinking like a programmer isn’t about learning a programming language and it’s more than treating configuration as code, what is it?

Software development is really about creating something out of lots of tiny feedback loops. When you write functions, you don’t just execute some task. You generally execute that task and then return a value. The value provides some immediate feedback about the outcome. In some cases, the function returns the value of a computation; in other cases, it simply returns an indication that the function succeeded or failed.

These values are obviously then used by other functions, which allows us to string together small building blocks into complex chains. The important part? These chains can then be repeatably executed in a deterministic way.

Networking workflows shouldn’t be that different. Each individual activity yields some value (sometimes a specific value as when looking at some counter, other times a success or failure as with a ping). The problem is that while networking commands frequently return information, it is up to the operator themselves to parse this information, analyze what it means, and then take the next action.

Workflow state

What we need if we really want to make automation happen in ways that extend beyond just scripting keystrokes is a means of creating deterministic networking workflows. For this to happen, we need people who construct workflows to think more like developers. Each activity within a workflow needs to be a tiny feedback loop with explicit workflow state that is programmatically passed between workflow elements.

We actually instinctively do this at times. XML, NETCONF, and the like have been used to encapsulate networking inputs and outputs for awhile with the intent of making things parseable and thus more automatable.

But we stopped short. We made the outputs more automation-friendly without ever really creating workflows. So while we can programmatically act on values, it only works if someone has automated a particular workflow. As an industry, we haven’t gotten to actually addressing the workflow problem.

Maybe it’s the highly conditional nature of networking combined with the uniqueness of individual networks. Or maybe it’s that outside of a few automation savants, our industry doesn’t generally think about workflows the way a software developer would.

The bottom line

Networking workflows rely way too heavily on an iterative pass through referential space. The reason change is so scary and troubleshooting so hard is that very little in networking is actually deterministic. But if we really want to improve the overall user experience en route to making workflows both repeatable and reliable, we do need to start thinking a bit more like developers. It all starts with a more explicit understanding of the workflows we rely on, and the expression of feedback via some form of workflow state.

And for everyone betting on abstractions, just know that abstracting a poorly-defined workflow results in an equally poor abstraction. We need to be starting elsewhere.

[Today’s fun fact: Only male fireflies can fly. Take that, females!]

The post Automation through workflow state appeared first on Plexxi.

Read the original blog entry...

More Stories By Michael Bushong

The best marketing efforts leverage deep technology understanding with a highly-approachable means of communicating. Plexxi's Vice President of Marketing Michael Bushong has acquired these skills having spent 12 years at Juniper Networks where he led product management, product strategy and product marketing organizations for Juniper's flagship operating system, Junos. Michael spent the last several years at Juniper leading their SDN efforts across both service provider and enterprise markets. Prior to Juniper, Michael spent time at database supplier Sybase, and ASIC design tool companies Synopsis and Magma Design Automation. Michael's undergraduate work at the University of California Berkeley in advanced fluid mechanics and heat transfer lend new meaning to the marketing phrase "This isn't rocket science."

Latest Stories
21st International Cloud Expo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy. Me...
SYS-CON Events announced today that Enroute Lab will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Enroute Lab is an industrial design, research and development company of unmanned robotic vehicle system. For more information, please visit http://elab.co.jp/.
SYS-CON Events announced today that MIRAI Inc. will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. MIRAI Inc. are IT consultants from the public sector whose mission is to solve social issues by technology and innovation and to create a meaningful future for people.
With the rise of DevOps, containers are at the brink of becoming a pervasive technology in Enterprise IT to accelerate application delivery for the business. When it comes to adopting containers in the enterprise, security is the highest adoption barrier. Is your organization ready to address the security risks with containers for your DevOps environment? In his session at @DevOpsSummit at 21st Cloud Expo, Chris Van Tuin, Chief Technologist, NA West at Red Hat, will discuss: The top security r...
IBM helps FinTechs and financial services companies build and monetize cognitive-enabled financial services apps quickly and at scale. Hosted on IBM Bluemix, IBM’s platform builds in customer insights, regulatory compliance analytics and security to help reduce development time and testing. In his session at 21st Cloud Expo, Lennart Frantzell, a Developer Advocate with IBM, will discuss how these tools simplify the time-consuming tasks of selection, mapping and data integration, allowing devel...
SYS-CON Events announced today that Mobile Create USA will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Mobile Create USA Inc. is an MVNO-based business model that uses portable communication devices and cellular-based infrastructure in the development, sales, operation and mobile communications systems incorporating GPS capabi...
Today traditional IT approaches leverage well-architected compute/networking domains to control what applications can access what data, and how. DevOps includes rapid application development/deployment leveraging concepts like containerization, third-party sourced applications and databases. Such applications need access to production data for its test and iteration cycles. Data Security? That sounds like a roadblock to DevOps vs. protecting the crown jewels to those in IT.
SYS-CON Events announced today that Interface Corporation will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Interface Corporation is a company developing, manufacturing and marketing high quality and wide variety of industrial computers and interface modules such as PCIs and PCI express. For more information, visit http://www.i...
SYS-CON Events announced today that Keisoku Research Consultant Co. will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Keisoku Research Consultant, Co. offers research and consulting in a wide range of civil engineering-related fields from information construction to preservation of cultural properties. For more information, vi...
There is huge complexity in implementing a successful digital business that requires efficient on-premise and cloud back-end infrastructure, IT and Internet of Things (IoT) data, analytics, Machine Learning, Artificial Intelligence (AI) and Digital Applications. In the data center alone, there are physical and virtual infrastructures, multiple operating systems, multiple applications and new and emerging business and technological paradigms such as cloud computing and XaaS. And then there are pe...
SYS-CON Events announced today that SIGMA Corporation will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. uLaser flow inspection device from the Japanese top share to Global Standard! Then, make the best use of data to flip to next page. For more information, visit http://www.sigma-k.co.jp/en/.
SYS-CON Events announced today that B2Cloud will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. B2Cloud specializes in IoT devices for preventive and predictive maintenance in any kind of equipment retrieving data like Energy consumption, working time, temperature, humidity, pressure, etc.
Agile has finally jumped the technology shark, expanding outside the software world. Enterprises are now increasingly adopting Agile practices across their organizations in order to successfully navigate the disruptive waters that threaten to drown them. In our quest for establishing change as a core competency in our organizations, this business-centric notion of Agile is an essential component of Agile Digital Transformation. In the years since the publication of the Agile Manifesto, the conn...
While some developers care passionately about how data centers and clouds are architected, for most, it is only the end result that matters. To the majority of companies, technology exists to solve a business problem, and only delivers value when it is solving that problem. 2017 brings the mainstream adoption of containers for production workloads. In his session at 21st Cloud Expo, Ben McCormack, VP of Operations at Evernote, will discuss how data centers of the future will be managed, how th...
SYS-CON Events announced today that NetApp has been named “Bronze Sponsor” of SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. NetApp is the data authority for hybrid cloud. NetApp provides a full range of hybrid cloud data services that simplify management of applications and data across cloud and on-premises environments to accelerate digital transformation. Together with their partners, NetApp em...