Blog Feed Post

The network won’t fit in your head anymore

Triggered by a discussion with a customer yesterday, it occurred to me (again?) that network engineers are creatures of habit and control. We have strong beliefs of how networks should be architected, designed and build. We have done so for long times and understand it well. We have tweaked our methods, our tools, our configuration templates. We understand our networks inside out. We have a very clear mental view of how they behave and how packets get forwarded, how they should be forwarded. It’s comfort, it’s habit, we feel (mostly) in control of the network because we have a clear model in our head.

I don’t believe this is a network engineering trait per se. Software engineers want to understand algorithms inside out, they want to understand the data modeling, types structures and relationships.


Many of us know the feeling. Something new comes around and it’s hard to put your head around it. It challenges the status quo, it changes how we do things, it changes what we (think we) know. When we are giving responsibility of something new, there is a desire to understand “it” inside out, as a mechanism to be able to control “it”.

I will bet you almost all good network engineers have a complete view of the network in their head, or at least the portion they are responsible for. They know exactly what it looks like, understand how things are connected, understand how traffic is supposed to flow. It makes it easy to detect subtle changes, they usually point to something not being right. To that purpose we have gone through great lengths to understand all the protocols that govern connectivity.

Challenging the Status Quo

When solutions come around that do not easily fit into the model we know, we prod and poke. Initially it for engineering curiosity, but ultimately it comes back to wanting or needing to understand how it works in great detail. We want to be able to create a model of the network in our mind. We want to be able to grab a piece of paper and explain how things are expected to work and why.

Totally understandable, but it is extremely limiting. Our desire to reproduce in our head what our network devices do in the real world will limit our solution space. Like it or not, except for those extremely smart few, modern day algorithms and the compute power available can create more optimal solutions than we can put together in our head. Or that piece of paper.

Multiple Dimensions

Most of the rules that govern our networks are singular in their metric space. We have a single metric that describe resources in our network and create behavior based on that. Except perhaps BGP which has a multitude of metrics, but even there we nicely ordered them in a way that we can simulate the decision process in our minds.

As Mat mentioned yesterday in his blog, the solutions we create have to evolve to much more abstract and descriptive requirements that are based in business policy. These are by definition multi dimensional, and in many cases conflicting, or at least interacting with each other. Fairly complex policy and rule engines are being developed to interpret these business policies and ultimately bring them back to the basic building blocks our datacenter resources can understand.

That is no longer something you can simulate in your head. For one rule, perhaps. Maybe even 2 or 3, or 10. But once you get to 1000s of resources and 100s of business rules that govern them, our heads are simply not big enough to compute the needed or expected result.

Simply Trust?

I am not suggesting you should simply trust whatever magic answer the policy engines (and the controllers as receivers of the resulting resource assignments) come up with. But we should realize that those results may not always be simple (or possible) to validate in our head considering the complexity of the problem that has been solved. And that is something we need to get used to. As vendors we need to provide the tools so you can carefully examine the results we created. We need to make sure you can look at the results before you or your controller implements them onto your network.

Of course the results need to look reasonable, but we may have to let go of the notion that we can recreate every choice made by hand in normal time. This is hard. It will take getting used to. But like so many other things that we as users cannot necessarily recreate, we can get there. We have to.

[Today's fun fact: Leonardo da Vinci could draw with one hand while writing with the other. Most kids today have a similar skill: play video games while texting friends and ignoring parents.]

The post The network won’t fit in your head anymore appeared first on Plexxi.

Read the original blog entry...

More Stories By Marten Terpstra

Marten Terpstra is a Product Management Director at Plexxi Inc. Marten has extensive knowledge of the architecture, design, deployment and management of enterprise and carrier networks.

Latest Stories
Is advanced scheduling in Kubernetes achievable? Yes, however, how do you properly accommodate every real-life scenario that a Kubernetes user might encounter? How do you leverage advanced scheduling techniques to shape and describe each scenario in easy-to-use rules and configurations? In his session at @DevOpsSummit at 21st Cloud Expo, Oleg Chunikhin, CTO at Kublr, will answer these questions and demonstrate techniques for implementing advanced scheduling. For example, using spot instances ...
DevOps is under attack because developers don’t want to mess with infrastructure. They will happily own their code into production, but want to use platforms instead of raw automation. That’s changing the landscape that we understand as DevOps with both architecture concepts (CloudNative) and process redefinition (SRE). Rob Hirschfeld’s recent work in Kubernetes operations has led to the conclusion that containers and related platforms have changed the way we should be thinking about DevOps and...
SYS-CON Events announced today that Taica will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Taica manufacturers Alpha-GEL brand silicone components and materials, which maintain outstanding performance over a wide temperature range -40C to +200C. For more information, visit http://www.taica.co.jp/english/.
When it comes to cloud computing, the ability to turn massive amounts of compute cores on and off on demand sounds attractive to IT staff, who need to manage peaks and valleys in user activity. With cloud bursting, the majority of the data can stay on premises while tapping into compute from public cloud providers, reducing risk and minimizing need to move large files. In his session at 18th Cloud Expo, Scott Jeschonek, Director of Product Management at Avere Systems, discussed the IT and busine...
SYS-CON Events announced today that SourceForge has been named “Media Sponsor” of SYS-CON's 21st International Cloud Expo, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. SourceForge is the largest, most trusted destination for Open Source Software development, collaboration, discovery and download on the web serving over 32 million viewers, 150 million downloads and over 460,000 active development projects each and every month.
Enterprises have taken advantage of IoT to achieve important revenue and cost advantages. What is less apparent is how incumbent enterprises operating at scale have, following success with IoT, built analytic, operations management and software development capabilities – ranging from autonomous vehicles to manageable robotics installations. They have embraced these capabilities as if they were Silicon Valley startups. As a result, many firms employ new business models that place enormous impor...
SYS-CON Events announced today that TidalScale will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. TidalScale is the leading provider of Software-Defined Servers that bring flexibility to modern data centers by right-sizing servers on the fly to fit any data set or workload. TidalScale’s award-winning inverse hypervisor technology combines multiple commodity servers (including their ass...
As popularity of the smart home is growing and continues to go mainstream, technological factors play a greater role. The IoT protocol houses the interoperability battery consumption, security, and configuration of a smart home device, and it can be difficult for companies to choose the right kind for their product. For both DIY and professionally installed smart homes, developers need to consider each of these elements for their product to be successful in the market and current smart homes.
SYS-CON Events announced today that MIRAI Inc. will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. MIRAI Inc. are IT consultants from the public sector whose mission is to solve social issues by technology and innovation and to create a meaningful future for people.
In his general session at 21st Cloud Expo, Greg Dumas, Calligo’s Vice President and G.M. of US operations, will go over the new Global Data Protection Regulation and how Calligo can help business stay compliant in digitally globalized world. Greg Dumas is Calligo's Vice President and G.M. of US operations. Calligo is an established service provider that provides an innovative platform for trusted cloud solutions. Calligo’s customers are typically most concerned about GDPR compliance, applicatio...
Companies are harnessing data in ways we once associated with science fiction. Analysts have access to a plethora of visualization and reporting tools, but considering the vast amount of data businesses collect and limitations of CPUs, end users are forced to design their structures and systems with limitations. Until now. As the cloud toolkit to analyze data has evolved, GPUs have stepped in to massively parallel SQL, visualization and machine learning.
In his Opening Keynote at 21st Cloud Expo, John Considine, General Manager of IBM Cloud Infrastructure, will lead you through the exciting evolution of the cloud. He'll look at this major disruption from the perspective of technology, business models, and what this means for enterprises of all sizes. John Considine is General Manager of Cloud Infrastructure Services at IBM. In that role he is responsible for leading IBM’s public cloud infrastructure including strategy, development, and offering ...
As hybrid cloud becomes the de-facto standard mode of operation for most enterprises, new challenges arise on how to efficiently and economically share data across environments. In his session at 21st Cloud Expo, Dr. Allon Cohen, VP of Product at Elastifile, will explore new techniques and best practices that help enterprise IT benefit from the advantages of hybrid cloud environments by enabling data availability for both legacy enterprise and cloud-native mission critical applications. By rev...
SYS-CON Events announced today that Dasher Technologies will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Dasher Technologies, Inc. ® is a premier IT solution provider that delivers expert technical resources along with trusted account executives to architect and deliver complete IT solutions and services to help our clients execute their goals, plans and objectives. Since 1999, we'v...
SYS-CON Events announced today that NetApp has been named “Bronze Sponsor” of SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. NetApp is the data authority for hybrid cloud. NetApp provides a full range of hybrid cloud data services that simplify management of applications and data across cloud and on-premises environments to accelerate digital transformation. Together with their partners, NetApp emp...