Welcome!

Related Topics: SDN Journal, Java IoT, Microservices Expo, Containers Expo Blog

SDN Journal: Blog Post

Scripting Is Automation, But Automation Is Not Scripting

There are many extremely complex clustered applications that rely entirely on exchanging information through APIs

Last week Greg Ferro (@etherealmind) wrote this article about his experience with scripting as a method for network automation, with the ultimate conclusion that scripting does not scale.

Early in my career I managed a small network that grew to be a IP over X.25 hub of Europe for a few years providing many countries with their first Internet connectivity. Scripts were everywhere, small ones to grab stats and create pretty graphs, others that continuously checked the status of links and would send emails when things went wrong.

While it is hard to argue with Greg’s complaints per se, I believe the key point is missing. And it has nothing to do with scripting. In a reply, Ivan’s last comment touches on the real issue.

We have been scripting our networks against CLIs forever and I will bet you most folks will consider it successful, even though it may be a pain. The article lists the pains, but not the reasons why. As a network industry, we have never ever considered the interaction with our network devices an API. Not in the true software engineering sense of an API.

There are many extremely complex clustered applications that rely entirely on exchanging information through APIs that are well documented, well versioned, well abstracted and properly promoted or deprecated. Creating and maintaining APIs is a real software engineering effort, a skill that requires true architecture, engineering and discipline. And we have not given our users anything close to it.

If we (that collective network industry) had truly considered our CLI an API, we would (and should) have been pushed aside a long time ago. The CLI is and always has been a simple interface for a human to tell a device what to do. It was not designed to be automated. It is not structured enough to be automated. Even large vendors have multiple flavors that are all industry standard, but all slightly different. And nowhere would you find a formal, full and complete dictionary of that CLI with all inputs, outputs, versions and options. The closest the network industry has had to a true API is SNMP, and that is indeed a very sad statement.

I think we have mentioned before that the networking industry is a bit slow to get to modern software engineering methods and practices, but the tide is changing. And whether you want to call it SDN or something else, the sheer volume and complexity of interaction with the network is pushing us to provide truly automated access to our devices and our networks.

And creating and maintaining APIs is far more than the technology used to access them. It does not matter whether its XML, JSON, REST, NETCONF or anything else. Those are definitions of how information is carried to and from the device and network. I can build a wonderful REST API that takes a CLI command as an argument and spits me back the output from that CLI command in some format. I am sure that sounds familiar to some, but this is not an API. Not in a truly meaningful way that would elevate our automation abilities.

Designing and implementing APIs is not trivial. Believe me, as an entirely API driven solution, we spend a tremendous amount of time discussing our APIs and abstractions to make sure they find that find balance between granularity, functionality, abstraction, scaling and a few other relevant qualifiers.  But the key is that they are part of any feature design from day one, they are part of the overarching architecture, not bolted on at the end. Our APIs are not perfect, there is no such thing, but they are modeled after the workflow of you the user doing the work required to keep the network running and thriving.

So when you need to configure MLAG on a set of Plexxi switches, we do not have a series of API calls to bundle ports together on a switch, give them a unique ID, then tie the switches together as an MLAG pair that shares that unique ID. Oh, and create an MLAG control channel between them, and make each of the switch local LAGs have the same set of VLANs on them. Our API will simply take a list of port objects from any amount of switches in a Plexxi network and turn them into an MLAG. An then you can simply take that entire entity and stick a VLAN on top, we will make sure the participating switches get the pieces they need. That is abstraction, that is workflow encapsulation, that is what APIs are supposed to give you. That is how simple LAG is supposed to be.

We have a long way to go as an industry to get to full APIs the way real software folks think about APIs. The CLI is not it. Scripting against a CLI (or a CLI hidden behind a layer of official sounding API terms) is a useful tool, but one that should be mostly retired to get to true programmable networks that are controlled by real controller (in the broadest definition of the word) using real APIs. Automation is not scripting.

[Today's fun fact: to make sure you do not think I am anti scripting, I once wrote a large chunk of a 10,000 line Perl4 system. It functioned very nicely for years as the RIPE database for IP address allocations back in the mid 90s. Thankfully it has since been tackled by real software engineers.]

The post Scripting is automation, but automation is not scripting appeared first on Plexxi.

Read the original blog entry...

More Stories By Marten Terpstra

Marten Terpstra is a Product Management Director at Plexxi Inc. Marten has extensive knowledge of the architecture, design, deployment and management of enterprise and carrier networks.

Latest Stories
As the world moves toward more DevOps and Microservices, application deployment to the cloud ought to become a lot simpler. The Microservices architecture, which is the basis of many new age distributed systems such as OpenStack, NetFlix and so on, is at the heart of Cloud Foundry - a complete developer-oriented Platform as a Service (PaaS) that is IaaS agnostic and supports vCloud, OpenStack and AWS. Serverless computing is revolutionizing computing. In his session at 19th Cloud Expo, Raghav...
The 19th International Cloud Expo has announced that its Call for Papers is open. Cloud Expo, to be held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, brings together Cloud Computing, Big Data, Internet of Things, DevOps, Digital Transformation, Microservices and WebRTC to one location. With cloud computing driving a higher percentage of enterprise IT budgets every year, it becomes increasingly important to plant your flag in this fast-expanding business opportuni...
Enterprises have forever faced challenges surrounding the sharing of their intellectual property. Emerging cloud adoption has made it more compelling for enterprises to digitize their content, making them available over a wide variety of devices across the Internet. In his session at 19th Cloud Expo, Santosh Ahuja, Director of Architecture at Impiger Technologies, will introduce various mechanisms provided by cloud service providers today to manage and share digital content in a secure manner....
StarNet Communications Corp has announced the addition of three Secure Remote Desktop modules to its flagship X-Win32 PC X server. The new modules enable X-Win32 to safely tunnel the remote desktops from Linux and Unix servers to the user’s PC over encrypted SSH. Traditionally, users of PC X servers deploy the XDMCP protocol to display remote desktop environments such as the Gnome and KDE desktops on Linux servers and the CDE environment on Solaris Unix machines. XDMCP is used primarily on comp...
Fact: storage performance problems have only gotten more complicated, as applications not only have become largely virtualized, but also have moved to cloud-based infrastructures. Storage performance in virtualized environments isn’t just about IOPS anymore. Instead, you need to guarantee performance for individual VMs, helping applications maintain performance as the number of VMs continues to go up in real time. In his session at Cloud Expo, Dhiraj Sehgal, Product and Marketing at Tintri, wil...
DevOps at Cloud Expo, taking place Nov 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 19th Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to wait for long dev...
SYS-CON Events announced today that Isomorphic Software will exhibit at DevOps Summit at 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Isomorphic Software provides the SmartClient HTML5/AJAX platform, the most advanced technology for building rich, cutting-edge enterprise web applications for desktop and mobile. SmartClient combines the productivity and performance of traditional desktop software with the simp...
With so much going on in this space you could be forgiven for thinking you were always working with yesterday’s technologies. So much change, so quickly. What do you do if you have to build a solution from the ground up that is expected to live in the field for at least 5-10 years? This is the challenge we faced when we looked to refresh our existing 10-year-old custom hardware stack to measure the fullness of trash cans and compactors.
Extreme Computing is the ability to leverage highly performant infrastructure and software to accelerate Big Data, machine learning, HPC, and Enterprise applications. High IOPS Storage, low-latency networks, in-memory databases, GPUs and other parallel accelerators are being used to achieve faster results and help businesses make better decisions. In his session at 18th Cloud Expo, Michael O'Neill, Strategic Business Development at NVIDIA, focused on some of the unique ways extreme computing is...
The emerging Internet of Everything creates tremendous new opportunities for customer engagement and business model innovation. However, enterprises must overcome a number of critical challenges to bring these new solutions to market. In his session at @ThingsExpo, Michael Martin, CTO/CIO at nfrastructure, outlined these key challenges and recommended approaches for overcoming them to achieve speed and agility in the design, development and implementation of Internet of Everything solutions wi...
Cloud computing is being adopted in one form or another by 94% of enterprises today. Tens of billions of new devices are being connected to The Internet of Things. And Big Data is driving this bus. An exponential increase is expected in the amount of information being processed, managed, analyzed, and acted upon by enterprise IT. This amazing is not part of some distant future - it is happening today. One report shows a 650% increase in enterprise data by 2020. Other estimates are even higher....
Today we can collect lots and lots of performance data. We build beautiful dashboards and even have fancy query languages to access and transform the data. Still performance data is a secret language only a couple of people understand. The more business becomes digital the more stakeholders are interested in this data including how it relates to business. Some of these people have never used a monitoring tool before. They have a question on their mind like “How is my application doing” but no id...
Identity is in everything and customers are looking to their providers to ensure the security of their identities, transactions and data. With the increased reliance on cloud-based services, service providers must build security and trust into their offerings, adding value to customers and improving the user experience. Making identity, security and privacy easy for customers provides a unique advantage over the competition.
Qosmos has announced new milestones in the detection of encrypted traffic and in protocol signature coverage. Qosmos latest software can accurately classify traffic encrypted with SSL/TLS (e.g., Google, Facebook, WhatsApp), P2P traffic (e.g., BitTorrent, MuTorrent, Vuze), and Skype, while preserving the privacy of communication content. These new classification techniques mean that traffic optimization, policy enforcement, and user experience are largely unaffected by encryption. In respect wit...
19th Cloud Expo, taking place November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy. Meanwhile, 94% of enterpri...