Blog Feed Post

IP Multicast: we should do so much better

IP multicast is an wonderful tool, but it’s very hard to control and debug at any decent scale. Several years ago I spent a few weeks at Dehli’s new airport trying to tune a network that carried all critical airport applications like check-in, baggage handling, signage etc, but also had to transport video feeds from 2500 security cameras. The requirements were simple: each camera spits out two 4Mbit/sec feeds, one unicast to distributed DVRs, one multicast to a set of monitoring stations. With up to 40 monitoring stations looking at 16 feeds at a time, ensure that the network converged in less than 3 seconds for unicast, less than a minute for multicast for any switch failure. The 3 seconds at the time was the magic threshold to keep IP phones connected to their signaling server. Piece of cake, right?

IP Multicast can be broken down into 2 distinctly different problems: membership management and packet delivery. If you think of multicast as a selective broadcast, you somehow need to track who has requested to receive this broadcast. Local to a switch or router this is the simplest part of IP Multicast. IGMP is used between end devices and their first router to indicate that the end device is interested in a specific multicast stream (or group). The router tracks these, and whenever a multicast packet arrives, it checks to see who had requested this group and forwards the packet out those ports. It is when you connect multiple routers together that this gets more complicated.

Protocol Independent Multicast (PIM) is pretty much the standard for IP Multicast control between routers on a network. To reach across wide area domains or create some policy control, MSDP and MBGP provide the ability to glue PIM domains together. In a way its very similar to how OSPF or ISIS are used inside a routing domain, and BGP between them.

The challenge with PIM is that it somewhat straddles membership control and actual packet forwarding, but without full control of the forwarding paths. PIM relies fully on the unicast topology to build its multicast forwarding topologies. The center point of forwarding in PIM is an entity called a Rendez-vous Point (RP). The RP is one or more PIM routers that have been selected to become the anchor point of packet distribution. Packets flow from multicast sources to the RP, then the RP sends those packets back out towards all registered receivers of that specific multicast group.

The distribution of packets from the RP to the destinations is done using a shared tree. This shared tree is a graph with the RP as its root, and each of the routers that have members of this multicast group as the leafs. The tree is constructed using unicast routing information, the individual paths of the tree towards the RP are the same of how listener would be routed to the RP for unicast. It is called a reverse tree sometimes because the tree is constructed using unicast information from the listener to the RP, but the actual traffic flow is from the RP to the listener. In his model, the RP is the center of the distribution universe and its placement in the network needs to be very carefully considered.

There are several optimizations of the traffic distribution. Where in standard PIM the traffic from the source to the RP is encapsulated as unicast, then distributed from the RP down to the listeners, there is an ability to create direct trees between the source and one or more listeners, called source trees. It removes pressure from the RP and creates more direct paths between the source and its listeners, but creates a tremendous amount of bookkeeping to track all these groups, trees and who needs to receive what. A later extension to PIM called BiDir (for Bi-Directional shared trees), allows the source to use the same RP based tree to send its traffic to the RP, which then flows back from the RP to all its listeners. It puts the RP back in the center, but significantly reduces the amount of state that needs to be tracked.

In the end, all of these are variations and optimizations based on the same theme. IP Multicast distribution is based on how unicast is delivered. Which means that different multicast streams to the same listener follow the same path to get there. The only tool to change that is to anchor different groups to different RPs, which is a completely manual exercise.

Like unicast applications, or perhaps even more pronounced, IP multicast applications have very different networking needs. Database, financial quote/transaction and other real time synchronization applications need relatively low volume but very low latency multicast distribution. Backups and archiving need lots of multicast bandwidth, but are just fine with multi microsecond or more latency. Video and voice multicast applications fit somewhere in between.

It is completely possible to build L2 and L3 multicast topologies that are different than their unicast brethren. If you have a complete view of a forwarding domain, you can calculate multicast distribution trees that use links with lots of bandwidth. Or ones that use the fewest amount of hops between sources and listeners. And of course you would take into account the amount of other (unicast) traffic that would flow on the same links to ensure they do not clash.

Perhaps easier to see than others, many multicast applications have well articulated needs and desires. My surveillance camera exercise in Delhi needed lots and lots of raw bandwidth. Close to 10Gbit/sec worth of multicast video would be traveling in and out of the monitoring stations. At the time we did not have tools to separate that from the other critical security information (smoke detectors, door alarms, you name it) that flowed into the same control room. Times are changing. Taking an application first approach, then having the means to translate that into a network and forwarding behavior will give us exactly those tools.

[Today's fun fact: Human teeth are almost as hard as rocks. Key word is "almost"]

The post IP Multicast: we should do so much better appeared first on Plexxi.

Read the original blog entry...

More Stories By Michael Bushong

The best marketing efforts leverage deep technology understanding with a highly-approachable means of communicating. Plexxi's Vice President of Marketing Michael Bushong has acquired these skills having spent 12 years at Juniper Networks where he led product management, product strategy and product marketing organizations for Juniper's flagship operating system, Junos. Michael spent the last several years at Juniper leading their SDN efforts across both service provider and enterprise markets. Prior to Juniper, Michael spent time at database supplier Sybase, and ASIC design tool companies Synopsis and Magma Design Automation. Michael's undergraduate work at the University of California Berkeley in advanced fluid mechanics and heat transfer lend new meaning to the marketing phrase "This isn't rocket science."

Latest Stories
The current age of digital transformation means that IT organizations must adapt their toolset to cover all digital experiences, beyond just the end users’. Today’s businesses can no longer focus solely on the digital interactions they manage with employees or customers; they must now contend with non-traditional factors. Whether it's the power of brand to make or break a company, the need to monitor across all locations 24/7, or the ability to proactively resolve issues, companies must adapt to...
"Loom is applying artificial intelligence and machine learning into the entire log analysis process, from start to finish and at the end you will get a human touch,” explained Sabo Taylor Diab, Vice President, Marketing at Loom Systems, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
After more than five years of DevOps, definitions are evolving, boundaries are expanding, ‘unicorns’ are no longer rare, enterprises are on board, and pundits are moving on. Can we now look at an evolution of DevOps? Should we? Is the foundation of DevOps ‘done’, or is there still too much left to do? What is mature, and what is still missing? What does the next 5 years of DevOps look like? In this Power Panel at DevOps Summit, moderated by DevOps Summit Conference Chair Andi Mann, panelists loo...
"Tintri focuses on the Ops side of the DevOps, which basically is pushing more and more of the accessibility of the infrastructure to the developers and trying to get behind the scenes," explained Dhiraj Sehgal of Tintri in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
@DevOpsSummit at Cloud Expo taking place Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center, Santa Clara, CA, is co-located with the 21st International Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is ...
You know you need the cloud, but you’re hesitant to simply dump everything at Amazon since you know that not all workloads are suitable for cloud. You know that you want the kind of ease of use and scalability that you get with public cloud, but your applications are architected in a way that makes the public cloud a non-starter. You’re looking at private cloud solutions based on hyperconverged infrastructure, but you’re concerned with the limits inherent in those technologies.
In the world of DevOps there are ‘known good practices’ – aka ‘patterns’ – and ‘known bad practices’ – aka ‘anti-patterns.' Many of these patterns and anti-patterns have been developed from real world experience, especially by the early adopters of DevOps theory; but many are more feasible in theory than in practice, especially for more recent entrants to the DevOps scene. In this power panel at @DevOpsSummit at 18th Cloud Expo, moderated by DevOps Conference Chair Andi Mann, panelists discussed...
A look across the tech landscape at the disruptive technologies that are increasing in prominence and speculate as to which will be most impactful for communications – namely, AI and Cloud Computing. In his session at 20th Cloud Expo, Curtis Peterson, VP of Operations at RingCentral, highlighted the current challenges of these transformative technologies and shared strategies for preparing your organization for these changes. This “view from the top” outlined the latest trends and developments i...
The current age of digital transformation means that IT organizations must adapt their toolset to cover all digital experiences, beyond just the end users’. Today’s businesses can no longer focus solely on the digital interactions they manage with employees or customers; they must now contend with non-traditional factors. Whether it's the power of brand to make or break a company, the need to monitor across all locations 24/7, or the ability to proactively resolve issues, companies must adapt to...
"We focus on composable infrastructure. Composable infrastructure has been named by companies like Gartner as the evolution of the IT infrastructure where everything is now driven by software," explained Bruno Andrade, CEO and Founder of HTBase, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
Hardware virtualization and cloud computing allowed us to increase resource utilization and increase our flexibility to respond to business demand. Docker Containers are the next quantum leap - Are they?! Databases always represented an additional set of challenges unique to running workloads requiring a maximum of I/O, network, CPU resources combined with data locality.
For organizations that have amassed large sums of software complexity, taking a microservices approach is the first step toward DevOps and continuous improvement / development. Integrating system-level analysis with microservices makes it easier to change and add functionality to applications at any time without the increase of risk. Before you start big transformation projects or a cloud migration, make sure these changes won’t take down your entire organization.
Cloud promises the agility required by today’s digital businesses. As organizations adopt cloud based infrastructures and services, their IT resources become increasingly dynamic and hybrid in nature. Managing these require modern IT operations and tools. In his session at 20th Cloud Expo, Raj Sundaram, Senior Principal Product Manager at CA Technologies, will discuss how to modernize your IT operations in order to proactively manage your hybrid cloud and IT environments. He will be sharing bes...
Artificial intelligence, machine learning, neural networks. We’re in the midst of a wave of excitement around AI such as hasn’t been seen for a few decades. But those previous periods of inflated expectations led to troughs of disappointment. Will this time be different? Most likely. Applications of AI such as predictive analytics are already decreasing costs and improving reliability of industrial machinery. Furthermore, the funding and research going into AI now comes from a wide range of com...
In this presentation, Striim CTO and founder Steve Wilkes will discuss practical strategies for counteracting fraud and cyberattacks by leveraging real-time streaming analytics. In his session at @ThingsExpo, Steve Wilkes, Founder and Chief Technology Officer at Striim, will provide a detailed look into leveraging streaming data management to correlate events in real time, and identify potential breaches across IoT and non-IoT systems throughout the enterprise. Strategies for processing massive ...