Blog Feed Post

IP Multicast: we should do so much better

IP multicast is an wonderful tool, but it’s very hard to control and debug at any decent scale. Several years ago I spent a few weeks at Dehli’s new airport trying to tune a network that carried all critical airport applications like check-in, baggage handling, signage etc, but also had to transport video feeds from 2500 security cameras. The requirements were simple: each camera spits out two 4Mbit/sec feeds, one unicast to distributed DVRs, one multicast to a set of monitoring stations. With up to 40 monitoring stations looking at 16 feeds at a time, ensure that the network converged in less than 3 seconds for unicast, less than a minute for multicast for any switch failure. The 3 seconds at the time was the magic threshold to keep IP phones connected to their signaling server. Piece of cake, right?

IP Multicast can be broken down into 2 distinctly different problems: membership management and packet delivery. If you think of multicast as a selective broadcast, you somehow need to track who has requested to receive this broadcast. Local to a switch or router this is the simplest part of IP Multicast. IGMP is used between end devices and their first router to indicate that the end device is interested in a specific multicast stream (or group). The router tracks these, and whenever a multicast packet arrives, it checks to see who had requested this group and forwards the packet out those ports. It is when you connect multiple routers together that this gets more complicated.

Protocol Independent Multicast (PIM) is pretty much the standard for IP Multicast control between routers on a network. To reach across wide area domains or create some policy control, MSDP and MBGP provide the ability to glue PIM domains together. In a way its very similar to how OSPF or ISIS are used inside a routing domain, and BGP between them.

The challenge with PIM is that it somewhat straddles membership control and actual packet forwarding, but without full control of the forwarding paths. PIM relies fully on the unicast topology to build its multicast forwarding topologies. The center point of forwarding in PIM is an entity called a Rendez-vous Point (RP). The RP is one or more PIM routers that have been selected to become the anchor point of packet distribution. Packets flow from multicast sources to the RP, then the RP sends those packets back out towards all registered receivers of that specific multicast group.

The distribution of packets from the RP to the destinations is done using a shared tree. This shared tree is a graph with the RP as its root, and each of the routers that have members of this multicast group as the leafs. The tree is constructed using unicast routing information, the individual paths of the tree towards the RP are the same of how listener would be routed to the RP for unicast. It is called a reverse tree sometimes because the tree is constructed using unicast information from the listener to the RP, but the actual traffic flow is from the RP to the listener. In his model, the RP is the center of the distribution universe and its placement in the network needs to be very carefully considered.

There are several optimizations of the traffic distribution. Where in standard PIM the traffic from the source to the RP is encapsulated as unicast, then distributed from the RP down to the listeners, there is an ability to create direct trees between the source and one or more listeners, called source trees. It removes pressure from the RP and creates more direct paths between the source and its listeners, but creates a tremendous amount of bookkeeping to track all these groups, trees and who needs to receive what. A later extension to PIM called BiDir (for Bi-Directional shared trees), allows the source to use the same RP based tree to send its traffic to the RP, which then flows back from the RP to all its listeners. It puts the RP back in the center, but significantly reduces the amount of state that needs to be tracked.

In the end, all of these are variations and optimizations based on the same theme. IP Multicast distribution is based on how unicast is delivered. Which means that different multicast streams to the same listener follow the same path to get there. The only tool to change that is to anchor different groups to different RPs, which is a completely manual exercise.

Like unicast applications, or perhaps even more pronounced, IP multicast applications have very different networking needs. Database, financial quote/transaction and other real time synchronization applications need relatively low volume but very low latency multicast distribution. Backups and archiving need lots of multicast bandwidth, but are just fine with multi microsecond or more latency. Video and voice multicast applications fit somewhere in between.

It is completely possible to build L2 and L3 multicast topologies that are different than their unicast brethren. If you have a complete view of a forwarding domain, you can calculate multicast distribution trees that use links with lots of bandwidth. Or ones that use the fewest amount of hops between sources and listeners. And of course you would take into account the amount of other (unicast) traffic that would flow on the same links to ensure they do not clash.

Perhaps easier to see than others, many multicast applications have well articulated needs and desires. My surveillance camera exercise in Delhi needed lots and lots of raw bandwidth. Close to 10Gbit/sec worth of multicast video would be traveling in and out of the monitoring stations. At the time we did not have tools to separate that from the other critical security information (smoke detectors, door alarms, you name it) that flowed into the same control room. Times are changing. Taking an application first approach, then having the means to translate that into a network and forwarding behavior will give us exactly those tools.

[Today's fun fact: Human teeth are almost as hard as rocks. Key word is "almost"]

The post IP Multicast: we should do so much better appeared first on Plexxi.

Read the original blog entry...

More Stories By Michael Bushong

The best marketing efforts leverage deep technology understanding with a highly-approachable means of communicating. Plexxi's Vice President of Marketing Michael Bushong has acquired these skills having spent 12 years at Juniper Networks where he led product management, product strategy and product marketing organizations for Juniper's flagship operating system, Junos. Michael spent the last several years at Juniper leading their SDN efforts across both service provider and enterprise markets. Prior to Juniper, Michael spent time at database supplier Sybase, and ASIC design tool companies Synopsis and Magma Design Automation. Michael's undergraduate work at the University of California Berkeley in advanced fluid mechanics and heat transfer lend new meaning to the marketing phrase "This isn't rocket science."

Latest Stories
"Digital transformation - what we knew about it in the past has been redefined. Automation is going to play such a huge role in that because the culture, the technology, and the business operations are being shifted now," stated Brian Boeggeman, VP of Alliances & Partnerships at Ayehu, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
The past few years have brought a sea change in the way applications are architected, developed, and consumed—increasing both the complexity of testing and the business impact of software failures. How can software testing professionals keep pace with modern application delivery, given the trends that impact both architectures (cloud, microservices, and APIs) and processes (DevOps, agile, and continuous delivery)? This is where continuous testing comes in. D
"Evatronix provides design services to companies that need to integrate the IoT technology in their products but they don't necessarily have the expertise, knowledge and design team to do so," explained Adam Morawiec, VP of Business Development at Evatronix, in this SYS-CON.tv interview at @ThingsExpo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
"WineSOFT is a software company making proxy server software, which is widely used in the telecommunication industry or the content delivery networks or e-commerce," explained Jonathan Ahn, COO of WineSOFT, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
Smart cities have the potential to change our lives at so many levels for citizens: less pollution, reduced parking obstacles, better health, education and more energy savings. Real-time data streaming and the Internet of Things (IoT) possess the power to turn this vision into a reality. However, most organizations today are building their data infrastructure to focus solely on addressing immediate business needs vs. a platform capable of quickly adapting emerging technologies to address future ...
Mobile device usage has increased exponentially during the past several years, as consumers rely on handhelds for everything from news and weather to banking and purchases. What can we expect in the next few years? The way in which we interact with our devices will fundamentally change, as businesses leverage Artificial Intelligence. We already see this taking shape as businesses leverage AI for cost savings and customer responsiveness. This trend will continue, as AI is used for more sophistica...
There is a huge demand for responsive, real-time mobile and web experiences, but current architectural patterns do not easily accommodate applications that respond to events in real time. Common solutions using message queues or HTTP long-polling quickly lead to resiliency, scalability and development velocity challenges. In his session at 21st Cloud Expo, Ryland Degnan, a Senior Software Engineer on the Netflix Edge Platform team, will discuss how by leveraging a reactive stream-based protocol,...
In his Opening Keynote at 21st Cloud Expo, John Considine, General Manager of IBM Cloud Infrastructure, led attendees through the exciting evolution of the cloud. He looked at this major disruption from the perspective of technology, business models, and what this means for enterprises of all sizes. John Considine is General Manager of Cloud Infrastructure Services at IBM. In that role he is responsible for leading IBM’s public cloud infrastructure including strategy, development, and offering m...
Sanjeev Sharma Joins June 5-7, 2018 @DevOpsSummit at @Cloud Expo New York Faculty. Sanjeev Sharma is an internationally known DevOps and Cloud Transformation thought leader, technology executive, and author. Sanjeev's industry experience includes tenures as CTO, Technical Sales leader, and Cloud Architect leader. As an IBM Distinguished Engineer, Sanjeev is recognized at the highest levels of IBM's core of technical leaders.
Product connectivity goes hand and hand these days with increased use of personal data. New IoT devices are becoming more personalized than ever before. In his session at 22nd Cloud Expo | DXWorld Expo, Nicolas Fierro, CEO of MIMIR Blockchain Solutions, will discuss how in order to protect your data and privacy, IoT applications need to embrace Blockchain technology for a new level of product security never before seen - or needed.
The 22nd International Cloud Expo | 1st DXWorld Expo has announced that its Call for Papers is open. Cloud Expo | DXWorld Expo, to be held June 5-7, 2018, at the Javits Center in New York, NY, brings together Cloud Computing, Digital Transformation, Big Data, Internet of Things, DevOps, Machine Learning and WebRTC to one location. With cloud computing driving a higher percentage of enterprise IT budgets every year, it becomes increasingly important to plant your flag in this fast-expanding busin...
Digital transformation is about embracing digital technologies into a company's culture to better connect with its customers, automate processes, create better tools, enter new markets, etc. Such a transformation requires continuous orchestration across teams and an environment based on open collaboration and daily experiments. In his session at 21st Cloud Expo, Alex Casalboni, Technical (Cloud) Evangelist at Cloud Academy, explored and discussed the most urgent unsolved challenges to achieve f...
SYS-CON Events announced today that Synametrics Technologies will exhibit at SYS-CON's 22nd International Cloud Expo®, which will take place on June 5-7, 2018, at the Javits Center in New York, NY. Synametrics Technologies is a privately held company based in Plainsboro, New Jersey that has been providing solutions for the developer community since 1997. Based on the success of its initial product offerings such as WinSQL, Xeams, SynaMan and Syncrify, Synametrics continues to create and hone inn...
Digital Transformation (DX) is not a "one-size-fits all" strategy. Each organization needs to develop its own unique, long-term DX plan. It must do so by realizing that we now live in a data-driven age, and that technologies such as Cloud Computing, Big Data, the IoT, Cognitive Computing, and Blockchain are only tools. In her general session at 21st Cloud Expo, Rebecca Wanta explained how the strategy must focus on DX and include a commitment from top management to create great IT jobs, monitor ...
In a recent survey, Sumo Logic surveyed 1,500 customers who employ cloud services such as Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP). According to the survey, a quarter of the respondents have already deployed Docker containers and nearly as many (23 percent) are employing the AWS Lambda serverless computing framework. It’s clear: serverless is here to stay. The adoption does come with some needed changes, within both application development and operations. Tha...