Welcome!

Blog Feed Post

This Space Almost Intentionally Left Blank.

I had just finished writing this blog and was about to post it, taking that moment to go do other things before one final read-through, and guess what? I saw this article on Dell’s site cross my Twitter account. I’ll blame the fact that I’m the shiny new Technical Marketing Manager as the reason I did not know this was going on. I thought about just ditching the blog at that point… After all, the article linked to on Dell’s site is much more in-depth. But the blog was done. And I think it still adds value in the generic replication sense. So I leave it as-is. Though I suggest that if you’re interested in speeding replications between EqualLogic boxes, or in how F5 specifically speeds up massive data transfers, the article linked above is worth a read also.


IS IT REPLICATION OR SNAPSHOTS?

The geeks over at Dell recently did a Tech Report on replication between two Dell EqualLogic Groups, with the claim that it didn’t matter where those two groups were geographically located. Like their claim in the same PDF that what they’re doing – which by description sounds suspiciously like everyone else’s snapshotting technology – is replication, I had to congenially come to the conclusion that I disagreed with their contention that it didn’t matter where the two groups were. Don’t get me wrong, this is an excellent tech article, and if you’re a customer (or considering becoming one), it’s worth a read. Just hit a couple of items that felt more like marketing than tech to me.


EITHER WAY, ITS A TON OF DATA

Remote replication – and remote snapshotting since they share the fact that they ship a lot of data over the wire – has always been bedeviled by the simple truth of the Internet. Latency, retransmits, bandwidth, and security all interfere with what would be a simple procedure on the LAN.

Of course for every problem in technology there are people to solve it, and speeding remote replication is not a new problem, so there are a large pool of vendors out there to solve it. Chances are pretty good that at least one of your SAN/NAS/backup vendors solves it between their equipment. That’s the age old struggle in storage though, homogeneous or heterogeneous storage infrastructure. With homogeneous you get undeniable benefits like faster replication, but you only get it for select equipment, which is rarely ideal.


WHAT’S THE BIG DEAL?

For those that do not know, the key points that interfere with replication are actually pretty straight-forward and have been known entities for a rather long time (contrary to some of the hype I see out there now, particularly in the “cloud gateway” category), and have been resolved by a decent number of organizations. They are as follows…

  1. TCP Chattiness
  2. Packet loss/latency
  3. Data Volume in replication/snapshot
  4. Protocol overhead
  5. Security

The first point is simple acknowledgment that TCP chatters a lot maintaining a connection, and some of that overhead doesn’t necessarily have to cross the wire.

The second are the perennial problems of long distance computer communications. TCP is designed to deal with them, but the solutions create more chattiness and redundant transmissions. The worse the connection is, the more overhead is introduced… Which of course makes the connection appear even worse.

Data volume inevitably plagues remote data transfers. There are two sides to this issue that you have to contend with. The first is the window of time it takes to do the backup, the second is the usage of the connection into and out of the target data centers. Most organizations cannot have replication or snapshots taking up 100% – or even 50% – of their bandwidth.

Of course, the app layer protocol that you’re using – NDMP, CIFS, NFS, iSCSI, any of the half-dozen others – has overhead also. That overhead ranges from not a lot to outrageous, and slows down the actual copying of data.

And perhaps the most painful of all, you can’t ship that data over the Internet in the clear. If you have permanently encrypted links, this is no problem, if you have something like our iSessions that create a secure tunnel, it’s not much of a problem either. You just have to know what you need encrypted, and have a way to get it moved.

So, in short, you need answers to all of these issues. TCP chattiness can be reduced by devices that essentially locally proxy unnecessary ACK sequences, packet loss and latency can not easily be handled, but there is some hope in a symmetric solution or (far less optimally) tweaking TCP settings, data volume can be handled in de-duplication… Though there are two flavors of dedupe, those used by storage vendors that may or may not require rehydration prior to transmission for backup/replication/snapshot, and those used by network devices to transmit less data. It is a much simpler task, technologically, to pull replicated bit patterns out of a TCP stream and send a key to rehydrate instead than it is to replace a block on-disk and store that key for replacement… Well, potentially forever. And the same types of things that TCP requires to reduce chattiness will work with protocols also if needed, they just have to be developed separately. And security is listed last because it stands outside the others. All of the above can be done through an encrypted tunnel easily enough. As long as all of the above is done before the encryption takes place ;-).


WHAT DO WE NEED?

You need a replication scheme that will provide timely backups, in a consistent manner, securely, with minimal impact to the machines being backed up. Whether you call your snapshots replicas or not, whether you still call it “nightly backup” or “replication”, the requirements are the same. Over the long haul solving the issues above will reduce your backup window, improve the integrity of your restore volumes, and generally allow you to sleep better at night.

While writing this blog, I read George Crump’s latest InformationWeek post where he mentions shifting the focus back to the backups and away from the restores. I agree, he’s hit the nail on the head. Make a solid backup, then worry about restoring it, for you’ll have nothing to restore without a workable, reliable, and timely backup/replica/snapshot.


Follow me on Twitter    icon_facebook

AddThis Feed Button Bookmark and Share

Read the original blog entry...

More Stories By Don MacVittie

Don MacVittie is founder of Ingrained Technology, A technical advocacy and software development consultancy. He has experience in application development, architecture, infrastructure, technical writing,DevOps, and IT management. MacVittie holds a B.S. in Computer Science from Northern Michigan University, and an M.S. in Computer Science from Nova Southeastern University.

Latest Stories
DevOps at Cloud Expo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 21st Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to w...
SYS-CON Events announced today that Massive Networks will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Massive Networks mission is simple. To help your business operate seamlessly with fast, reliable, and secure internet and network solutions. Improve your customer's experience with outstanding connections to your cloud.
In the enterprise today, connected IoT devices are everywhere – both inside and outside corporate environments. The need to identify, manage, control and secure a quickly growing web of connections and outside devices is making the already challenging task of security even more important, and onerous. In his session at @ThingsExpo, Rich Boyer, CISO and Chief Architect for Security at NTT i3, discussed new ways of thinking and the approaches needed to address the emerging challenges of security i...
"We want to show that our solution is far less expensive with a much better total cost of ownership so we announced several key features. One is called geo-distributed erasure coding, another is support for KVM and we introduced a new capability called Multi-Part," explained Tim Desai, Senior Product Marketing Manager at Hitachi Data Systems, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
There is a huge demand for responsive, real-time mobile and web experiences, but current architectural patterns do not easily accommodate applications that respond to events in real time. Common solutions using message queues or HTTP long-polling quickly lead to resiliency, scalability and development velocity challenges. In his session at 21st Cloud Expo, Ryland Degnan, a Senior Software Engineer on the Netflix Edge Platform team, will discuss how by leveraging a reactive stream-based protocol,...
FinTechs use the cloud to operate at the speed and scale of digital financial activity, but are often hindered by the complexity of managing security and compliance in the cloud. In his session at 20th Cloud Expo, Sesh Murthy, co-founder and CTO of Cloud Raxak, showed how proactive and automated cloud security enables FinTechs to leverage the cloud to achieve their business goals. Through business-driven cloud security, FinTechs can speed time-to-market, diminish risk and costs, maintain continu...
DX World EXPO, LLC., a Lighthouse Point, Florida-based startup trade show producer and the creator of "DXWorldEXPO® - Digital Transformation Conference & Expo" has announced its executive management team. The team is headed by Levent Selamoglu, who has been named CEO. "Now is the time for a truly global DX event, to bring together the leading minds from the technology world in a conversation about Digital Transformation," he said in making the announcement.
In his session at 20th Cloud Expo, Mike Johnston, an infrastructure engineer at Supergiant.io, discussed how to use Kubernetes to set up a SaaS infrastructure for your business. Mike Johnston is an infrastructure engineer at Supergiant.io with over 12 years of experience designing, deploying, and maintaining server and workstation infrastructure at all scales. He has experience with brick and mortar data centers as well as cloud providers like Digital Ocean, Amazon Web Services, and Rackspace. H...
Internet of @ThingsExpo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 21st Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The Internet of Things (IoT) is the most profound change in personal and enterprise IT since the creation of the Worldwide Web more than 20 years ago. All major researchers estimate there will be tens of billions devic...
"The Striim platform is a full end-to-end streaming integration and analytics platform that is middleware that covers a lot of different use cases," explained Steve Wilkes, Founder and CTO at Striim, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
Everything run by electricity will eventually be connected to the Internet. Get ahead of the Internet of Things revolution and join Akvelon expert and IoT industry leader, Sergey Grebnov, in his session at @ThingsExpo, for an educational dive into the world of managing your home, workplace and all the devices they contain with the power of machine-based AI and intelligent Bot services for a completely streamlined experience.
With tough new regulations coming to Europe on data privacy in May 2018, Calligo will explain why in reality the effect is global and transforms how you consider critical data. EU GDPR fundamentally rewrites the rules for cloud, Big Data and IoT. In his session at 21st Cloud Expo, Adam Ryan, Vice President and General Manager EMEA at Calligo, will examine the regulations and provide insight on how it affects technology, challenges the established rules and will usher in new levels of diligence...
SYS-CON Events announced today that Calligo, an innovative cloud service provider offering mid-sized companies the highest levels of data privacy and security, has been named "Bronze Sponsor" of SYS-CON's 21st International Cloud Expo ®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Calligo offers unparalleled application performance guarantees, commercial flexibility and a personalised support service from its globally located cloud plat...
SYS-CON Events announced today that Calligo has been named “Bronze Sponsor” of SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Calligo is an innovative cloud service provider offering mid-sized companies the highest levels of data privacy. Calligo offers unparalleled application performance guarantees, commercial flexibility and a personalized support service from its globally located cloud platfor...
What sort of WebRTC based applications can we expect to see over the next year and beyond? One way to predict development trends is to see what sorts of applications startups are building. In his session at @ThingsExpo, Arin Sime, founder of WebRTC.ventures, discussed the current and likely future trends in WebRTC application development based on real requests for custom applications from real customers, as well as other public sources of information.