|By Lori MacVittie||
|January 3, 2017 11:45 PM EST||
I've been reading up on APIs cause, coolness. And in particular I really enjoyed reading Best Practices for Designing a Pragmatic RESTful API because it had a lot of really good information and advice.
And then I got to the part about compressing your APIs.
Before we go too far let me first say I'm not saying you shouldn't compress your API or app responses. You probably should. What I am saying is that where you compress data and when are important considerations.
That's because generally speaking no one has put their web server (which is ultimately what tends to serve up responses, whether they're APIs or objects, XML or JSON) at the edge of the Internet. You know, where it's completely vulnerable. It's usually several devices back in the networking gauntlet that has be run before data gets from the edge of your network to the server.
This is because there are myriad bad actors out salivating at the prospect of a return to an early aughts data center architecture in which firewalls, DDoS protection, and other app security services were not physically and logically located upstream from the apps they protect today.
Cause if you don't have to navigate the network, it's way easier to launch an attack on an app.
Today, we employ an average of 11 different services in the network, upstream from the app, to provide security, scale, and performance-enhancing services. Like compression.
Now, you can enable compression on the web server. It's a standard thing in HTTP and it's little more than a bit to flip in the configuration. Easy peasy performance-enhancing change, right?
Except that today that's not always true.
The primary reason compression improves performance is because when it reduces the size of data it reduces the number of packets that must be transmitted. That reduces the potential for congestion that causes a Catch-22 where TCP retransmits increase congestion that increases packet loss that increases... well, you get the picture. This is particularly true when mobile clients are connecting via cellular networks, because latency is a real issue for them and the more round trips it takes, the worse the application experience.
Suffice to say that the primary reason compression improves performance is that it reduces the amount of data needing to be transmitted which means "faster" delivery to the client. Fewer packets = less time = happier users.
That's a good thing. Except when compression gets in the way or doesn't provide any real reduction that would improve performance.
What? How can that be, you ask.
Remember that we're looking for compression to reduce the number of packets transmitted, especially when it has to traverse a higher latency, lower capacity link between the data center and the client.
It turns out that sometimes compression doesn't really help with that.
Consider the aforementioned article and its section on compressing. The author ran some tests, and concluded that compression of text-based data produces some really awesome results:
Let's look at this with a real world example. I've pulled some data from GitHub's API, which uses pretty print by default. I'll also be doing some gzip comparisons:
$ curl https://api.github.com/users/veesahni > with-whitespace.txt $ ruby -r json -e 'puts JSON JSON.parse(STDIN.read)' < with-whitespace.txt > without-whitespace.txt
$ gzip -c with-whitespace.txt > with-whitespace.txt.gz
$ gzip -c without-whitespace.txt > without-whitespace.txt.gz
The output files have the following sizes:
- without-whitespace.txt - 1252 bytes
- with-whitespace.txt - 1369 bytes
- without-whitespace.txt.gz - 496 bytes
- with-whitespace.txt.gz - 509 bytes
In this example, the whitespace increased the output size by 8.5% when gzip is not in play and 2.6% when gzip is in play. On the other hand, the act of gzipping in itself provided over 60% in bandwidth savings. Since the cost of pretty printing is relatively small, it's best to pretty print by default and ensure gzip compression is supported!
To further hammer in this point, Twitter found that there was an 80% savings (in some cases)when enabling gzip compression on their Streaming API. Stack Exchange went as far as to never return a response that's not compressed!
Wow! I mean, from a purely mathematical perspective, that's some awesome results. And the author is correct in saying it will provide bandwidth savings.
What those results won't necessarily do is improve performance because the original size of the file was already less than the MSS for a single packet. Which means compressed or not, that data takes exactly one packet to transmit. That's it. I won't bore you with the mathematics, but the speed of light and networking says one packet takes the same amount of time to transit whether it's got 496 bytes of payload or 1396 bytes of payload. The typical MSS for Ethernet packets is 1460 bytes, which means compressing something smaller than that effectively nets you nothing in terms of performance. It's like a plane. It takes as long to fly from point A to point B whether there are 14 passengers or 140. Fuel efficiency (bandwidth) is impacted, but that doesn't really change performance, just the cost.
Furthermore, compressing the payload at the web server means that web app security services upstream have to decompress if they want to do their job, which is to say scan responses for sensitive or excessive data indicative of a breach of security policies. This is a big deal, kids. 42% of respondents in our annual security strategy to prevent data leaks. Which means they have to spend extra time to decompress the data to evaluate it and then recompress it, or perhaps they can't inspect it at all.
State of Application Delivery survey always scan responses as part of their overall
Now, that said, bandwidth savings are a good thing. It's part of any comprehensive scaling strategy to consider the impact of increasing use of an app on bandwidth. And a clogged up network can impact performance negatively so compression is a good idea. But not necessarily at the web server. This is akin to carefully considering where you enforce SSL/TLS security measures, as there are similar impacts on security services upstream from the app / web server.
That's why the right place for compression and SSL/TLS is generally upstream, in the network, after security has checked out the response and it's actually ready to be delivered to the client. That's usually the load balancing service or the ADC, where compression can not only be applied most efficiently and without interfering with security services and offsetting the potential gains by forcing extra processing upstream.
As with rate limiting APIs, it's not always a matter of whether or not you should, it's a matter of where you should.
Architecture, not algorithms, are the key to scale and performance of modern applications.
Feb. 27, 2017 01:00 PM EST Reads: 1,353
Feb. 27, 2017 12:47 PM EST
Feb. 27, 2017 12:45 PM EST Reads: 2,079
Feb. 27, 2017 12:45 PM EST Reads: 405
LogMeIn has completed its previously disclosed merger with Citrix Systems, Inc.’s GetGo, Inc. subsidiary, a wholly owned subsidiary consisting of Citrix’s GoTo family of service offerings. The merger officially closed after market hours on January 31, 2017. Effected through a Reverse Morris Trust transaction, the merger brings together two of the preeminent players in cloud connectivity to instantly create one of the world’s top 10 public SaaS companies, and a market leader with the scale, resou...
Feb. 27, 2017 12:41 PM EST Reads: 105
For basic one-to-one voice or video calling solutions, WebRTC has proven to be a very powerful technology. Although WebRTC’s core functionality is to provide secure, real-time p2p media streaming, leveraging native platform features and server-side components brings up new communication capabilities for web and native mobile applications, allowing for advanced multi-user use cases such as video broadcasting, conferencing, and media recording.
Feb. 27, 2017 12:30 PM EST Reads: 6,782
DevOps is often described as a combination of technology and culture. Without both, DevOps isn't complete. However, applying the culture to outdated technology is a recipe for disaster; as response times grow and connections between teams are delayed by technology, the culture will die. A Nutanix Enterprise Cloud has many benefits that provide the needed base for a true DevOps paradigm.
Feb. 27, 2017 12:30 PM EST Reads: 470
Addteq is one of the top 10 Platinum Atlassian Experts who specialize in DevOps, custom and continuous integration, automation, plugin development, and consulting for midsize and global firms. Addteq firmly believes that automation is essential for successful software releases. Addteq centers its products and services around this fundamentally unique approach to delivering complete software release management solutions. With a combination of Addteq's services and our extensive list of partners,...
Feb. 27, 2017 12:00 PM EST Reads: 1,195
SYS-CON Events announced today that Outlyer, a monitoring service for DevOps and operations teams, has been named “Bronze Sponsor” of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Outlyer is a monitoring service for DevOps and Operations teams running Cloud, SaaS, Microservices and IoT deployments. Designed for today's dynamic environments that need beyond cloud-scale monitoring, we make monitoring effortless so you...
Feb. 27, 2017 11:45 AM EST Reads: 2,509
Feb. 27, 2017 11:45 AM EST Reads: 289
While not quite mainstream yet, WebRTC is starting to gain ground with Carriers, Enterprises and Independent Software Vendors (ISV’s) alike. WebRTC makes it easy for developers to add audio and video communications into their applications by using Web browsers as their platform. But like any market, every customer engagement has unique requirements, as well as constraints. And of course, one size does not fit all. In her session at WebRTC Summit, Dr. Natasha Tamaskar, Vice President, Head of C...
Feb. 27, 2017 11:45 AM EST Reads: 7,141
Cloud Expo, Inc. has announced today that Andi Mann and Aruna Ravichandran have been named Co-Chairs of @DevOpsSummit at Cloud Expo 2017. The @DevOpsSummit at Cloud Expo New York will take place on June 6-8, 2017, at the Javits Center in New York City, New York, and @DevOpsSummit at Cloud Expo Silicon Valley will take place Oct. 31-Nov. 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
Feb. 27, 2017 11:30 AM EST Reads: 2,518
SYS-CON Events announced today that CA Technologies has been named “Platinum Sponsor” of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY, and the 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. CA Technologies helps customers succeed in a future where every business – from apparel to energy – is being rewritten by software. From ...
Feb. 27, 2017 11:15 AM EST Reads: 2,822
Feb. 27, 2017 11:15 AM EST Reads: 403
SYS-CON Events announced today that CrowdReviews.com has been named “Media Sponsor” of SYS-CON's 20th International Cloud Expo, which will take place on June 6–8, 2017, at the Javits Center in New York City, NY. CrowdReviews.com is a transparent online platform for determining which products and services are the best based on the opinion of the crowd. The crowd consists of Internet users that have experienced products and services first-hand and have an interest in letting other potential buyers...
Feb. 27, 2017 11:00 AM EST Reads: 1,910