Welcome!

Related Topics: @CloudExpo, Microservices Expo, Containers Expo Blog

@CloudExpo: Article

Do-It-Yourself Cloud Computing Management – Is It Worth It?

Do all the math, not just some of it

Does your IT or devops team run a lot of do-it-yourself projects? DIY is tempting, isn't it? You have smart engineers, easy access to storage and computing power, and a mandate this year to start moving some of (or more of) your applications to the cloud. Why not do it yourself?

I'm not saying that you shouldn't try DIY cloud computing management, but if you're going that route, you should know what you're in for. In particular, you should keep a sharp lookout for the point of no (easy) return.

The Five Cs of DIY
First of all, what compels people to try to roll their own end-to-end cloud management strategy? There are five arguments we often hear for DIY (all of which begin with "C"), depicted in Figure 1.

  1. Control: When you do it yourself, you're able to control everything, right, wrong or indifferent. If there are problems, you can handle them internally. If your business requires custom scripts or exporting usage data into your own warehouse, you'll want to be able flip the levers and configure it yourself. And the images will be yours, all yours.
  2. Complexity: "What's to manage?" you ask. "We're just spinning up a dozen or so dev and stage servers in an Amazon EC2 public cloud to get our feet wet. Plenty of companies do this without all the fuss of additional management tools." When your initial objectives are modest and you want your DevOps team to learn how things work first, cloud computing management looks like just a nice-to-have.
  3. Conquest: Speaking of DevOps, plenty of them want the challenge of understanding what's possible and how to build it. Or some of them say, "If I can't get approval, I'll just do it myself on the side," and they jump in. Because we are still in the early stages of cloud uptake, your engineers and managers are experimenting to discover just how far DIY and cobbling different tool sets together will take them, as opposed to researching what is available in the market. Who can blame them? The cloud presents itself as one of the most powerful APIs ever, and what self-respecting developer or IT department can resist that?
  4. Cost: Everybody's eye is on cost. "For about $200 a month, I can get five large, on-demand Linux instances for 100 hours, 50GB of storage and 10GB of data in and out. With pricing like that, we'll figure out the management piece ourselves. Most of what we'd buy in a cloud management product is stuff we can do ourselves anyway." It certainly looks like that at first glance, and if the budget is tight and your cloud-based DIY deployment is humming along, you probably can't make the case for spending money on management expertise that you're convinced you can provide on your own.
  5. Convenience: It's convenient and easy to think about DIY cloud management because Amazon, for instance, gives you so many options and tools to start with. If there were many long-standing public clouds from which to choose, the water would be muddier, you'd use a sharper pencil in the vetting process, and doing it yourself would seem like less of a slam dunk.

People stack these five Cs against off-the-shelf cloud management offerings all the time. Our industry spends a lot of time hearing these arguments, patiently nodding our heads and repeating the counter-arguments, based on our own experience with customers:

If you really need control, you'd be surprised by how much control and customization you can have with a cloud management product, even when you start with pre-configured images and templates.

Cloud computing management is complex, but cloud management products are designed to shield you from most of the complexity. Besides, the sooner you see how simple it is to automate the management of five or ten servers, the sooner you can get up to the 50 or 100 your business really needs.

There's plenty of technology to conquer in cloud computing management, but the industry has already conquered most of it, which is why off-the-shelf products are so comprehensive and accessible.

The cost of DIY is usually a lot more than your monthly fees. There's recruiting, training, non-recurring engineering expenses, headcount and the risk involved in building and maintaining your team. People don't always think that far down the road.

The flip-side of convenience is lock-in. It's hard to resist the ease of spinning up a cloud with just a browser and a credit card; however, is the product you conveniently start with today going to grow and scale gracefully with your business? Do you want all of your eggs in one basket? You've got to ask yourself, "Do I feel lucky?"

Let's examine the technology more closely.

Eyes Wide Open: Know What You're in For
What does cloud computing management mean? How many different layers are there to it, and when will you hit each one? Figure 2 depicts the layers of cloud computing management you'll traverse eventually, whether you DIY or license a product.

Basic cloud offerings from Amazon and open source tools can cover the entry-level echelon of service (bottom of Figure 2). A combination of DIY and open source is not especially dangerous at these levels:

  1. You provision basic or pre-configured images to meet your specifications and the needs of your users and customers, with memory, computing power, storage, OS and geographic proximity.
  2. Once you've installed applications in your cloud, you'll want to monitor them. Are they running properly? Have any of them gone down? Can you get alerts if something goes wrong?
  3. After you've tweaked your images, you'll want to clone them, say, for development/staging/production, for multiple developers or to meet increasing demand and traffic.
    Once you've gotten this far, you can also try products for cloud auto-scaling (originally invented by RightScale). With a few months and several servers under your belt, you arrive at the advanced echelon of service in need of yet another tool, because entry-level products don't cover these (middle of Figure 2):
  4. It's easy to take snapshots of images; sometimes too easy. You forget why you created them, what's inside them and whether anybody is still using them. You need configuration management to deal with image sprawl. Once you have that under control, you also need configuration management to create repeatable applications and services (for your own mini Platforms-as-a-Service).

With people all over the organization clamoring for cloud-based apps, you're ready for user management to set permissions and audit activity.

The cloud may be inexpensive, but it isn't free. When Finance asks you about cost allocation and ROI by department, project and region, you need tools that can break out your expenses and revenues from activity in the cloud.

The last three items are the state of the art in cloud computing management, and if you can get that far on DIY, you deserve a big raise. On the horizon is a final layer, still in its infancy (top of Figure 2).

Most of your cloud assets need to work together; e.g., start the database server first, then start up the application tier, then start the proxy, run some tests to make sure the whole app is working, and then turn on the website. Orchestration and workflow automation will soon allow you to code how your system should operate so that you don't need to intervene.

That's the long view of cloud computing management. Set your expectations accordingly.

Are You Close to the Tipping Point?
The market is torn at the moment. On the DIY side, cloud computing is not very old, and lots of organizations are scrambling to figure out what it can do for them and their business. There are plenty of eager, curious engineers ready to dive in, fire up a few servers, cut their teeth on cloud computing and a few open source tools, and do their company (and their résumé) some good.

On the off-the-shelf side, the market is filled with entrants and it's growing up fast. Cloud management products are feature-rich because those of us who focus on them have already hit most of the roadblocks.

For some companies, the choice of off-the-shelf is obvious. They've looked at Figure 2 and decided that they don't want to have to do it themselves. They tried rolling their own CRM until they saw how effortless Salesforce.com made it, and they remember tinkering for ages with their own Web servers until they realized Apache had nailed it. They've learned that lesson.

What's the tipping point for everybody else? When do features, capabilities and price point tip in favor of off-the-shelf products? From our experience, here are three telling metrics:

  1. Forty images - Once people try to keep track of more than 40 images, DIY cloud management tools start to get creaky. "I see a whole slew of servers, and some have really short names...I've forgotten what that one does..."
  2. Fifteen users - If they have 15 people operating in the cloud after only a couple of months, they're liable to be at 200 in a year. User management tools need to scale and work with all the other DIY tools in use.
  3. Five accounts - To sort out their billing and ROI headaches, they write discovery apps or go through the Amazon API trying to figure out which instances are in use and which images they come from so they can allocate costs correctly.

Alas, some organizations stick with DIY past that tipping point, to the point of no (easy) return. They've made it well into the advanced echelon, but only by cobbling together a patchwork of three to four disparate tool sets and growing their DevOps teams to 50 or more. Or, perhaps they decide to move all their DIY stuff to a different public cloud provider, or to their own private cloud. An off-the-shelf cloud computing management can still help them when they get to these points, but the effort will cost much more time and money than if they had started there in the first place.

Do All the Math, Not Just Some of It
Many customers come to us after they've outgrown their DIY efforts. Eventually, they discover that there are too many things to stitch together: configuration management, systems automation, monitoring, application automation, provisioning, user permissions, reporting and more.

Even if you're happy with the DIY cloud computing management you've put in place, are you really sure that it's worth the investment in time, money and manpower, compared to an off-the-shelf offering? Not only that, but are you sure you're far enough away from the tipping point that DIY will still look good a year from now?


Cloud Management - Obstacles Overcome in Off-the-Shelf Products

  • Removal of a single user's SSH key from all managed instances
  • Volume striping for better database performance
  • Image fingerprinting to identify pre-rolled images in private clouds accurately

More Stories By Darryl Eaton

Darryl Eaton is the director of product management at RightScale Inc. Not only does he ship products that are ahead of their time and blog about it, but he has also been known to play the Carillon, the largest and loudest musical instrument in the world. He wants you to make the right decision about DIY and off-the-shelf, so contact him at [email protected] and find out more about cloud management and the Five Cloud Commandments in RightScale’s library.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


Latest Stories
WebRTC is bringing significant change to the communications landscape that will bridge the worlds of web and telephony, making the Internet the new standard for communications. Cloud9 took the road less traveled and used WebRTC to create a downloadable enterprise-grade communications platform that is changing the communication dynamic in the financial sector. In his session at @ThingsExpo, Leo Papadopoulos, CTO of Cloud9, discussed the importance of WebRTC and how it enables companies to focus...
DevOps at Cloud Expo – being held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA – announces that its Call for Papers is open. Born out of proven success in agile development, cloud computing, and process automation, DevOps is a macro trend you cannot afford to miss. From showcase success stories from early adopters and web-scale businesses, DevOps is expanding to organizations of all sizes, including the world's largest enterprises – and delivering real results. Am...
There is little doubt that Big Data solutions will have an increasing role in the Enterprise IT mainstream over time. Big Data at Cloud Expo - to be held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA - has announced its Call for Papers is open. Cloud computing is being adopted in one form or another by 94% of enterprises today. Tens of billions of new devices are being connected to The Internet of Things. And Big Data is driving this bus. An exponential increase is...
Big Data has been changing the world. IoT fuels the further transformation recently. How are Big Data and IoT related? In his session at @BigDataExpo, Tony Shan, a renowned visionary and thought leader, will explore the interplay of Big Data and IoT. He will anatomize Big Data and IoT separately in terms of what, which, why, where, when, who, how and how much. He will then analyze the relationship between IoT and Big Data, specifically the drilldown of how the 4Vs of Big Data (Volume, Variety,...
19th Cloud Expo, taking place November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy. Meanwhile, 94% of enterpri...
Cloud computing is being adopted in one form or another by 94% of enterprises today. Tens of billions of new devices are being connected to The Internet of Things. And Big Data is driving this bus. An exponential increase is expected in the amount of information being processed, managed, analyzed, and acted upon by enterprise IT. This amazing is not part of some distant future - it is happening today. One report shows a 650% increase in enterprise data by 2020. Other estimates are even higher....
The Internet of Things can drive efficiency for airlines and airports. In their session at @ThingsExpo, Shyam Varan Nath, Principal Architect with GE, and Sudip Majumder, senior director of development at Oracle, will discuss the technical details of the connected airline baggage and related social media solutions. These IoT applications will enhance travelers' journey experience and drive efficiency for the airlines and the airports. The session will include a working demo and a technical d...
For basic one-to-one voice or video calling solutions, WebRTC has proven to be a very powerful technology. Although WebRTC’s core functionality is to provide secure, real-time p2p media streaming, leveraging native platform features and server-side components brings up new communication capabilities for web and native mobile applications, allowing for advanced multi-user use cases such as video broadcasting, conferencing, and media recording.
Adobe is changing the world though digital experiences. Adobe helps customers develop and deliver high-impact experiences that differentiate brands, build loyalty, and drive revenue across every screen, including smartphones, computers, tablets and TVs. Adobe content solutions are used daily by millions of companies worldwide-from publishers and broadcasters, to enterprises, marketing agencies and household-name brands. Building on its established design leadership, Adobe enables customers not o...
Creating replica copies to tolerate a certain number of failures is easy, but very expensive at cloud-scale. Conventional RAID has lower overhead, but it is limited in the number of failures it can tolerate. And the management is like herding cats (overseeing capacity, rebuilds, migrations, and degraded performance). Download Slide Deck: ▸ Here In his general session at 18th Cloud Expo, Scott Cleland, Senior Director of Product Marketing for the HGST Cloud Infrastructure Business Unit, discusse...
SYS-CON Events announced today that Interface Masters Technologies, a leader in Network Visibility and Uptime Solutions, will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Interface Masters Technologies is a leading vendor in the network monitoring and high speed networking markets. Based in the heart of Silicon Valley, Interface Masters' expertise lies in Gigabit, 10 Gigabit and 40 Gigabit Eth...
SYS-CON Events announced today the Enterprise IoT Bootcamp, being held November 1-2, 2016, in conjunction with 19th Cloud Expo | @ThingsExpo at the Santa Clara Convention Center in Santa Clara, CA. Combined with real-world scenarios and use cases, the Enterprise IoT Bootcamp is not just based on presentations but with hands-on demos and detailed walkthroughs. We will introduce you to a variety of real world use cases prototyped using Arduino, Raspberry Pi, BeagleBone, Spark, and Intel Edison. Y...
What does it look like when you have access to cloud infrastructure and platform under the same roof? Let’s talk about the different layers of Technology as a Service: who cares, what runs where, and how does it all fit together. In his session at 18th Cloud Expo, Phil Jackson, Lead Technology Evangelist at SoftLayer, an IBM company, spoke about the picture being painted by IBM Cloud and how the tools being crafted can help fill the gaps in your IT infrastructure.
Much of the value of DevOps comes from a (renewed) focus on measurement, sharing, and continuous feedback loops. In increasingly complex DevOps workflows and environments, and especially in larger, regulated, or more crystallized organizations, these core concepts become even more critical. In his session at @DevOpsSummit at 18th Cloud Expo, Andi Mann, Chief Technology Advocate at Splunk, showed how, by focusing on 'metrics that matter,' you can provide objective, transparent, and meaningful f...
Why do your mobile transformations need to happen today? Mobile is the strategy that enterprise transformation centers on to drive customer engagement. In his general session at @ThingsExpo, Roger Woods, Director, Mobile Product & Strategy – Adobe Marketing Cloud, covered key IoT and mobile trends that are forcing mobile transformation, key components of a solid mobile strategy and explored how brands are effectively driving mobile change throughout the enterprise.