Welcome!

Related Topics: @CloudExpo, Microservices Expo, Containers Expo Blog

@CloudExpo: Article

Do-It-Yourself Cloud Computing Management – Is It Worth It?

Do all the math, not just some of it

Does your IT or devops team run a lot of do-it-yourself projects? DIY is tempting, isn't it? You have smart engineers, easy access to storage and computing power, and a mandate this year to start moving some of (or more of) your applications to the cloud. Why not do it yourself?

I'm not saying that you shouldn't try DIY cloud computing management, but if you're going that route, you should know what you're in for. In particular, you should keep a sharp lookout for the point of no (easy) return.

The Five Cs of DIY
First of all, what compels people to try to roll their own end-to-end cloud management strategy? There are five arguments we often hear for DIY (all of which begin with "C"), depicted in Figure 1.

  1. Control: When you do it yourself, you're able to control everything, right, wrong or indifferent. If there are problems, you can handle them internally. If your business requires custom scripts or exporting usage data into your own warehouse, you'll want to be able flip the levers and configure it yourself. And the images will be yours, all yours.
  2. Complexity: "What's to manage?" you ask. "We're just spinning up a dozen or so dev and stage servers in an Amazon EC2 public cloud to get our feet wet. Plenty of companies do this without all the fuss of additional management tools." When your initial objectives are modest and you want your DevOps team to learn how things work first, cloud computing management looks like just a nice-to-have.
  3. Conquest: Speaking of DevOps, plenty of them want the challenge of understanding what's possible and how to build it. Or some of them say, "If I can't get approval, I'll just do it myself on the side," and they jump in. Because we are still in the early stages of cloud uptake, your engineers and managers are experimenting to discover just how far DIY and cobbling different tool sets together will take them, as opposed to researching what is available in the market. Who can blame them? The cloud presents itself as one of the most powerful APIs ever, and what self-respecting developer or IT department can resist that?
  4. Cost: Everybody's eye is on cost. "For about $200 a month, I can get five large, on-demand Linux instances for 100 hours, 50GB of storage and 10GB of data in and out. With pricing like that, we'll figure out the management piece ourselves. Most of what we'd buy in a cloud management product is stuff we can do ourselves anyway." It certainly looks like that at first glance, and if the budget is tight and your cloud-based DIY deployment is humming along, you probably can't make the case for spending money on management expertise that you're convinced you can provide on your own.
  5. Convenience: It's convenient and easy to think about DIY cloud management because Amazon, for instance, gives you so many options and tools to start with. If there were many long-standing public clouds from which to choose, the water would be muddier, you'd use a sharper pencil in the vetting process, and doing it yourself would seem like less of a slam dunk.

People stack these five Cs against off-the-shelf cloud management offerings all the time. Our industry spends a lot of time hearing these arguments, patiently nodding our heads and repeating the counter-arguments, based on our own experience with customers:

If you really need control, you'd be surprised by how much control and customization you can have with a cloud management product, even when you start with pre-configured images and templates.

Cloud computing management is complex, but cloud management products are designed to shield you from most of the complexity. Besides, the sooner you see how simple it is to automate the management of five or ten servers, the sooner you can get up to the 50 or 100 your business really needs.

There's plenty of technology to conquer in cloud computing management, but the industry has already conquered most of it, which is why off-the-shelf products are so comprehensive and accessible.

The cost of DIY is usually a lot more than your monthly fees. There's recruiting, training, non-recurring engineering expenses, headcount and the risk involved in building and maintaining your team. People don't always think that far down the road.

The flip-side of convenience is lock-in. It's hard to resist the ease of spinning up a cloud with just a browser and a credit card; however, is the product you conveniently start with today going to grow and scale gracefully with your business? Do you want all of your eggs in one basket? You've got to ask yourself, "Do I feel lucky?"

Let's examine the technology more closely.

Eyes Wide Open: Know What You're in For
What does cloud computing management mean? How many different layers are there to it, and when will you hit each one? Figure 2 depicts the layers of cloud computing management you'll traverse eventually, whether you DIY or license a product.

Basic cloud offerings from Amazon and open source tools can cover the entry-level echelon of service (bottom of Figure 2). A combination of DIY and open source is not especially dangerous at these levels:

  1. You provision basic or pre-configured images to meet your specifications and the needs of your users and customers, with memory, computing power, storage, OS and geographic proximity.
  2. Once you've installed applications in your cloud, you'll want to monitor them. Are they running properly? Have any of them gone down? Can you get alerts if something goes wrong?
  3. After you've tweaked your images, you'll want to clone them, say, for development/staging/production, for multiple developers or to meet increasing demand and traffic.
    Once you've gotten this far, you can also try products for cloud auto-scaling (originally invented by RightScale). With a few months and several servers under your belt, you arrive at the advanced echelon of service in need of yet another tool, because entry-level products don't cover these (middle of Figure 2):
  4. It's easy to take snapshots of images; sometimes too easy. You forget why you created them, what's inside them and whether anybody is still using them. You need configuration management to deal with image sprawl. Once you have that under control, you also need configuration management to create repeatable applications and services (for your own mini Platforms-as-a-Service).

With people all over the organization clamoring for cloud-based apps, you're ready for user management to set permissions and audit activity.

The cloud may be inexpensive, but it isn't free. When Finance asks you about cost allocation and ROI by department, project and region, you need tools that can break out your expenses and revenues from activity in the cloud.

The last three items are the state of the art in cloud computing management, and if you can get that far on DIY, you deserve a big raise. On the horizon is a final layer, still in its infancy (top of Figure 2).

Most of your cloud assets need to work together; e.g., start the database server first, then start up the application tier, then start the proxy, run some tests to make sure the whole app is working, and then turn on the website. Orchestration and workflow automation will soon allow you to code how your system should operate so that you don't need to intervene.

That's the long view of cloud computing management. Set your expectations accordingly.

Are You Close to the Tipping Point?
The market is torn at the moment. On the DIY side, cloud computing is not very old, and lots of organizations are scrambling to figure out what it can do for them and their business. There are plenty of eager, curious engineers ready to dive in, fire up a few servers, cut their teeth on cloud computing and a few open source tools, and do their company (and their résumé) some good.

On the off-the-shelf side, the market is filled with entrants and it's growing up fast. Cloud management products are feature-rich because those of us who focus on them have already hit most of the roadblocks.

For some companies, the choice of off-the-shelf is obvious. They've looked at Figure 2 and decided that they don't want to have to do it themselves. They tried rolling their own CRM until they saw how effortless Salesforce.com made it, and they remember tinkering for ages with their own Web servers until they realized Apache had nailed it. They've learned that lesson.

What's the tipping point for everybody else? When do features, capabilities and price point tip in favor of off-the-shelf products? From our experience, here are three telling metrics:

  1. Forty images - Once people try to keep track of more than 40 images, DIY cloud management tools start to get creaky. "I see a whole slew of servers, and some have really short names...I've forgotten what that one does..."
  2. Fifteen users - If they have 15 people operating in the cloud after only a couple of months, they're liable to be at 200 in a year. User management tools need to scale and work with all the other DIY tools in use.
  3. Five accounts - To sort out their billing and ROI headaches, they write discovery apps or go through the Amazon API trying to figure out which instances are in use and which images they come from so they can allocate costs correctly.

Alas, some organizations stick with DIY past that tipping point, to the point of no (easy) return. They've made it well into the advanced echelon, but only by cobbling together a patchwork of three to four disparate tool sets and growing their DevOps teams to 50 or more. Or, perhaps they decide to move all their DIY stuff to a different public cloud provider, or to their own private cloud. An off-the-shelf cloud computing management can still help them when they get to these points, but the effort will cost much more time and money than if they had started there in the first place.

Do All the Math, Not Just Some of It
Many customers come to us after they've outgrown their DIY efforts. Eventually, they discover that there are too many things to stitch together: configuration management, systems automation, monitoring, application automation, provisioning, user permissions, reporting and more.

Even if you're happy with the DIY cloud computing management you've put in place, are you really sure that it's worth the investment in time, money and manpower, compared to an off-the-shelf offering? Not only that, but are you sure you're far enough away from the tipping point that DIY will still look good a year from now?


Cloud Management - Obstacles Overcome in Off-the-Shelf Products

  • Removal of a single user's SSH key from all managed instances
  • Volume striping for better database performance
  • Image fingerprinting to identify pre-rolled images in private clouds accurately

More Stories By Darryl Eaton

Darryl Eaton is the director of product management at RightScale Inc. Not only does he ship products that are ahead of their time and blog about it, but he has also been known to play the Carillon, the largest and loudest musical instrument in the world. He wants you to make the right decision about DIY and off-the-shelf, so contact him at [email protected] and find out more about cloud management and the Five Cloud Commandments in RightScale’s library.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


Latest Stories
Your homes and cars can be automated and self-serviced. Why can't your storage? From simply asking questions to analyze and troubleshoot your infrastructure, to provisioning storage with snapshots, recovery and replication, your wildest sci-fi dream has come true. In his session at @DevOpsSummit at 20th Cloud Expo, Dan Florea, Director of Product Management at Tintri, will provide a ChatOps demo where you can talk to your storage and manage it from anywhere, through Slack and similar services ...
SYS-CON Events announced today that Ocean9will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Ocean9 provides cloud services for Backup, Disaster Recovery (DRaaS) and instant Innovation, and redefines enterprise infrastructure with its cloud native subscription offerings for mission critical SAP workloads.
The taxi industry never saw Uber coming. Startups are a threat to incumbents like never before, and a major enabler for startups is that they are instantly “cloud ready.” If innovation moves at the pace of IT, then your company is in trouble. Why? Because your data center will not keep up with frenetic pace AWS, Microsoft and Google are rolling out new capabilities In his session at 20th Cloud Expo, Don Browning, VP of Cloud Architecture at Turner, will posit that disruption is inevitable for c...
Building a cross-cloud operational model can be a daunting task. Per-cloud silos are not the answer, but neither is a fully generic abstraction plane that strips out capabilities unique to a particular provider. In his session at 20th Cloud Expo, Chris Wolf, VP & Chief Technology Officer, Global Field & Industry at VMware, will discuss how successful organizations approach cloud operations and management, with insights into where operations should be centralized and when it’s best to decentraliz...
Providing the needed data for application development and testing is a huge headache for most organizations. The problems are often the same across companies - speed, quality, cost, and control. Provisioning data can take days or weeks, every time a refresh is required. Using dummy data leads to quality problems. Creating physical copies of large data sets and sending them to distributed teams of developers eats up expensive storage and bandwidth resources. And, all of these copies proliferating...
DevOps is often described as a combination of technology and culture. Without both, DevOps isn't complete. However, applying the culture to outdated technology is a recipe for disaster; as response times grow and connections between teams are delayed by technology, the culture will die. A Nutanix Enterprise Cloud has many benefits that provide the needed base for a true DevOps paradigm. In his Day 3 Keynote at 20th Cloud Expo, Chris Brown, a Solutions Marketing Manager at Nutanix, will explore t...
DevOps has often been described in terms of CAMS: Culture, Automation, Measuring, Sharing. While we’ve seen a lot of focus on the “A” and even on the “M”, there are very few examples of why the “C" is equally important in the DevOps equation. In her session at @DevOps Summit, Lori MacVittie, of F5 Networks, explored HTTP/1 and HTTP/2 along with Microservices to illustrate why a collaborative culture between Dev, Ops, and the Network is critical to ensuring success.
Interoute has announced the integration of its Global Cloud Infrastructure platform with Rancher Labs’ container management platform, Rancher. This approach enables enterprises to accelerate their digital transformation and infrastructure investments. Matthew Finnie, Interoute CTO commented “Enterprises developing and building apps in the cloud and those on a path to Digital Transformation need Digital ICT Infrastructure that allows them to build, test and deploy faster than ever before. The int...
SYS-CON Events announced today that SoftLayer, an IBM Company, has been named “Gold Sponsor” of SYS-CON's 18th Cloud Expo, which will take place on June 7-9, 2016, at the Javits Center in New York, New York. SoftLayer, an IBM Company, provides cloud infrastructure as a service from a growing number of data centers and network points of presence around the world. SoftLayer’s customers range from Web startups to global enterprises.
SYS-CON Events announced today that Conference Guru has been named “Media Sponsor” of SYS-CON's 20th International Cloud Expo, which will take place on June 6–8, 2017, at the Javits Center in New York City, NY. A valuable conference experience generates new contacts, sales leads, potential strategic partners and potential investors; helps gather competitive intelligence and even provides inspiration for new products and services. Conference Guru works with conference organizers to pass great dea...
SYS-CON Events announced today that Technologic Systems Inc., an embedded systems solutions company, will exhibit at SYS-CON's @ThingsExpo, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Technologic Systems is an embedded systems company with headquarters in Fountain Hills, Arizona. They have been in business for 32 years, helping more than 8,000 OEM customers and building over a hundred COTS products that have never been discontinued. Technologic Systems’ pr...
SYS-CON Events announced today that CA Technologies has been named “Platinum Sponsor” of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY, and the 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. CA Technologies helps customers succeed in a future where every business – from apparel to energy – is being rewritten by software. From ...
With major technology companies and startups seriously embracing Cloud strategies, now is the perfect time to attend @CloudExpo | @ThingsExpo, June 6-8, 2017, at the Javits Center in New York City, NY and October 31 - November 2, 2017, Santa Clara Convention Center, CA. Learn what is going on, contribute to the discussions, and ensure that your enterprise is on the right path to Digital Transformation.
Deep learning has been very successful in social sciences and specially areas where there is a lot of data. Trading is another field that can be viewed as social science with a lot of data. With the advent of Deep Learning and Big Data technologies for efficient computation, we are finally able to use the same methods in investment management as we would in face recognition or in making chat-bots. In his session at 20th Cloud Expo, Gaurav Chakravorty, co-founder and Head of Strategy Development ...
SYS-CON Events announced today that Telecom Reseller has been named “Media Sponsor” of SYS-CON's 20th International Cloud Expo, which will take place on June 6–8, 2017, at the Javits Center in New York City, NY. Telecom Reseller reports on Unified Communications, UCaaS, BPaaS for enterprise and SMBs. They report extensively on both customer premises based solutions such as IP-PBX as well as cloud based and hosted platforms.