Welcome!

Blog Feed Post

Vyopta: Shipping fast and reducing operational costs with MongoDB Atlas

This is a guest post from Brent Mills, Principal Infrastructure Engineer at Vyopta, an analytics company that enterprises use to monitor, support, and optimize their video and unified collaboration systems.

MongoDB has always been very attractive to developers in part due to the ease with which you can set up, manage, and interact with the database. Time-to-market is increasingly important, as businesses require increasing levels of speed and flexibility in how they deliver new products and features to their customers.

But as with any other back-end system, you have to manage monitoring and alerting, the clusters (and all of the idiosyncrasies of every distributed system), backups, encryption, provisioning for scale, etc.; These are all things operations teams have to consider in order to deliver production-ready applications, but they’re also things that can delay innovation velocity and can present operational overhead.

By moving our database to the MongoDB cloud service, we’ve been able to remove much of the operational load, saving us over 15 hours of work per month, enabling us to increase efficiency as we’ve moved apps from development to production.

Our Use Case

Below is a simplified version of our pre-MongoDB Atlas setup. I would imagine this looks familiar to many of you:

First off, there’s nothing wrong with this infrastructure, in fact it’s served us pretty well we’ve experienced very few issues. Even so, we were spending a non-trivial amount of time each month on operational overhead. Troubleshooting backup failures, relocating instances because of a network revamp, spinning up new environments for testing, configuring encryption for SOC2 compliance and additional requirements for customers storing data outside of the United States. All of this maintenance work adds up, slowing us down. We’d rather have our development team pushing out new features, and our DevOps team running our CI/CD process, or supporting new initiatives like our move to a microservices architecture.

MongoDB Atlas to the Rescue

Personally I’ve never been a big fan of managed cloud services. They always seem to be lacking important features, have frustrating usability issues, and/or monetarily punish you in creative ways. MongoDB Atlas, however, was very attractive to us for the following reasons:

  • Fully functional MongoDB instance. With the ability to upgrade versions in a few clicks you can always have the newest feature set. It’s also close to identical to what you would deploy on-premise, not some specific “hosted” variation that can’t do x, y, or z.
  • Easy backup and restore. Click to enable, edit the schedule in one pane with cost estimation and options for separate daily, weekly, and monthly retention periods. Straight-forward restore with option to download or query data in place, without having to restore the complete backup.
  • Integrated metrics and alerts. No more setting up agents on every node. Built in alert integrations with services like PagerDuty.
  • Easy scale up/down. Need more RAM? Edit the cluster configuration and select a bigger size. Need more disk space? Drag a slider.
  • Easy to provision new clusters. A few mouse clicks or an API call and we can have a cluster with encryption, sharding, and backups. The GUI also shows you a direct cost estimation.
  • No nickel and diming. The folks at MongoDB could have easily added extra charges for each cluster modification, encryption, restores, alerts, and various other things to muddy the water, but they did not. You pay for the managed instances and backups you use and that’s it.
  • No lock-in. Migrating data in and out of Atlas is extremely simple.

After experimenting with the various features it became pretty obvious this would eliminate the vast majority of our overhead

With Atlas it now takes about 10 minutes to provision a full cluster with monitoring and any other features we need. We can even create a new group, which is sort of like a linked account, and give developers the ability to create their own clusters in an isolated environment without waiting on us.

This is Going to Hurt...Oh Wait...

So with all of this the only potential downside has to be the price, right? It turns out this was the most pleasant surprise of all.

Since our Atlas clusters are running on AWS and the cluster sizes are based on EC2 instances, we can make some rough comparisons on price:

  • M30 cluster w/ 8gb RAM and 100gb storage - ~$430/mo
  • 3 m4.large AWS instances w/ 8gb RAM and 100gb storage - ~265/mo

This does not include storage cost for backups or data transfer but gives us a good idea. If we’re running on an M30 instance in production, we don’t have to save many man hours to justify a ~$165/month difference. Another thing to note is that most people will over-provision their own clusters because it’s not easy to scale. With Atlas, we can eliminate this waste.

Going Forward

As we continue to build out our continuous deployment systems we keep looking for ways to dynamically provision environments and test for scale. Another thing Atlas offers in this respect is an API that encompasses most of their GUI features. This allows us to programmatically spin up a cluster as part of a test run:

We can also use a Docker container for this, but if we’re trying to replicate our production environment or test performance it can be quite difficult with containers.

The one thing missing from the API at the moment is access to metrics. It would be a killer feature if we could spin up a cluster, run some tests, then include production-like database performance stats in our test results. I know after speaking with the Atlas team, that this is coming in the next few months.

Conclusion

We have been using MongoDB Atlas in production for around 6 months and have been very happy with it. Our migration consisted of stopping MongoDB, running a mongodump, then simply restoring it with mongorestore using the new Atlas connection strings. We were able to schedule a maintenance window to accomplish this but there are also now options for doing a live migration if you require the uptime. Because of the simplicity and minimal effort involved in migrating the data, we were able to see cost savings in the first month. Clear, upfront cost estimates have been especially nice since I seem to spend a lot of my time in convoluted pricing calculators. Hopefully our experience inspires others to to give Atlas a try (which you can do without cost using the MongoDB Atlas free tier).

Read the original blog entry...

More Stories By Mat Rider

MongoDB is a document database with the scalability and flexibility that you want with the querying and indexing that you need.

Latest Stories
Cloud-based disaster recovery is critical to any production environment and is a high priority for many enterprise organizations today. Nearly 40% of organizations have had to execute their BCDR plan due to a service disruption in the past two years. Zerto on IBM Cloud offer VMware and Microsoft customers simple, automated recovery of on-premise VMware and Microsoft workloads to IBM Cloud data centers.
SYS-CON Events announced today that Daiya Industry will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Daiya Industry specializes in orthotic support systems and assistive devices with pneumatic artificial muscles in order to contribute to an extended healthy life expectancy. For more information, please visit https://www.daiyak...
The Internet giants are fully embracing AI. All the services they offer to their customers are aimed at drawing a map of the world with the data they get. The AIs from these companies are used to build disruptive approaches that cannot be used by established enterprises, which are threatened by these disruptions. However, most leaders underestimate the effect this will have on their businesses. In his session at 21st Cloud Expo, Rene Buest, Director Market Research & Technology Evangelism at Ara...
SYS-CON Events announced today that N3N will exhibit at SYS-CON's @ThingsExpo, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. N3N’s solutions increase the effectiveness of operations and control centers, increase the value of IoT investments, and facilitate real-time operational decision making. N3N enables operations teams with a four dimensional digital “big board” that consolidates real-time live video feeds alongside IoT sensor data a...
Internet of @ThingsExpo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 21st Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The Internet of Things (IoT) is the most profound change in personal and enterprise IT since the creation of the Worldwide Web more than 20 years ago. All major researchers estimate there will be tens of billions devic...
Mobile device usage has increased exponentially during the past several years, as consumers rely on handhelds for everything from news and weather to banking and purchases. What can we expect in the next few years? The way in which we interact with our devices will fundamentally change, as businesses leverage Artificial Intelligence. We already see this taking shape as businesses leverage AI for cost savings and customer responsiveness. This trend will continue, as AI is used for more sophistica...
SYS-CON Events announced today that SourceForge has been named “Media Sponsor” of SYS-CON's 21st International Cloud Expo, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. SourceForge is the largest, most trusted destination for Open Source Software development, collaboration, discovery and download on the web serving over 32 million viewers, 150 million downloads and over 460,000 active development projects each and every month.
SYS-CON Events announced today that DXWorldExpo has been named “Global Sponsor” of SYS-CON's 21st International Cloud Expo, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Digital Transformation is the key issue driving the global enterprise IT business. Digital Transformation is most prominent among Global 2000 enterprises and government institutions.
One of the biggest challenges with adopting a DevOps mentality is: new applications are easily adapted to cloud-native, microservice-based, or containerized architectures - they can be built for them - but old applications need complex refactoring. On the other hand, these new technologies can require relearning or adapting new, oftentimes more complex, methodologies and tools to be ready for production. In his general session at @DevOpsSummit at 20th Cloud Expo, Chris Brown, Solutions Marketi...
In the world of DevOps there are ‘known good practices’ – aka ‘patterns’ – and ‘known bad practices’ – aka ‘anti-patterns.' Many of these patterns and anti-patterns have been developed from real world experience, especially by the early adopters of DevOps theory; but many are more feasible in theory than in practice, especially for more recent entrants to the DevOps scene. In this power panel at @DevOpsSummit at 18th Cloud Expo, moderated by DevOps Conference Chair Andi Mann, panelists discussed...
Elon Musk is among the notable industry figures who worries about the power of AI to destroy rather than help society. Mark Zuckerberg, on the other hand, embraces all that is going on. AI is most powerful when deployed across the vast networks being built for Internets of Things in the manufacturing, transportation and logistics, retail, healthcare, government and other sectors. Is AI transforming IoT for the good or the bad? Do we need to worry about its potential destructive power? Or will we...
Many organizations adopt DevOps to reduce cycle times and deliver software faster; some take on DevOps to drive higher quality and better end-user experience; others look to DevOps for a clearer line-of-sight to customers to drive better business impacts. In truth, these three foundations go together. In this power panel at @DevOpsSummit 21st Cloud Expo, moderated by DevOps Conference Co-Chair Andi Mann, industry experts will discuss how leading organizations build application success from all...
SYS-CON Events announced today that SIGMA Corporation will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. uLaser flow inspection device from the Japanese top share to Global Standard! Then, make the best use of data to flip to next page. For more information, visit http://www.sigma-k.co.jp/en/.
The last two years has seen discussions about cloud computing evolve from the public / private / hybrid split to the reality that most enterprises will be creating a complex, multi-cloud strategy. Companies are wary of committing all of their resources to a single cloud, and instead are choosing to spread the risk – and the benefits – of cloud computing across multiple providers and internal infrastructures, as they follow their business needs. Will this approach be successful? How large is the ...
Your clients expect transactions to never fail, cloud access to be fast and always on, and their data to be protected - no exceptions. Hear about how Secure Service Container (SSC), an IBM-exclusive open technology, enables secure building and hosting of next-generation applications, both cloud and on-premises. SSC protects the full stack from external and insider threats, allows automatic encryption of data in-flight and at-rest, and is tamper-resistant during installation and runtime – with no...