Welcome!

News Feed Item

Databricks Expands Platform for Turnkey Production Apache Spark Deployments in the Cloud

Company Launches Enhanced Reliability and Security Capabilities for Data Engineering on its Managed Spark Platform

LAS VEGAS, NV -- (Marketwired) -- 11/30/16 -- Databricks®, the company founded by the team that created the popular Apache® Spark™ project, announced new capabilities to its platform that further simplify the production deployment of Spark in the cloud. The production enhancements complement the existing Databricks environment for data science, which enable users to collaboratively analyze data in real-time with data science notebooks and immediately deploy them as production Spark jobs and workflows. The announcement was made today at the 2016 Amazon Web Services (AWS) re:Invent conference.

The production features announced today enable users to effortlessly setup and run Spark jobs and workflows without humans in the loop via APIs, monitor performance and troubleshoot errors with detailed logs, manage AWS EC2 costs with AWS Tags, control access to resources with AWS IAM Roles, and increase the scalability of long-running workloads with encrypted AWS Elastic Block Storage (EBS). Databricks is the first and only vendor to offer a SOC2 and HIPAA compliant Spark platform that provides turnkey deployment of both real-time analysis and production Spark workloads with a seamless transition from analysis to production.

As organizations across industries deploy Apache Spark in the public cloud, the task of minimizing costly downtimes of mission-critical workloads, such as applications that predict equipment failure, falls on data engineering teams. Yet, building sophisticated systems around Spark to ensure that such workloads are resilient, easy to troubleshoot, and secure, requires a high level of technical expertise and meticulous efforts that most organizations struggle to spare.

"As enterprises increasingly rely on Apache Spark to power more diverse production workloads supporting more people, it becomes critical to prevent business system outages that could cost millions of dollars," said Nik Rouda, Senior Analyst at Enterprise Strategy Group.

In Databricks' production environment, data engineers can bypass the difficult and tedious tasks of developing, configuring, tuning and securing infrastructure to easily achieve production requirements with features such as:

  • HIPAA and SOC2-compliant Apache Spark clusters fully managed and tuned by the Spark committers at Databricks;
  • REST APIs to orchestrate and monitor sophisticated Spark jobs and workflows programmatically, without humans in the loop;
  • End-to-end logs and performance metrics to easily debug and fine-tune Spark workloads, accessible via APIs programmatically or in the Databricks user interface;
  • Customizable AWS tags to manage the AWS EC2 usage of each Spark cluster;
  • Encrypted AWS Elastic Block Storage (EBS) to increase the reliability of long-running Spark jobs on AWS EC2 instances by automatically providing additional storage;
  • AWS IAM Roles integration to provide secure access to AWS resources to diverse user groups in the same organization;
  • Direct integration with the data science environment to let organizations instantly move exploratory work to production without re-engineering;
  • SSH Access to provide engineers direct access to the production environment to troubleshoot and inspect the Spark clusters.

"Databricks is experiencing unprecedented demand for a robust and secure Apache Spark platform in the cloud to run production workloads," says Ali Ghodsi, CEO and Co-Founder of Databricks. "We are proud to enable one of our core user groups, the data engineers, to meet the most stringent of operational requirements."

Visit databricks.com or Booth #1341 at AWS re:Invent to learn more.

Contact Databricks to get started: http://go.databricks.com/contact-databricks.

About Databricks
Databricks' vision is to empower anyone to easily build and deploy advanced analytics solutions. The company was founded by the team who created Apache® Spark™, a powerful open source data processing engine built for sophisticated analytics, ease of use, and speed. Databricks is the largest contributor to the open source Apache Spark project. The company has also trained over 20,000 users on Apache Spark, and has the largest number of customers deploying Spark to date. Databricks provides a just-in-time data platform, to simplify data integration, real-time experimentation, and robust deployment of production applications. Databricks is venture-backed by Andreessen Horowitz and NEA. For more information, contact [email protected].

© Databricks 2016. All rights reserved. Apache, Apache Spark and Spark are trademarks of the Apache Software Foundation.

More Stories By Marketwired .

Copyright © 2009 Marketwired. All rights reserved. All the news releases provided by Marketwired are copyrighted. Any forms of copying other than an individual user's personal reference without express written permission is prohibited. Further distribution of these materials is strictly forbidden, including but not limited to, posting, emailing, faxing, archiving in a public database, redistributing via a computer network or in a printed form.

Latest Stories
Your homes and cars can be automated and self-serviced. Why can't your storage? From simply asking questions to analyze and troubleshoot your infrastructure, to provisioning storage with snapshots, recovery and replication, your wildest sci-fi dream has come true. In his session at @DevOpsSummit at 20th Cloud Expo, Dan Florea, Director of Product Management at Tintri, will provide a ChatOps demo where you can talk to your storage and manage it from anywhere, through Slack and similar services ...
Imagine having the ability to leverage all of your current technology and to be able to compose it into one resource pool. Now imagine, as your business grows, not having to deploy a complete new appliance to scale your infrastructure. Also imagine a true multi-cloud capability that allows live migration without any modification between cloud environments regardless of whether that cloud is your private cloud or your public AWS, Azure or Google instance. Now think of a world that is not locked i...
SYS-CON Events announced today that Auditwerx will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Auditwerx specializes in SOC 1, SOC 2, and SOC 3 attestation services throughout the U.S. and Canada. As a division of Carr, Riggs & Ingram (CRI), one of the top 20 largest CPA firms nationally, you can expect the resources, skills, and experience of a much larger firm combined with the accessibility and atten...
In his session at @ThingsExpo, Eric Lachapelle, CEO of the Professional Evaluation and Certification Board (PECB), will provide an overview of various initiatives to certifiy the security of connected devices and future trends in ensuring public trust of IoT. Eric Lachapelle is the Chief Executive Officer of the Professional Evaluation and Certification Board (PECB), an international certification body. His role is to help companies and individuals to achieve professional, accredited and worldw...
MongoDB Atlas leverages VPC peering for AWS, a service that allows multiple VPC networks to interact. This includes VPCs that belong to other AWS account holders. By performing cross account VPC peering, users ensure networks that host and communicate their data are secure. In his session at 20th Cloud Expo, Jay Gordon, a Developer Advocate at MongoDB, will explain how to properly architect your VPC using existing AWS tools and then peer with your MongoDB Atlas cluster. He'll discuss the secur...
SYS-CON Events announced today that Linux Academy, the foremost online Linux and cloud training platform and community, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Linux Academy was founded on the belief that providing high-quality, in-depth training should be available at an affordable price. Industry leaders in quality training, provided services, and student certification passes, its goal is to c...
DevOps is often described as a combination of technology and culture. Without both, DevOps isn't complete. However, applying the culture to outdated technology is a recipe for disaster; as response times grow and connections between teams are delayed by technology, the culture will die. A Nutanix Enterprise Cloud has many benefits that provide the needed base for a true DevOps paradigm. In his Day 3 Keynote at 20th Cloud Expo, Chris Brown, a Solutions Marketing Manager at Nutanix, will explore t...
In his session at Cloud Expo, Alan Winters, an entertainment executive/TV producer turned serial entrepreneur, will present a success story of an entrepreneur who has both suffered through and benefited from offshore development across multiple businesses: The smart choice, or how to select the right offshore development partner Warning signs, or how to minimize chances of making the wrong choice Collaboration, or how to establish the most effective work processes Budget control, or how to m...
Deep learning has been very successful in social sciences and specially areas where there is a lot of data. Trading is another field that can be viewed as social science with a lot of data. With the advent of Deep Learning and Big Data technologies for efficient computation, we are finally able to use the same methods in investment management as we would in face recognition or in making chat-bots. In his session at 20th Cloud Expo, Gaurav Chakravorty, co-founder and Head of Strategy Development ...
"delaPlex is a software development company. We do team-based outsourcing development," explained Mark Rivers, COO and Co-founder of delaPlex Software, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York City, NY.
SYS-CON Events announced today that SoftLayer, an IBM Company, has been named “Gold Sponsor” of SYS-CON's 18th Cloud Expo, which will take place on June 7-9, 2016, at the Javits Center in New York, New York. SoftLayer, an IBM Company, provides cloud infrastructure as a service from a growing number of data centers and network points of presence around the world. SoftLayer’s customers range from Web startups to global enterprises.
SYS-CON Events announced today that Technologic Systems Inc., an embedded systems solutions company, will exhibit at SYS-CON's @ThingsExpo, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Technologic Systems is an embedded systems company with headquarters in Fountain Hills, Arizona. They have been in business for 32 years, helping more than 8,000 OEM customers and building over a hundred COTS products that have never been discontinued. Technologic Systems’ pr...
SYS-CON Events announced today that CA Technologies has been named “Platinum Sponsor” of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY, and the 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. CA Technologies helps customers succeed in a future where every business – from apparel to energy – is being rewritten by software. From ...
In his keynote at @ThingsExpo, Chris Matthieu, Director of IoT Engineering at Citrix and co-founder and CTO of Octoblu, focused on building an IoT platform and company. He provided a behind-the-scenes look at Octoblu’s platform, business, and pivots along the way (including the Citrix acquisition of Octoblu).
SYS-CON Events announced today that Loom Systems will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Founded in 2015, Loom Systems delivers an advanced AI solution to predict and prevent problems in the digital business. Loom stands alone in the industry as an AI analysis platform requiring no prior math knowledge from operators, leveraging the existing staff to succeed in the digital era. With offices in S...