News Feed Item
Databricks Launches “Certified Spark Distribution” Program to Recognize Vendors Committed to Supporting the Apache Spark Application Ecosystem
|By Business Wire
|June 26, 2014 12:11 PM EDT
Databricks, the company founded by the creators of Apache Spark, the
next generation Big Data engine, today announced the “Certified Spark
Distribution” program for vendors with a commercial Spark distribution.
Certification indicates that the vendor’s Spark distribution is
compatible with the open source Apache Spark distribution, enabling
“Certified on Spark” applications - certified to work with Apache Spark
- to run on the vendor’s Spark distribution out-of-the-box.
“One of Databricks’ goals is to ensure users have a fantastic
experience. Our belief is that having the community work together to
maintain compatibility and therefore facilitate a vibrant application
ecosystem is crucial to this vision,” said Ion Stoica, Databricks CEO.
“We first launched the ‘Certified on Spark’ program to help build a
robust ecosystem of innovative applications on top of Apache Spark. The
‘Certified Spark Distribution’ program is the other half of the
equation, recognizing vendors that are committed to providing a home for
these applications to allow the ecosystem to flourish.”
In keeping with the open source nature of Spark, the certification
process is fully transparent with open-source tests, lightweight, and
100% free - a mirror image of the “Certified on Spark” process for Spark
applications. Vendors fill out a short questionnaire and then simply
execute a set of open-source tests - developed and maintained by the
community and used to test each release of Apache Spark - against their
build of Spark to demonstrate compatibility.
“Certification shouldn’t be used as a tool for lock-in: Certified Spark
Distributions are not required to ship all the bits of Apache Spark, or
be open source, or prevented from innovating significantly within and
around Spark,” said Arsalan Tavakoli-Shiraji, Business Development Lead
at Databricks. “They simply need to maintain compatibility with Apache
Spark to provide support for the application ecosystem.”
As part of the certification program launch, five vendors have completed
the certification process: DataStax,
Oracle, and Pivotal
- industry leaders that have recognized and embraced the power of Spark
when integrated with their respective platforms. Each of these vendors
put their distributions through the certification process, which
included a host of integration tests to ensure full compatibility with
the latest Apache Spark release.
“One of the big risks faced by open source projects is fragmentation
among distributors. Fragmentation is bad for both users and application
developers, and ultimately for the growth of the project,” said Matei
Zaharia, Databricks CTO and VP of the Spark project at Apache. “We are
delighted that these partners - along with others in the certification
pipeline - share our vision of an undivided Spark platform based
directly around Apache, and will ensure that all applications built on
Apache Spark run on their distributions.”
Vendors interested in certifying their Spark distribution should visit www.databricks.com
and select "Apply for Certification." Enterprise users can also visit
the Databricks site regularly to see the latest set of certified
distributions and applications, and read “spotlight” blog articles that
provide deep-dives on the Spark ecosystem by newly certified vendors.
All the inaugural members will be on hand at the upcoming Spark
Summit from June 30th to July 2nd in San Francisco to provide
greater information on the role of Spark in helping better serve their
customers. Additionally, there will be an “Application
Spotlight” segment that will highlight innovative “Certified on
"DataStax is strongly committed to making Cassandra and Spark the best
combination for today's online applications," said Robin Schumacher, VP
of products at DataStax. "We have demonstrated that commitment with the
integration work we have contributed back to both open source
communities as well as the certified versions of Spark and Cassandra we
provide in DataStax Enterprise for production environments."
“We support the fact that Apache Spark project provides enterprises with
an additional processing engine in Hadoop to execute in-memory
algorithms for advanced analytics,” said John Kreisa, vice president of
strategic marketing at Hortonworks. “We applaud Databricks’ vision to
ensure Spark is fully integrated on YARN, which enterprises have adopted
as the data OS for Hadoop.”
"Pivotal's open source credentials are quite extensive -
Apache-compatible Hadoop, MADLib, RabbitMQ, CloudFoundry - and now we've
added Spark to that set," said Sarabjeet Chugh, Head of Hadoop Product
Management at Pivotal. "Additionally, we recognize the importance of a
unified community to enable the ecosystem to grow and so are thrilled to
back this effort."
Databricks was founded by the creators of Apache Spark, who have been
working for the past six years on cutting-edge systems to analyze and
process Big Data. They believe that Big Data is a tremendous opportunity
that is still largely untapped, and are actively working to
revolutionize what enterprises can do with it. Databricks is
venture-backed by Andreessen Horowitz. For more information, visit http://www.databricks.com.
"We are an all-flash array storage provider but our focus has been on VM-aware storage specifically for virtualized applications," stated Dhiraj Sehgal of Tintri in this SYS-CON.tv interview at 19th Cloud Expo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.
Dec. 2, 2016 08:30 PM EST Reads: 321
IoT is rapidly changing the way enterprises are using data to improve business decision-making. In order to derive business value, organizations must unlock insights from the data gathered and then act on these. In their session at @ThingsExpo, Eric Hoffman, Vice President at EastBanc Technologies, and Peter Shashkin, Head of Development Department at EastBanc Technologies, discussed how one organization leveraged IoT, cloud technology and data analysis to improve customer experiences and effici...
Dec. 2, 2016 08:30 PM EST Reads: 4,964
Fact is, enterprises have significant legacy voice infrastructure that’s costly to replace with pure IP solutions. How can we bring this analog infrastructure into our shiny new cloud applications? There are proven methods to bind both legacy voice applications and traditional PSTN audio into cloud-based applications and services at a carrier scale. Some of the most successful implementations leverage WebRTC, WebSockets, SIP and other open source technologies.
In his session at @ThingsExpo, Da...
Dec. 2, 2016 08:15 PM EST Reads: 1,545
The cloud competition for database hosts is fierce. How do you evaluate a cloud provider for your database platform?
In his session at 18th Cloud Expo, Chris Presley, a Solutions Architect at Pythian, gave users a checklist of considerations when choosing a provider.
Chris Presley is a Solutions Architect at Pythian. He loves order – making him a premier Microsoft SQL Server expert. Not only has he programmed and administered SQL Server, but he has also shared his expertise and passion with b...
Dec. 2, 2016 07:00 PM EST Reads: 3,905
"IoT is going to be a huge industry with a lot of value for end users, for industries, for consumers, for manufacturers. How can we use cloud to effectively manage IoT applications," stated Ian Khan, Innovation & Marketing Manager at Solgeniakhela, in this SYS-CON.tv interview at @ThingsExpo, held November 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA.
Dec. 2, 2016 06:45 PM EST Reads: 3,975
As data explodes in quantity, importance and from new sources, the need for managing and protecting data residing across physical, virtual, and cloud environments grow with it. Managing data includes protecting it, indexing and classifying it for true, long-term management, compliance and E-Discovery. Commvault can ensure this with a single pane of glass solution – whether in a private cloud, a Service Provider delivered public cloud or a hybrid cloud environment – across the heterogeneous enter...
Dec. 2, 2016 06:30 PM EST Reads: 1,470
Without a clear strategy for cost control and an architecture designed with cloud services in mind, costs and operational performance can quickly get out of control. To avoid multiple architectural redesigns requires extensive thought and planning. Boundary (now part of BMC) launched a new public-facing multi-tenant high resolution monitoring service on Amazon AWS two years ago, facing challenges and learning best practices in the early days of the new service.
In his session at 19th Cloud Exp...
Dec. 2, 2016 05:15 PM EST Reads: 355
The cloud promises new levels of agility and cost-savings for Big Data, data warehousing and analytics. But it’s challenging to understand all the options – from IaaS and PaaS to newer services like HaaS (Hadoop as a Service) and BDaaS (Big Data as a Service). In her session at @BigDataExpo at @ThingsExpo, Hannah Smalltree, a director at Cazena, provided an educational overview of emerging “as-a-service” options for Big Data in the cloud. This is critical background for IT and data professionals...
Dec. 2, 2016 05:00 PM EST Reads: 4,072
Today we can collect lots and lots of performance data. We build beautiful dashboards and even have fancy query languages to access and transform the data. Still performance data is a secret language only a couple of people understand. The more business becomes digital the more stakeholders are interested in this data including how it relates to business. Some of these people have never used a monitoring tool before. They have a question on their mind like “How is my application doing” but no id...
Dec. 2, 2016 04:45 PM EST Reads: 2,098
@GonzalezCarmen has been ranked the Number One Influencer and @ThingsExpo has been named the Number One Brand in the “M2M 2016: Top 100 Influencers and Brands” by Onalytica.
Onalytica analyzed tweets over the last 6 months mentioning the keywords M2M OR “Machine to Machine.” They then identified the top 100 most influential brands and individuals leading the discussion on Twitter.
Dec. 2, 2016 04:45 PM EST Reads: 1,967
What happens when the different parts of a vehicle become smarter than the vehicle itself? As we move toward the era of smart everything, hundreds of entities in a vehicle that communicate with each other, the vehicle and external systems create a need for identity orchestration so that all entities work as a conglomerate. Much like an orchestra without a conductor, without the ability to secure, control, and connect the link between a vehicle’s head unit, devices, and systems and to manage the ...
Dec. 2, 2016 04:15 PM EST Reads: 349
More and more brands have jumped on the IoT bandwagon. We have an excess of wearables – activity trackers, smartwatches, smart glasses and sneakers, and more that track seemingly endless datapoints. However, most consumers have no idea what “IoT” means. Creating more wearables that track data shouldn't be the aim of brands; delivering meaningful, tangible relevance to their users should be.
We're in a period in which the IoT pendulum is still swinging. Initially, it swung toward "smart for smar...
Dec. 2, 2016 04:15 PM EST Reads: 327
In an era of historic innovation fueled by unprecedented access to data and technology, the low cost and risk of entering new markets has leveled the playing field for business. Today, any ambitious innovator can easily introduce a new application or product that can reinvent business models and transform the client experience. In their Day 2 Keynote at 19th Cloud Expo, Mercer Rowe, IBM Vice President of Strategic Alliances, and Raejeanne Skillern, Intel Vice President of Data Center Group and G...
Dec. 2, 2016 04:00 PM EST Reads: 1,862
"We are a modern development application platform and we have a suite of products that allow you to application release automation, we do version control, and we do application life cycle management," explained Flint Brenton, CEO of CollabNet, in this SYS-CON.tv interview at DevOps at 19th Cloud Expo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.
Dec. 2, 2016 03:45 PM EST Reads: 644
Information technology is an industry that has always experienced change, and the dramatic change sweeping across the industry today could not be truthfully described as the first time we've seen such widespread change impacting customer investments. However, the rate of the change, and the potential outcomes from today's digital transformation has the distinct potential to separate the industry into two camps: Organizations that see the change coming, embrace it, and successful leverage it; and...
Dec. 2, 2016 03:30 PM EST Reads: 3,189