Welcome!

Blog Feed Post

How Many Data Scientists are out there?

By

Editor’s note: This post by Gregory Piatetsky first appeared at KDnuggets.comIt it he dives into a key question regarding the possible shortage of data scientists. -bg

Many people have read the McKinsey report on Big Data (May 2011) which predicted 

The United States alone faces a shortage of 140,000 to 190,000 people with analytical expertise and 1.5 million managers and analysts with the skills to understand and make decisions based on the analysis of big data.


However, it seems that so far the shortage is much less. 

The job title “Data Scientist” has grown tremendously in popularity, according to job siteindeed.com 

Job trend<br /><br /><br />
      for Data Scientist positions, 2006-2014 

However, notice that the demand stopped increasing sometime in 2013. 

As of March 13, 2014, Search for “Data Scientist” jobs (US-based) on indeed.com gives only 1,000 positions. We find about 10,000 jobs when searching for Data Scientist - without quotes, but many of these jobs have title “Scientist” or something to do with data, and not necessarily represent “Data Scientist” positions. 

Of course, many people may do similar work without having the title of “data scientist”. 

Several estimates may be relevant. 

Kaggle is the leading platform for data science competitions and claims to be world’s largest community of data scientists. Kaggle reached 100,000 in July 2013, reported110,000 in Sep 2013, 120,000 members on Oct 23, 2013, reported to have 140,000 on Feb 24, 2014. 
Latest numbers, from Kaggle CEO Anthony Goldbloom are: 157,142 Kaggle members, of whom 67,776 active in the last 6 months. 

A quick examination of the top 10 ranked Kagglers shows that only one has a title of “Data Scientist”. Top 10 include neuroscience researchers, PhD mathematicians and physicists, and while they are clearly talented competitors on Kaggle, their actual job may not involve data science. 

LinkedIn has many groups related to data science, Big Data and Analytics – see my analysis Top 2013 LinkedIn Groups for Analytics, Big Data, Data Mining, and Data Science

The two largest of these groups are:


Most members of these groups do not have the job title “Data scientists”. There is a “Data Scientists” LinkedIn group, but it has at present only 6,750 members. 

LinkedIn Data Scientist Peter Skomoroch, @PeteSkomoroch wrote 

Using the public LinkedIn search interface, with the job title in quotes – I see 12,170 members with the phrase “data scientist” anywhere their profile. Using the advanced search facet to look only at profiles with a current or past title containing the phrase “data scientist”, I see 6,896 results. Doing a plain keyword search will return many members that mention the words “data” or “scientist” anywhere in their profile, but the majority of those people have nothing to do with data science.


He further estimated that perhaps 150-250K people would be a match for a data scientist based on their skills and education. 

I remain optimistic that data scientist is a great profession, but I doubt that there is a demand for 100,000 new data scientist positions. There may be a re-branding of existing positions, or creation of teams which collectively do the data science job.

 

Gregory Piatetsky-Shapiro, Ph.D., is a well-known expert in Business Analytics, Data Mining, and Data Science. Gregory is the Editor and Publisher of KDnuggets.com, a Business Analytics “Guru” on Twitter, and a Top Influencer in Big Data, Data Mining, and Data Science. Gregory is a co-founder of KDD (Knowledge Discovery and Data mining conferences) and SIGKDD, professional organization for Knowledge Discovery and Data Mining. Gregory has over 60 publications and edited several books and collections on data mining and knowledge discovery.

Read the original blog entry...

More Stories By Bob Gourley

Bob Gourley writes on enterprise IT. He is a founder and partner at Cognitio Corp and publsher of CTOvision.com

Latest Stories
Without a clear strategy for cost control and an architecture designed with cloud services in mind, costs and operational performance can quickly get out of control. To avoid multiple architectural redesigns requires extensive thought and planning. Boundary (now part of BMC) launched a new public-facing multi-tenant high resolution monitoring service on Amazon AWS two years ago, facing challenges and learning best practices in the early days of the new service. In his session at 19th Cloud Exp...
The cloud promises new levels of agility and cost-savings for Big Data, data warehousing and analytics. But it’s challenging to understand all the options – from IaaS and PaaS to newer services like HaaS (Hadoop as a Service) and BDaaS (Big Data as a Service). In her session at @BigDataExpo at @ThingsExpo, Hannah Smalltree, a director at Cazena, provided an educational overview of emerging “as-a-service” options for Big Data in the cloud. This is critical background for IT and data professionals...
Today we can collect lots and lots of performance data. We build beautiful dashboards and even have fancy query languages to access and transform the data. Still performance data is a secret language only a couple of people understand. The more business becomes digital the more stakeholders are interested in this data including how it relates to business. Some of these people have never used a monitoring tool before. They have a question on their mind like “How is my application doing” but no id...
@GonzalezCarmen has been ranked the Number One Influencer and @ThingsExpo has been named the Number One Brand in the “M2M 2016: Top 100 Influencers and Brands” by Onalytica. Onalytica analyzed tweets over the last 6 months mentioning the keywords M2M OR “Machine to Machine.” They then identified the top 100 most influential brands and individuals leading the discussion on Twitter.
What happens when the different parts of a vehicle become smarter than the vehicle itself? As we move toward the era of smart everything, hundreds of entities in a vehicle that communicate with each other, the vehicle and external systems create a need for identity orchestration so that all entities work as a conglomerate. Much like an orchestra without a conductor, without the ability to secure, control, and connect the link between a vehicle’s head unit, devices, and systems and to manage the ...
More and more brands have jumped on the IoT bandwagon. We have an excess of wearables – activity trackers, smartwatches, smart glasses and sneakers, and more that track seemingly endless datapoints. However, most consumers have no idea what “IoT” means. Creating more wearables that track data shouldn't be the aim of brands; delivering meaningful, tangible relevance to their users should be. We're in a period in which the IoT pendulum is still swinging. Initially, it swung toward "smart for smar...
"We are an all-flash array storage provider but our focus has been on VM-aware storage specifically for virtualized applications," stated Dhiraj Sehgal of Tintri in this SYS-CON.tv interview at 19th Cloud Expo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.
In an era of historic innovation fueled by unprecedented access to data and technology, the low cost and risk of entering new markets has leveled the playing field for business. Today, any ambitious innovator can easily introduce a new application or product that can reinvent business models and transform the client experience. In their Day 2 Keynote at 19th Cloud Expo, Mercer Rowe, IBM Vice President of Strategic Alliances, and Raejeanne Skillern, Intel Vice President of Data Center Group and G...
"We are a modern development application platform and we have a suite of products that allow you to application release automation, we do version control, and we do application life cycle management," explained Flint Brenton, CEO of CollabNet, in this SYS-CON.tv interview at DevOps at 19th Cloud Expo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.
Information technology is an industry that has always experienced change, and the dramatic change sweeping across the industry today could not be truthfully described as the first time we've seen such widespread change impacting customer investments. However, the rate of the change, and the potential outcomes from today's digital transformation has the distinct potential to separate the industry into two camps: Organizations that see the change coming, embrace it, and successful leverage it; and...
In IT, we sometimes coin terms for things before we know exactly what they are and how they’ll be used. The resulting terms may capture a common set of aspirations and goals – as “cloud” did broadly for on-demand, self-service, and flexible computing. But such a term can also lump together diverse and even competing practices, technologies, and priorities to the point where important distinctions are glossed over and lost.
All clouds are not equal. To succeed in a DevOps context, organizations should plan to develop/deploy apps across a choice of on-premise and public clouds simultaneously depending on the business needs. This is where the concept of the Lean Cloud comes in - resting on the idea that you often need to relocate your app modules over their life cycles for both innovation and operational efficiency in the cloud. In his session at @DevOpsSummit at19th Cloud Expo, Valentin (Val) Bercovici, CTO of Soli...
Predictive analytics tools monitor, report, and troubleshoot in order to make proactive decisions about the health, performance, and utilization of storage. Most enterprises combine cloud and on-premise storage, resulting in blended environments of physical, virtual, cloud, and other platforms, which justifies more sophisticated storage analytics. In his session at 18th Cloud Expo, Peter McCallum, Vice President of Datacenter Solutions at FalconStor, discussed using predictive analytics to mon...
Keeping pace with advancements in software delivery processes and tooling is taxing even for the most proficient organizations. Point tools, platforms, open source and the increasing adoption of private and public cloud services requires strong engineering rigor – all in the face of developer demands to use the tools of choice. As Agile has settled in as a mainstream practice, now DevOps has emerged as the next wave to improve software delivery speed and output. To make DevOps work, organization...
With major technology companies and startups seriously embracing IoT strategies, now is the perfect time to attend @ThingsExpo 2016 in New York. Learn what is going on, contribute to the discussions, and ensure that your enterprise is as "IoT-Ready" as it can be! Internet of @ThingsExpo, taking place June 6-8, 2017, at the Javits Center in New York City, New York, is co-located with 20th Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry p...