It’s hard to believe that the hottest job in America at one time was blacksmith. Today, it’s data scientist, according to a growing number of sources, including job site Glassdoor. It recently reported that the annual median salary for data scientist positions is $116,840. Other studies have found that those with master’s degrees or doctorates sometimes pull down double or triple that sum.
As the digital age unfolds, finding talent to unravel the mysteries of data is paramount. The invisible lines and spaces between connected things offers insights and answers that can fuel a business, disrupt an industry and produce exponential gains.
A new report from data science firm CrowdFlower, 2016 Data Scientist Report, offers a glimpse into how the field is evolving and what challenges CIOs and others face. Last year, 79 percent of respondents indicated that there was a shortage of data scientists. This year’s report found that the number had jumped to 83 percent.
And while data scientists said that they love their job—more than 80 percent indicated that they are happy and optimistic about the evolution of the field—many also hope their work becomes more interesting and less repetitive in the future.
In fact, the most interesting finding from the report is that two-thirds of the respondents spend most of their time cleaning and organizing data—a task that is sometimes referred to as “data wrangling” or serving as a “digital janitor.” This includes everything from list verification to removing commas and debugging databases.
It’s messy and time-consuming work—and in many cases it’s a poor use of talent.
Essentially, the study found that the tasks data scientists do most they like the least. A whopping 60 percent clean and organize data but only 9 percent focus on mining data for patterns, and 4 percent said they devote time to refining algorithms.
In addition, 14 percent noted that they feel held back by the tools they use and 27 percent would like to see more support and direction from their management or executive team.
All of this is significant because, in a world where there’s a shortage of data scientists, organizations that fail to make these jobs interesting and rewarding risk losing the talent they have. In fact, the report concludes that “the status quo probably isn’t sustainable.” While data labeling and cleaning is important, organizations and data scientists are “much better served doing predictive analysis and building out machine learning practices.”
