In this special guest feature, Ashok Reddy, Group General Manager for DevOps at CA, discusses the data scientist talent gap and how in this high-demand market for data scientists, companies need to think differently about the position, and how it relates to other parts of the organization. Ashok is completing a MS degree in CS with specialization in Interactive Intelligence, Machine Learning at Georgia Tech.
It doesn’t take a data scientist to figure out that data scientists are in very high- demand.
“America’s hottest job!” screams a Bloomberg headline. “Best job in America,” says Glassdoor two years in a row, citing a number of job openings, high salaries and high job satisfaction rates. The country is experiencing a “quant crunch,” according to an IBM report, with demand for data scientists expected to surge 28 percent higher by 2020.
Identified six years ago by former Facebook and LinkedIn luminaries Jeff Hammerbacher and D.J. Patil, as “the sexiest job of the 21st Century” in Harvard Business Review, the role of the data scientist has taken center stage thanks to digitization and the continued rapid rise of incoming data as the new corporate currency.
It’s the data scientist’s job to look for patterns and trends in data and derive insights that can help a business grow. The best data scientists don’t merely examine data as it stands, they interpret it to predict what it means or will mean.
In this high-demand market for data scientists, companies need to think differently about the position, and how it relates to other parts of the organization.
In a new book, “Building Intelligent Systems,” Gene Hutton argues that data science is at the core of intelligent systems, but there remains a significant amount of work to be done between developing a working model, using machine learning, and delivering sustainable customer impact.
Organizations need to leverage machine learning and data from user interactions to achieve success and continually improve over time. This process is a team effort from data scientists, software engineers and business owners to build intelligence that powers these intelligent systems, helps them expand over time and guides their orchestration throughout the life-cycle.
From the perspective of a data scientist, understanding how these groups work together and support each other is key to better modeling. The demands and constraints that these facets of the organization put on modeling ultimately determine the process and usefulness of the model. Where will the model run? What data will it have access to? How fast does it need to be? What is the business impact of a false positive? Or a false negative? How is the model tuned to maximize business results? These are the kinds of cross-silo questions that need to be answered for truly insightful and effective modeling.
Collaboration is crucial for successful data science. To focus data scientists on the most productive activities, leveraging a DevOps approach can be quite useful in bridging the gap between the software developers writing the code, and the operations engineers continuously deploying and maintaining it. Similarly, the role of the data scientist, responsible for developing AI and machine learning models, could also benefit from tighter collaboration with software engineers and business owners, as many of these processes can be automated using a continuous delivery approach to match business goals and constraints
Universities and colleges can also do their part to address the data scientist shortage by providing appropriate bachelor and graduate programs and there are encouraging signs that this is happening. Carnegie-Mellon, Columbia and the University of California, San Diego, are among the institutions that have launched new graduate and undergraduate degree programs in data science and AI. I’ve personally enrolled in an online master’s program at Georgia Tech, with specialization in AI and machine learning. Thousands of others worldwide have also enrolled in these advanced, online degrees who were not previously able to make the sacrifices of time or money that are often required for a traditional program.
The data science talent gap can be filled, but it will take imaginative approaches by both businesses and academia. Businesses can apply DevOps and embrace automation, while university programs can play an active role in addressing the needs of the industry and preparing their students for the future. Either way, the role of the data scientist is sure to remain one of America’s hottest jobs for years to come.
Sign up for the free insideBIGDATA newsletter.
Speak Your Mind