Within the past five years the term “big data” has gone from “what the heck is that?” at the CIO level to “we need a big data strategy” at the CEO level. We know what big data is, how best to leverage it, and how it’s transforming every aspect of our lives – from the way we work, to the way we live, to the way we play. However, we still need to consider the overhead a big data strategy places on an organization, and how to best bring staff up to speed on the latest skills while staying focused on the day to day of running the business.
Modern big data strategies tend to rely on adopting cutting edge open source technologies like Apache Spark, Kafka, Cassandra and Mesos. These types of tools allow development teams to build scalable distributed applications that can ingest data faster than ever before, process and analyze it on the fly, and move from the batch-oriented approach of the earlier big data technologies to a near real-time capability. As these technologies are evolving rapidly, and new ones are constantly emerging, how do CIOs staff up around these tools when skills are in such short supply, and all the qualified developers are being aggressively recruited by the hottest startups?
Without investing in staff now, IT teams won’t make the transition from big data technology cost center to profit center. Those skills will not be mainstream enough to ensure they don’t command the premium they do today (survey data from Stack Overflow here). Furthermore, a recent DataStax survey of more than 250 developers, engineers and architects revealed only eight percent of respondents believe the current pool of skilled NoSQL experts is enough for the demand in the DBMS industry.
The most popular open-source technology has backing from commercial companies to provide education, support, software and services. Enterprises should steer clear of any technologies that don’t have some sort of vendor support. Who will you call when it all goes horribly wrong? These vendors also share the same desires as CIOs: to make the skills around their system so prevalent and commoditized that there’s no barrier to adoption.
However, most open-source start-ups depend on revenue from training for growth, but by charging for training they put a financial hurdle in place that limits the number of people who are willing to pay to learn the technology. Open-source companies, like DataStax, are going one step further and offering online training for free to address the growing need to develop and expand the NoSQL and Cassandra talent pool. While these initiatives are in their infancy, they will help commoditize big data skills over time and enable CIOs to retrain their staff at a much lower cost, ensuring that the lack of skills in the market doesn’t continue to be a cost inhibitor to adopting big data technologies.
Contributed by: Christian Hasker, Director of DataStax Academy at DataStax. Christian has over 15 years of experience in technology, from tiny startups to larger organizations. He has held a variety of roles over the years, including sales management, product marketing, product management and community development. He is passionate about education and lives in San Francisco with his wife and twin daughters.
Sign up for the free insideBIGDATA newsletter.
Speak Your Mind