In this special guest feature, Rohit Gupta of Palerra discusses some specific new uses of machine learning technology—especially as it pertains to cybersecurity. Rohit founded Palerra in 2013 with the vision of ushering in a new paradigm in security and devops; one that would enable enterprises to confidently embrace and accelerate the move to the cloud. As CEO at Palerra, Rohit is responsible for creating, communicating and delivering on the overall vision and strategy for the company.
In today’s world, there isn’t a week gone by when a major breach or hack is announced. The change from a decade ago is that the new adversary is particularly focused, keeps shifting origins, drives multi-pronged & changing attack patterns and is generally part of a well funded syndicate. This “new normal” that we live in, requires us to leverage new tools and techniques to combat these sophisticated professionals who are out to disrupt our daily existence.
Machine Learning techniques can help organizations in finding usable patterns and then help them make informed decisions and predictions. The primary thesis behind machine learning is to train an algorithm to predict a certain set of outputs within a set of probabilistic boundaries when given input data. These generally get refined and granular over time; and this process called predictive modeling, once mastered, can be applied to predict critical insights, anomalies and variances to combat cyber-adversaries. Machine learning techniques are either supervised (such as the predictive modeling technique mentioned above) or unsupervised, if you don’t know what you are looking for when you start. Unsupervised learning can also produce clustering and hierarchy charts that show inherent relationships in the data, and can also discover dependencies that generate valuable predictive insights.
I’m particularly optimistic about the fit for machine learning techniques in the cyber security industry. First, the collection and storage of large volumes of data has been steadily adopted as a best practice, particularly in financial services and defense industries. However, a typical challenge faced by these organizations is the approach and tooling required to sort, mine and interpret the insights and patterns from the aggregated data. Secondly, the talent pool to deal with these tasks has been continually on the decline. This leaves opportunities for things to fall through the proverbial crack and cause security incidents to increase. These two points are illustrations of why machine learning can improve the security posture of an organization.
Now, lets take a real world example of machine learning in action. Consider an organization that has traditional network infrastructure such as firewalls, web proxies, leverages SaaS and Cloud applications such as Salesforce.com or collaboration tools such as those from Microsoft, Google etc. If this organization is faced with possible data exfiltration from their sensitive applications, there are several tasks that are required. These include identifying what was compromised and stolen, the method or approach to the theft, and then fixing the vulnerable medium so that the action cannot be repeated.
Traditional approaches to forensics are massively cumbersome. These would include analyzing network traffic, continually instrumenting and analyzing log files and then watching for anomalous behavior in the environment or application where the exfiltration took place.
However, with machine learning many of these tasks can be automated; and can be deployed such that anomalies could be caught before any real damage is done. Well-trained machine learning algorithms can analyze large traffic data sets for patterns and lock down connections in the event of an anomaly. Similarly, a mature machine learning toolset can automatically baseline user activity over time and any deviation from normal baselines can result in triggered corrective activity, or notifications to suspicious or malicious insiders.
The benefits of machine learning when applied to cyber security include rapid detection rates, discovery of new attack vectors, and faster identification and correlation of anomalous behavior. In a world where Cloud computing and SaaS applications are being adopted with breakneck velocity, machine learning offers tremendous possibilities to counter and manage cyber adversaries.
Today’s cyber security landscape requires a mix of both automated machine learning systems working in collaboration with human analysis. The next few years will be interesting in the cyber security landscape. The massive amounts of data that can be generated, along with the problems of conducting large scale analytics to find the proverbial “needle in the haystack”, offer the perfect combination for successful machine learning deployments.
Sign up for the free insideBIGDATA newsletter.
There are a lot of upsides to cyber security and machine learning techniques like quicker identification and discovery vectors. The only issue I’ve com across in the past though is the fact that in the current state that AI is (infancy) they do need to be maintained and checked regularly to make sure than no false positives occur.
Although its worth noting that as time moves so does machine learning, and it will continue to develop past the point it is now. But for me, i think AI and machine learning needs to move forward more to truly reach the potential of these systems.