In large organizations today, the amount of data that needs to be sifted through continues to grow in volume. Consider this stunning prediction from John Rydning, the research vice president of IDC’s Global DataSphere: the global datasphere is expected to more than double between 2022 and 2026. Not only is there a massive amount of information, but there aren’t enough people to properly analyze all of it.
Within the security function alone, there is a massive amount of data, with different tools all generating information. But when you apply advanced analytics in a meaningful way, you can drive valuable insights, cut through the noise and, ultimately, increase efficiency. It’s helpful to start small and then build up toward more advanced use cases.
Below are recommendations for three areas to consider when it comes to your advanced analytics initiative, along with four best practices.
Understanding every asset on your infrastructure
Every organization struggles with understanding what is in their tech stack while trying to prevent shadow IT from going rogue and creating security risks. Data can help you see the gaps as it helps reveal connections from one point to another. Relying on manual efforts can result in human error, and a lack of real-time asset management can delay response or inhibit effective workflows.
Since data is entropic and cloud-based services can be ephemeral, the lack of real-time insights into the relationships of your assets and configurations can delay response or inhibit security workflows and the IT service delivery process.
Don’t underestimate the value of “simple” queries
Security information and event management (SIEMs) tools can be computationally expensive and intensive; security analysts want to “save” advanced analytics for SIEMs to perform, but that undercuts the value of what you might think of as “simple” queries.
Simple queries enhance advanced analytics. Basic queries, such as “how many failed login attempts have there been in the past hour,” can help to paint a more complete picture of what’s really happening in your environment when combined with the results of more advanced queries. An example would be a search for specific CVEs to discover vulnerabilities in the organization. The net result of these connected queries is a better and more complete response to the question you’re trying to answer.
Using advanced analytics to reduce the noise
Today’s organizations are grappling with thousands of alerts. While precise numbers vary, it’s widely estimated that the average security operations center (SOC) receives around 10,000 alerts a day. Yet as the number of alerts continues to grow, the number of people you have in the SOC who can address those alerts isn’t scaling. Traditional approaches to handling these alerts won’t suffice today.
Advanced analytics can play a key role in helping drive down the number of threats that your human analysts have to work with. They can be used, for example, to help sort out the false positives so that the alerts fed to human analysts are only those that are deemed credible.
They can also be used to sort and catalog these alerts in a meaningful way, using:
- User analytics – Determining what typical user behavior looks like versus abnormal behavior. What does a user’s average day look like, and what kind of job function do they perform? If you bring these elements together, you’re almost able to recreate a day in the life of someone’s job. So, one way you could use advanced analytics is to profile baseline user behaviors.
- Computer analytics – in a similar vein, think of the computer itself as a user. For instance, in looking at large server farms, what does normal look like? What do each of the different servers typically do? What do they typically not do?
Best practices for getting started
As you begin your advanced analytics journey, here are some tips to keep in mind.
- Make more data readily available for analytics. You can start this process by going to the data leads for each department and asking them to share data. That could include working with the desktop team to get all of their endpoint detection and response (EDR) data or working with the Configuration Management Database (CMBD) team to obtain all of the asset inventory information.
- Restructure low-priority data so that it is more cost-effective to store, or interweave it with other data points where it can add value.
- Leverage a data lake for storage where data does not need to be rehydrated for access.
- Use a data fabric, in which data management and access can be better automated and integrated.
Embracing advanced analytics
Embarking on an advanced analytics journey is vital for organizations facing data overload and limited resources. Focusing on three areas — understanding network assets, using “simple” queries and reducing noise — drives valuable insights, enhances efficiency and mitigates risks. By applying advanced analytics techniques, organizations can filter out false positives, catalog alerts effectively, and provide human analysts with credible and relevant threats. In adopting these strategies, organizations can navigate the challenges, extract insights and optimize analytics capabilities. Advanced analytics is the key to unlocking the power of data and staying ahead in today’s data-driven landscape.
About the Author
Amish Amin, executive director, security and analytics, Comcast. Amish leads the end-to-end data pipeline for cyber threat detection within Comcast. From developing high-value datasets to fine-tuning AI techniques, Amish’s team wears many hats. During his tenure at Comcast he has been awarded “Data Science Manager of the Year” by Snowflake. Prior to this role Amish was the Director of Advanced Analytics at Nationwide insurance. He holds a bachelor’s degree in Accounting and Economics from The Ohio State University.
Sign up for the free insideBIGDATA newsletter.
Join us on Twitter: https://twitter.com/InsideBigData1
Join us on LinkedIn: https://www.linkedin.com/company/insidebigdata/
Join us on Facebook: https://www.facebook.com/insideBIGDATANOW
Speak Your Mind