TL;DR
Digital transformation across business units has increased demand on cybersecurity teams who are still predominantly using legacy data platform technologies. The good news is that the very same modern data stacks and AI technologies that have made digital transformation possible for business units can also be leveraged for cybersecurity threat investigation and threat hunting use cases, removing the scalability, cost, and operational overhead challenges of legacy solutions.
Why is Cybersecurity Analytics Technology Adoption Lagging?
Most businesses today have adopted digital transformation strategies and modernized their business models, social culture, and technology stacks. On one hand, this resulted in significant productivity gains, but on the other hand, it increased demand on cybersecurity teams supporting these initiatives while protecting the organization from cyber risk. The reality is that as organizations adopt cloud technologies, enable remote workforces, and more, security must also change and adapt. If security measures don’t match the digital pace, organizations face cyber risks that can impact the success of digital initiatives. It would be reasonable to expect that the success of digital transformation would translate across the organization, including cybersecurity, allowing security initiatives to keep pace with the rest of the company.
Unfortunately, executives have focused less on making cybersecurity teams data-driven, which would help them match security measures to the digital pace. Cybersecurity teams are still predominantly using legacy data platform technologies which were built to process gigabytes of log data in a time when all organizations are generating terabytes of log data. (Even still, many need petabytes or even zettabytes of storage for long-term data retention.) Additionally, security teams are often not equipped with the data engineering skills required to wrangle the current volume of data. Both legacy technology and limited access to advanced data talent are impacting cybersecurity teams’ ability to secure their organizations.
The graph below illustrates the gap between digital transformation in a corporation’s business units and the cybersecurity team. At the beginning of 2020, there was a distinct upturn in digital transformation efforts across corporate America. However, cybersecurity teams often did not take part in this evolution, and today the gap in capabilities is wider than ever. This makes it challenging for security teams to keep up with the resulting increase in log data volume. Cybersecurity teams need to develop a data-driven culture to close this gap.
In an effort to maintain a secure environment while adopting new cloud technologies, enabling remote workforces, and more, organizations have added new security point solutions for new specific security use cases such as UEBA, EDR, NDR, etc. These new solutions capture high-volume log data across the corporate network, cloud implementations, and remote endpoints and store the data in their respective proprietary data stores. This results in an increase of data silos which are rarely integrated and prevent security analysts’ visibility across the corporate network and user activity, also complicating the threat analysis process. This leads to low cybersecurity productivity and efficacy.
Cybersecurity Analytics and Modern Data Stacks
The good news is that the very same modern data stacks and AI technologies that have made digital transformation possible and successful for business units can also be leveraged for cybersecurity use cases. Now, the same way e-commerce personalization and recommendation engines tap into big data’s massive processing power to draw insights from vast amounts of data, cybersecurity can leverage these modern platforms to gain actionable insights into corporate cybersecurity risk. For example, there is a growing ecosystem of Snowflake partners who have built cybersecurity solutions that leverage Snowflake’s leading data cloud for storage and compute.
This new approach to security analytics, building the solution on a cloud-native stack, removes the scalability, cost, and operational overhead challenges security teams are facing when operating legacy solutions. Snowflake, with its near-infinite elastic compute and usage-based billing model, allows users to access the compute they need when they need it, and at a far lower cost than before. Being fully managed, the expensive operational overhead burden security teams carry today is also eliminated.
What are the Data Challenges for Cyber Security Analytics?
The two key challenges are:
1. Loading data to Snowflake and utilizing their compute is not enough in itself to create a security data lake. Collecting and loading data from multiple log sources is an important first step. But without applying a robust data model to the data, you will end up with several separate data tables—in effect data silos within the data store—that are challenging to use for threat investigations, threat hunting, and even monitoring. An open data model will unify, correlate, enrich, and transform event data from all log sources. You can read more about how to bring all your security data together with an open data model here.
2. When you have your security event data in a security data lake and applied your data model, the next challenge is how to extract value from your data. Most data lake platforms, such as Snowflake, can be queried with a SQL query tool such as SQL Workbench. It is also often suggested that using BI Tools like Tableau or Looker is a viable way to build visualizations and dashboards. However, most security analysts in a SOC are not familiar with SQL and do not have the time or skill set to build advanced dashboards.
Enter Elysium Analytics
This is why Elysium Analytics developed a snowflake-native security analytics solution with a turn-key open data model, along with applications for KQL full-text search, out-of-the-box security dashboards, ML-based anomaly detection, alerting, and more. Security teams can now keep pace with corporate digital transformation and implement truly data-driven security analytics for threat investigations, threat hunting, and monitoring. With Snowflake’s data cloud and our open data model, we remove any data engineering requirements, allowing security analysts to focus on investigations and build their use cases. Additionally, our security data lake and open data model can be leveraged by data scientists for advanced use cases and custom applications with native integration of Jupyter Notebook and developer access to our ML pipeline.
Although the Elysium Analytics security data lake solution can be implemented as a “SIEM light” solution with strong alerting capabilities, larger organizations typically implement this solution as a capability that is complimentary to their primary SIEM. This allows them to continue the use of current investments in technology and workflow processes, while at the same time enabling the ability to capture all log data—even DNS logs, proxy logs, and EDR telemetry—and process and analyze every event for actionable insights.
With the rapid adoption of digital transformation and the technology supporting this modernization, it will only be a matter of time before corporations also look to leverage the latest cloud and data science developments in their cybersecurity efforts. Fortunately, there are cybersecurity data analytics vendors such as Elysium Analytics who recognize this, building solutions that come as turn-key solutions, essentially eliminating the need for data scientists and data engineers that also have the required cybersecurity experience.
Contact Elysium Analytics for more information and a demo on how you can implement a security data lake for threat investigations, threat hunting, and monitoring.
