Long Term Cybersecurity Event Retention for New York State Compliance


Author: Gregory Hedge
August 14, 2018

The regulation known as 23 NYCRR 500 is a set of cybersecurity requirements from the New York Department of Financial Services (NYDFS) for financial and insurance institutions that conduct business within the State. The regulation began March 1, 2017 with implementation deadlines occurring at various dates thereafter. The next deadline is September 3, 2018. One of the requirements of this upcoming milestone requires the covered entities to retain for 3 years “audit trails designed to detect and respond to Cybersecurity Events that have a reasonable likelihood of materially harming any material part of the normal operations of the Covered Entity.” For even the most mature information security operations, 3 years of event data retention can be a challenge. For many organizations, security incident and event management (SIEM) systems, which are collecting and alerting on cybersecurity events in real-time, are deployed as part of a security operation centers. The SOC and its main weapon, the SIEM, are often focused on rapid breach detection and incident response rather than long-term storage.

The ArcSight ESM, the original and industry leading SIEM, is often at the core of these successful security operation centers around the world. The ESM looks through thousands and even tens of thousands of events per second ingested from the logs, alerts and data feeds of a multitude of sources to find the needle in the proverbial haystack. Thus a SIEM is often described being about speeds and feeds but the ArcSight platform also includes many options to achieve New York State’s regulatory compliance for the storage of cybersecurity event data for 3 years. These options include the heart of the ArcSight Data Platform – the Logger, the leading log management and big data solution. The Logger as part of the ArcSight Data Platform consumes data information such as cybersecurity event logs from anywhere and stores it in a unified format.

In addition to its log management capabilities such as reporting, alerting, searching and visualization, a single Logger instance can store years’ worth of data through a high compression ratio of up to 10:1. And multiple Loggers can be peered providing more searching and petabytes of storage capacity. Event data can also be stored off the Logger in a SAN or in a data archive for even more capacity and longer retention.

Regardless of the source or original format, the ArcSight Data Platform (ADP) converts the incoming event data into a single unifying format – CEF, the Common Event Format. ADP normalizes, categorizes and enriches the raw data whether it is Windows, firewall or custom application events into CEF. Since the open ArcSight format has become the de facto industry standard for log data, many of the leading information security solutions natively log their event activity in CEF.

The open architecture of ADP provides the capability to integrate with other storage, big data and search solutions such as the ELK stack, Hadoop, Vertica and Splunk. The ArcSight Data Platform can deliver to these other solutions the events from any or all its sources with all the data unified in the Common Event Format regardless of the storage repository.

These capabilities make the ArcSight Data Platform the best choice to meet the technical challenges of the 3-year retention requirement even if an organization already utilizes Splunk, ELK or the ArcSight ESM. ADP combines an open format and architecture as well as unparalleled capacities for data ingestion and storage.

If you want to learn more about how Castle Ventures can help you achieve NYDFS 23 NYCRR 500 compliance contact us or read more about our application monitoring packages.