Bolstering Regulatory Compliance Efforts with Real-Time Data Management

One of the most problematic data management issues facing organizations today is the bewildering array of regulatory requirements they face, and not just from within their own country, but often worldwide as well.

A single U.S. organization, for instance, may be subject to data privacy laws in its state of residence, the Fair Credit Reporting Act, IRS regulations, HIPAA provisions, and even foreign laws such as the European Union General Data Protection Regulation (GDPR). Each applicable law or regulation may have its own particular requirements, which at times may be contradictory. Worse yet, the non-compliance penalties (including both damages payable to consumers and administrative fines) are significant. 

One of the most well-publicized examples of high regulatory penalties involved Austrian privacy advocate Max Schrems and his organization NOYB (None Of Your Business). Within hours of the GDPR entering into force, Schrems had filed four complaints against four of the world’s largest companies (Google, Instagram, WhatsApp and Facebook) seeking nearly 8 billion euros in GDPR penalties.

When coupled with the explosion in the amount of data organizations maintain, compliance burdens are not just complex, they also verge on unmanageable. Enter real-time data management. Using real-time data management tools and services, an organization can minimize business disruption caused by compliance efforts and put itself in the best possible position if (or more likely when) issues do arise. 

Why Real-Time Data Management?

The natural question is “Why real-time?” Why can’t an organization simply review data for compliance once a week (or less), rather than using continuous processing? The answer is simple: because many regulations don’t provide grace periods or the opportunity to cure violations. Once a violation has occurred, liability is both possible and, increasingly, probable.

But real-time data management need not be overly burdensome. In fact, it likely will provide long-term efficiencies that offset onboarding time and costs. Using the proper tools, an organization can build a ruleset applicable to all its data. The tools then allow the organization to view data automatically in real-time for compliance, for instance with dashboards that show data aging and compliance status. 

The organization can then purge or process data as necessary. When laws or regulations change, the organization modifies the ruleset rather than reviewing each set of data, simplifying compliance efforts and minimizing the chances that an organization will be subject to non-compliance penalties. 

Of course, any real-time data management tool an organization uses is only as good as the programming underlying it. Thus, the tools an organization uses, whether built in-house or purchased from vendors, must be thoroughly vetted for both operational and security issues, including rigorous testing during all phases of the software development life cycle (SDLC). 

Use of tools such as static application system security testing (SAST) can help ensure implementation of effective security measures during development, minimizing opportunities for hackers to corrupt compliance algorithms and jeopardize compliance efforts. SAST, for example, is a testing process applied early in the SDLC that, according to Cloud Defense, can “prevent new vulnerabilities from passing through the build process.”  

As a concrete example, assume an e-commerce business collects private personal data from a customer during an online transaction and that data is subject to the GDPR. While the GDPR does not contain specific data retention times, it also states that data permitting identification of a “data subject” (i.e. a person) may be kept “no longer than is necessary for the purposes for which the personal data are processed.” 

So, assuming no other regulations are in play and the organization decides to keep data for a certain amount of time after the transaction (for purposes such as returns or refunds), a real-time data management system can assign a deletion date to the data. A dashboard can then be used to show an information systems manager upcoming deletion events so that they can verify deletion of the data if desired, otherwise the data will be deleted according to schedule. 

Why should organizations be concerned about data management?

While corporate accounting, tax and employment regulations each present their own sets of data management issues, one of the areas where data management regulations are most significant is in the protection of personal data, and in particular personal financial and health data. 

Many of the principles at the heart of data protection hearken back to the Guidelines on the Protection of Privacy and Transborder Flows of Personal Data published by the Organization for Economic Cooperation and Development in 1980. 

The OECD Guidelines provided eight guiding principles, several of which relate to limitations on collection and use of personal data and several of which deal with how “data processors” (i.e. organizations) should handle data internally once they have collected it. 

These principles have been widely applied in the implementation of data privacy laws and regulations by countries and multinational groups since 1980, and have been expanded to include additional limitations such as limits on how long organizations should or can maintain data. 

Two of the most discussed data privacy regimes – partially because they are some of the most far-ranging and burdensome – are those of the European Union (GDPR) and California (CCPA and CPRA). Both of these regimes have some basis in the original OECD Guidelines, although they have evolved significantly in the past decade and continue to do so. Both also have severe potential penalties for non-compliance. And, as the classic saying goes, “Ignorance of the law is no excuse” – liability does not require intent under either set of regulations. 

Far from being limited to GDPR and CCPA, strict data privacy protections, along with severe penalties for violating those protections, have become more and more prevalent worldwide. The Personal Information Protection and Electronic Documents Act (PIPEDA) in Canada, the Privacy Act 1988 in Australia, the Act on the Protection of Personal Information (APPI) in Japan are just a few examples of data privacy regimes with strict requirements about minimizing how long personal information is maintained. 

Even China is in the process of implementing comprehensive personal data protections with its proposed Personal Information Protection Law. Indeed, according to the United Nations Conference of Trade and Development, 66% of countries in the world have enacted data protection laws and another 10% are in the process of doing so.

These regulations contain many common features that support the use of real-time data management tools and services.

Extraterritorial applications

While it may seem counterintuitive to many, there are international regulations that frequently apply to businesses that think they are not applicable. The GDPR, for example, has extraterritorial application; that is, even though it is an EU regulation, it specifically states that it applies to any organization anywhere in the world, so long as that organization collects data from EU citizens. And it is not just giant multinational organizations like Microsoft and Google that are subject to regulations like the GDPR, although they are certainly popular targets.

The California Consumer Privacy Act (CCPA) has similar provisions allowing it to apply to businesses that do not reside in California, although it is restricted to larger businesses and businesses that collect information from at least 50,000 California citizens. In one of the few examples of easing of data protection rights, the upcoming California Privacy Rights Act, which replaces the CCPA, increases this to 100,000.  

Significant fines for violations

The GDPR has been an issue of great concern among companies worldwide since it went into effect in May 2018, precisely because it mandates significant fines in the event of a violation. 

The GDPR provides two levels of administrative fines, dependent on the severity of a violation, with the lower-level violation having maximum fines of up to the greater of 10 million euros or 2% of annual worldwide turnover (revenue), and the higher-level violation twice that. The potential for large fines caused significant business disruption. As just one example, many major international news organizations quickly blocked European readers from their websites in May 2018. Many still do.  

The danger of GDPR proceedings is not just a phantom concern; indeed, there have been more than 59,000 data breaches recorded in GDPR proceedings since 2018, with fines ranging from just a few euros to the 50 million euro fine against Google by French authorities resulting from the Schrems case. The concern is quite real. 

Administrative fines for CCPA violations are substantially lower at $2500-$7500 per violation, depending upon the nature and severity of the violation. And the CCPA also provides a private right of action where individuals may seek damages between $100 and $750 per consumer whose data is mishandled. These numbers can add up quickly.

Vague or Conflicting Data Management Provisions

The GDPR is also problematically vague. As noted above, there are no specific data retention provisions in the GDPR, just the general principle that data should be retained for as short a time as is necessary. The CCPA, in contrast, requires businesses to inform data subjects of retention periods when collecting data, and the businesses must adhere to those retention periods. 

The CCPA specifically recognizes that there may be different retention periods for different data. And tax and employment data is likely subject to different retention periods. All of which suggests that an automated real-time process for data management is not only desirable, but a necessity.

Real-time data management is not only useful with respect to retention and deletion of data collected from consumers, but also to track requests made by consumers for access to their personal data, or correction or deletion of it, all of which are specific rights provided by both the GDPR and the CCPA.

Audit Provisions

Some regulations, such as the CPRA, also require regular cybersecurity audits and risk assessments. Application of real-time data management methods would go a long way towards satisfying concerned regulators reviewing periodic audits. 

The compliance requirements for an organization are difficult enough when it is a local business with local customers. They are exponentially more complex and difficult for multi-state and multinational companies. Fortunately, real-time data processing is an effective compliance aid for businesses of all sizes.

Conclusion

Organizations today face a bewildering, and often conflicting, set of compliance challenges with respect to collection, processing, maintenance and deletion of data, particularly when that data contains sensitive personal information such as credit card numbers or health information. 

Coupled with the enormous amount of data maintained by even small organizations, it is clear that organizations can no longer effectively rely on manual efforts at compliance, nor can they simply engage in periodic processing, whether manual or automated. Application of real-time data management tools and services provide an effective solution to many of these compliance issues.

You will also like

---

About the author

Nahla Davies is a software developer and tech writer. Before devoting her work full time to technical writing, she managed—among other intriguing things—to serve as a lead programmer at an Inc. 5,000 experiential branding organization whose clients include Samsung, Time Warner, Netflix, and Sony.