Data Security and Privacy by Design

Executive summary

Most data protection failures are not failures of technology. They are failures of design: data collected without a defined purpose, copied into environments nobody classified, accessible to roles nobody reviewed, and retained long after its usefulness ended. Privacy by design and security by design reverse that pattern by making protection a property of how data is modeled, stored, moved and used, decided at the point of design rather than negotiated at the point of incident.

This paper takes a data-centric view. Perimeter and endpoint security remain necessary, but the asset that regulators, customers and attackers care about is the data itself, and data moves. A control set that travels with the data, triggered by its classification and enforced wherever the data lands, is the approach that scales across cloud platforms, analytics environments and AI systems, so the paper organizes controls by lifecycle stage, from collection through disposal.

• Principles come first: privacy by design and by default, security by design, least privilege, defense in depth, zero trust, data minimization and purpose limitation.
• Frameworks provide the anchors: ISO/IEC 27001, 27002 and 27701, and the NIST Cybersecurity and Privacy Frameworks for outcome-based mapping.
• Data-centric controls do the work: classification, encryption and key management, tokenization and masking, pseudonymization and anonymization, and secure deletion.
• Access control is where design meets daily use: role-, attribute-, policy- and purpose-based models, row- and column-level security, and disciplined privileged access.
• Detection, response and privacy engineering close the loop: audit logging, data loss prevention, breach notification, records of processing, impact assessments and data subject rights.

The operating model matters as much as the controls. The paper describes how the chief information security officer (CISO), the data protection officer (DPO), the chief data officer (CDO) and business data owners share control ownership within the Three Lines Model, which metrics show whether the design is holding, and which pitfalls most often undo good intentions. It closes with a staged maturity path, from reactive protection to protection embedded by default, and concrete first steps.

Why protection must be designed in

Protection that is designed into data is cheaper, more reliable and more defensible than protection that is bolted on after the fact.

Every organization already protects its data in some way: firewalls, passwords, backups, a privacy notice on the website. What distinguishes a mature organization is not the presence of these measures but the point at which protection decisions are made. In the reactive pattern, a new data set is created, copied and shared, and only later does someone ask who may see it, how long it should be kept and whether it contains personal data. In the by-design pattern, those questions are answered when the data set is specified, and the answers are enforced automatically wherever the data travels.

The scope is deliberately data-centric. Network, endpoint, application and identity security are prerequisites, referred to where data controls depend on them but not covered. The subject is the set of controls that attach to data itself: how data is classified, encrypted, tokenized, masked, accessed, logged, retained and destroyed, and how those controls are governed. Security and privacy are treated together because, in practice, they share the same inventory, the same classification scheme, many of the same controls and the same incident process. Regulation, chiefly the General Data Protection Regulation (GDPR), is cited for orientation; obligations vary by jurisdiction and sector and should be confirmed with counsel.

Principles of security and privacy by design

Principles are the criteria an architect applies when a design choice is ambiguous and the tests a reviewer applies when a design is challenged. The principles below recur throughout this paper, in the order in which they are usually invoked.

Privacy by design and by default

Privacy by design embeds privacy protection into the design of systems, processes and infrastructure rather than treating it as a compliance check at the end. Its seven foundational principles were articulated by Ann Cavoukian while she served as Information and Privacy Commissioner of Ontario. Written for information systems in general, they apply with unusual directness to data platforms, where defaults, lifecycle and transparency are design decisions made daily.

GDPR Article 25 turned the idea into a legal obligation for controllers. Article 25(1) requires appropriate technical and organizational measures, such as pseudonymization, designed to implement the data protection principles effectively, both when the means of processing are determined and during the processing itself. Article 25(2) adds data protection by default: only personal data necessary for each specific purpose should be processed, an obligation that covers the amount of data collected, the extent of processing, the storage period and accessibility. For a data platform, that is a direct instruction about default schemas, default retention and default access.

Security by design, least privilege and defense in depth

Security by design is the parallel discipline for confidentiality, integrity and availability: threat modeling at design time, secure defaults, controls that fail closed, and the assumption that every component will eventually be misconfigured or compromised. Two long-standing principles do most of the work. Least privilege grants each identity, human or machine, only the access its current task requires, for no longer than the task lasts. Defense in depth layers independent controls so that the failure of one does not expose the data: an attacker who defeats network segmentation still meets authentication, authorization, encryption and monitoring.

Zero trust applied to data

Zero trust is an architectural posture, described in NIST Special Publication 800-207, in which no implicit trust is granted to a user account or asset based solely on its physical or network location; every access is authenticated, authorized and continuously evaluated against policy. Applied to data, zero trust means that the data platform does not assume a query is legitimate because it arrived from inside the corporate network or from an approved tool. It evaluates the identity, the device posture, the sensitivity of the data requested and the stated purpose, and grants the narrowest access that satisfies the request. Data classification and fine-grained policy enforcement are therefore prerequisites for zero trust, not optional extras.

Minimization and purpose limitation

GDPR Article 5 sets out the principles relating to processing of personal data: lawfulness, fairness and transparency; purpose limitation; data minimization; accuracy; storage limitation; integrity and confidentiality; and, in Article 5(2), accountability. Two of these shape data design more than any other. Purpose limitation requires that personal data be collected for specified, explicit and legitimate purposes and not further processed in a manner incompatible with them. Data minimization requires that data be adequate, relevant and limited to what is necessary for those purposes. Together they argue against the reflex to collect everything in case it is useful later, and they give data architects a principled basis for narrowing schemas, shortening retention and challenging requests for unrestricted access.

Data that is never collected cannot be breached, subpoenaed, misused or forgotten about. Minimization is the cheapest control there is.

Frameworks and standards as anchors

Principles tell a team what good looks like; frameworks and standards tell it how to organize the work and how to demonstrate it to auditors, regulators and customers. Four anchors are used by most large organizations, and a data management function should understand how its own artifacts map onto each.

ISO/IEC 27001, 27002 and 27701

ISO/IEC 27001 specifies the requirements for an information security management system (ISMS): context and scope, leadership, planning (including risk assessment, risk treatment and a statement of applicability), support, operation, performance evaluation and improvement. Its Annex A lists reference controls that ISO/IEC 27002 explains with implementation guidance, organized as 93 controls in four themes: organizational, people, physical and technological.

For data management, the relevant controls include classification and labeling, access control and privileged access rights, use of cryptography, logging and monitoring, data masking, data leakage prevention, information deletion and supplier relationships. ISO/IEC 27701 specifies requirements and guidance for a privacy information management system (PIMS), distinguishing the obligations of controllers of personally identifiable information (PII) from those of processors. Earlier editions were written as an extension to ISO/IEC 27001 and 27002; the standard has since become a standalone management system standard that can be certified in its own right, without an ISO/IEC 27001 certificate. It remains aligned with ISO/IEC 27001, so an organization that already runs an ISMS can integrate the two rather than build a separate privacy program; organizations certified to the earlier edition should confirm transition arrangements with their certification body.

NIST Cybersecurity Framework 2.0

The NIST Cybersecurity Framework (CSF) 2.0 organizes cybersecurity outcomes into six functions: Govern, Identify, Protect, Detect, Respond and Recover. Govern, added in version 2.0, makes risk strategy, roles, policy, oversight and supply chain risk management outcomes in their own right. Because the framework is outcome-based and technology-neutral, it is a useful common language between a security team that thinks in controls and a data team that thinks in assets and flows. Data inventories, classification and life cycle management, including secure destruction under the retention policy, serve Identify; encryption, access control, masking and the generation of logs serve Protect; monitoring of access and data movement serves Detect; and the breach process serves Respond and Recover.

NIST Privacy Framework

The NIST Privacy Framework applies a similar structure to privacy risk, with five functions in version 1.0. Identify-P covers the inventory and mapping of data processing and the assessment of privacy risk. Govern-P covers policy, roles, risk strategy and training. Control-P covers managing data processing in line with risk, including honoring individuals' preferences, destroying data according to policy, audit logging that respects minimization, and disassociated processing such as pseudonymization. Communicate-P covers transparency with individuals and other parties, including notification of a privacy breach. Protect-P covers access control, data security, protection against data leaks, and response and recovery planning. The explicit overlap with the security framework tells both teams which controls they share and which they do not.

The mapping is not an end in itself. It lets the data function show a certification auditor or a regulator where the inventory, the classification scheme and the lineage graph sit in a recognized control structure, and it lets the security and privacy functions see which of their outcomes depend on data management artifacts they do not own.

Data-centric controls

Data-centric controls protect the data itself, so that protection survives copying, movement and the eventual compromise of any single system.

Classification as the trigger

No data-centric control can be applied consistently until the organization knows what each data set contains and how sensitive it is. Classification assigns each data set, and ideally each column or field, to one of a few sensitivity levels, typically public, internal, confidential and restricted, with an independent marking for personal data, for the special categories that the GDPR protects most strictly (such as health data) and for other sensitive types such as financial account data.

Labeling attaches that classification to the data as metadata that platforms can read. Once labels exist, the remaining controls can be expressed as rules: restricted columns are encrypted with a dedicated key, personal data outside production is masked, and anything labeled as a special category is excluded from self-service exports. Classification is a discipline in its own right; here it matters as the precondition for everything that follows.

Encryption at rest, in transit and in use

Encryption at rest protects stored data against theft of media, misconfigured storage and unauthorized administrators, and is a default option in most cloud storage and database services. Encryption in transit, usually with Transport Layer Security (TLS), protects data moving between systems and should be enforced for internal traffic as well as external. Encryption in use is the least mature of the three: confidential computing runs workloads inside hardware-based, attested trusted execution environments, so that data stays protected from the host operating system and the cloud operator even while it is processed. It suits high-sensitivity analytics, multi-party computation and jurisdictions where operator access is a concern, rather than every workload.

Encryption is only as strong as its key management: who generates keys, where they are stored, who can use them, how often they rotate and how they are destroyed. Customer-managed keys put key policy, rotation and revocation under the customer's control in a key management service or hardware security module, so that the customer decides which services and identities may use each key and can cut off access by disabling it. Where the threat model includes the cloud provider itself, keys can be held in an external key manager outside the provider's infrastructure, at some cost in complexity and availability. Separating key administration from data administration, and logging every key use, turns encryption from a checkbox into an enforceable control.

Tokenization and masking

Tokenization replaces a sensitive value, such as a payment card number or national identifier, with a surrogate token that has no exploitable meaning, while a separate, tightly controlled vault or cryptographic service can reverse the mapping when a legitimate process needs the original. It is widely used to shrink the footprint of systems that handle payment data, because systems that hold only tokens are simpler to secure and can reduce the scope of a Payment Card Industry Data Security Standard (PCI DSS) assessment.

Static masking permanently transforms data when it is copied, for example when production data is provisioned to a test environment, and is the right control where the original value is never needed. Dynamic masking transforms data at query time according to the requester's entitlements, so that a service agent sees the last four digits of an account number while a fraud analyst sees the whole value. Dynamic masking preserves a single copy of the data but depends entirely on the quality of the access policy that drives it.

Pseudonymization versus anonymization

The distinction matters legally and technically. GDPR Article 4(5) defines pseudonymization as processing personal data so that it can no longer be attributed to a specific individual without additional information that is kept separately and protected by technical and organizational measures. For any organization that holds that information, or could reasonably obtain it, pseudonymized data is still personal data, as Recital 26 makes clear. The technique reduces risk and is explicitly encouraged in Articles 25 and 32, but it does not remove obligations.

Anonymization renders data such that the individual is no longer identifiable, and Recital 26 states that the principles of data protection do not apply to anonymous information. It also sets the test: account for all the means reasonably likely to be used to identify a person, by the controller or by anyone else, considering objective factors such as cost, time and available technology. The Court of Justice of the European Union has held that identifiability is assessed from the perspective of the party holding the data: pseudonymized data passed to a recipient with no means reasonably likely to re-identify the individuals may not be personal data for that recipient, while it remains personal data for the controller that holds the key, whose own obligations, such as telling individuals who will receive their data, are unaffected. Anonymization is therefore a documented and revisited judgment about re-identification risk in a specific context, not a label applied by removing names, and any de-identification label should record whose perspective it reflects and the assessment behind it.

Several techniques support that judgment, and data leaders should understand each at concept level even if specialists implement them. k-anonymity generalizes or suppresses quasi-identifiers such as postal code and birth date until each combination is shared by at least k records, so that no record stands alone. Its weakness is that every record in a group may share the same sensitive value, which is why refinements that require diversity within each group, such as l-diversity and t-closeness, were developed.

Differential privacy adds calibrated statistical noise to query results or released statistics so that the presence or absence of any single individual changes the probability of any result by at most a bounded factor, set by a privacy-loss parameter whose cumulative use is tracked as a privacy budget. It is well suited to aggregate statistics and poorly suited to record-level release. Synthetic data, generated from a statistical model of the original, can serve testing and exploration but inherits both the biases and, if poorly generated, the disclosure risks of its source.

Secure deletion

Retention policies are only credible if deletion actually happens, and deletion in distributed data platforms is harder than it looks: data persists in backups, replicas, caches, search indexes, data lake snapshots, feature stores and the exports of downstream consumers. Secure deletion therefore combines a lineage-informed inventory of every copy; deletion procedures appropriate to the medium, including cryptographic erasure, where destroying every copy of the key renders encrypted data unrecoverable; and evidence that deletion occurred. Where backups cannot be selectively edited, regulators such as the UK Information Commissioner's Office accept putting the data beyond use: be clear with individuals about what happens to backups, never restore or use the deleted data, and let the backup expire on its established schedule.

Access control models

Access control is where design principles meet daily use. It is also where breaches involving insiders or stolen credentials do their damage, because an attacker who holds a legitimate identity inherits all of its entitlements. The choice of access model determines how precisely the organization can express least privilege and how much effort it takes to keep entitlements current.

Role-, attribute- and policy-based control

Role-based access control (RBAC) assigns permissions to roles and roles to users. It is simple to understand and audit, and it works well where the population of roles is small and stable. Its failure mode is role explosion: as exceptions accumulate, organizations end up with thousands of roles, many held by a single person, and the model loses its auditability. Attribute-based access control (ABAC), described in NIST Special Publication 800-162, evaluates attributes of the user (department, clearance, location), the resource (classification, owner, jurisdiction) and the environment (time, device, network) against rules, which allows a few rules to replace many roles.

Policy-based access control (PBAC) builds on ABAC by expressing rules as centrally managed, versioned policies, decoupled from any single platform, that a policy engine evaluates at request time. In practice most organizations combine the three: coarse roles for entitlement to a domain, attributes for sensitivity and jurisdiction, and central policies for rules that must be consistent everywhere.

Row- and column-level security

Fine-grained controls operate inside a data set. Column-level security restricts specific fields, typically those labeled as sensitive, and pairs naturally with dynamic masking, so that an unauthorized user sees a masked value rather than an error. Row-level security filters records according to the requester's attributes, so that a regional manager sees only their region or a clinician sees only their patients. Both are native features of most analytical platforms, and both depend on consistent labels and attributes: a row filter keyed on a region code is only as reliable as the reference data that populates the code.

Purpose-based and consent-aware access

Most access models answer who may see what. Privacy law also asks why. Purpose-based access control requires the requester, or the application acting for them, to declare a processing purpose, and the policy engine checks that purpose against the purposes for which the data was collected and against the individual's recorded preferences. Consent-aware access extends the idea to data whose lawful basis is consent: records of individuals who have withdrawn consent for marketing are filtered out of marketing queries automatically, rather than relying on each analyst to remember a suppression list. Both patterns require purpose and consent to be captured as structured metadata at collection and propagated through lineage, which is why they are rare in organizations whose consent records live only in a web form database.

Identity integration, privileged access and break-glass

Data access controls inherit their trustworthiness from the identity system. Single sign-on, multi-factor authentication, automated joiner-mover-leaver processes and a single authoritative source of user attributes are prerequisites; a data platform with its own local accounts is an audit finding waiting to happen. Privileged access management (PAM) governs administrator, service and database owner accounts: credentials are vaulted and rotated, sessions are brokered and recorded, and standing privileges are replaced by just-in-time elevation that expires. Break-glass procedures provide a controlled path for emergency access when normal approval is impossible, for example during an outage at night. A sound design pre-stages the credentials, requires a reason, alerts the control owner immediately, records everything and triggers a mandatory review on the next business day.

Detection and response

Preventive controls will fail somewhere, at some point. A by-design posture therefore plans for detection and response with the same rigor as prevention, and it does so at design time, because the logs, classifications and contacts that an incident team needs cannot be created during the incident.

Audit logging and monitoring

Audit logging records who accessed which data, when, from where and, where the platform supports it, for what declared purpose. The design decisions are scope (every access to restricted data, sampled access to lower tiers), retention (long enough to investigate a slow-moving incident, short enough to respect storage limitation for the personal data the logs themselves contain), integrity (logs written to a store that administrators of the source system cannot alter) and centralization (a security information and event management platform that correlates data access with identity and network events).

User and entity behavior analytics builds on these logs to flag departures from normal patterns: an analyst exporting ten times their usual volume, a service account querying tables it has never touched, access at an unusual hour from an unusual location. Classification is what makes these alerts actionable, because an anomaly against restricted data deserves a different response from an anomaly against public reference data.

Data loss prevention

Data loss prevention (DLP) inspects data in motion and at rest for content that matches sensitive patterns or carries sensitivity labels, and blocks, quarantines or alerts on movements that violate policy, such as emailing a customer list to a personal address or uploading a labeled document to an unsanctioned service. DLP works best when it reads the same labels that drive access control, so that policy is defined once. Its common failure modes are tuning (too many false positives and the alerts are ignored; too few rules and the exfiltration goes unseen) and scope (coverage of email and endpoints but not of the analytics platform, which is where the largest extracts originate).

Incident response and breach notification

An incident response plan for data defines severity levels, the response team, decision rights for containment (who may take a system offline), forensic steps that preserve evidence, and communication paths to legal, privacy, communications and executive management. For personal data, regulatory deadlines shape the plan. Under GDPR Article 33, a controller must notify the competent supervisory authority of a personal data breach without undue delay and, where feasible, not later than 72 hours after becoming aware of it, unless the breach is unlikely to result in a risk to the rights and freedoms of individuals. A later notification must give the reasons for the delay, and a processor must notify the controller without undue delay.

Article 34 requires communication to the affected individuals without undue delay when a breach is likely to result in a high risk to them, with narrow exceptions, notably where encryption or similar measures have rendered the data unintelligible to unauthorized persons. Other regimes set their own thresholds and clocks, some of them shorter: the EU NIS2 Directive, for example, requires essential and important entities to give an early warning of a significant incident within 24 hours, and US state breach laws and sector rules set their own terms. Amendments to the GDPR's own threshold and deadline have also been proposed, so the plan should be built around the strictest applicable clock, with the text in force confirmed by counsel.

The 72-hour window is achievable only if the design work has been done. The team must be able to determine quickly which data sets were affected, what they contained, whose data it was, whether it was encrypted and with which keys, and which processors or recipients hold copies. Every one of those answers comes from data management artifacts: the inventory, the classification, the lineage graph, the key registry and the processor register.

1. Detect and triage. Confirm the event, assign severity using the classification of the data involved, and record the moment of awareness, because the notification clock starts there.
2. Contain and preserve. Stop the loss, revoke credentials or keys, and preserve logs and images for investigation before remediation alters them.
3. Assess. Identify affected data sets, data subjects, categories and volumes from the inventory and lineage, and determine the likely risk to individuals.
4. Notify. Inform the supervisory authority and, where high risk exists, the affected individuals, with the content the regulation requires; document any decision not to notify and its reasoning.
5. Recover and learn. Restore services, confirm the vulnerability is closed, update controls and the risk register, and record the breach in the internal breach log required by Article 33(5).

Tabletop exercises

A plan that has never been rehearsed is a document, not a capability. Tabletop exercises walk the response team through a realistic scenario (a misconfigured storage bucket exposing a customer extract, a ransomware event encrypting a data lake, a departing employee downloading a model training set) and test decisions, contacts and timings without touching production. The useful outputs are the gaps: a lineage graph that could not say which downstream systems held the exposed data, an on-call roster with no privacy representative, a key management service whose administrators were all in one time zone. At least one exercise a year, with the data owners of the most sensitive domains taking part, is a reasonable baseline.

Privacy engineering practices

Privacy engineering turns legal obligations into repeatable artifacts, workflows and controls that data teams can own and auditors can test.

Inventory, mapping and records of processing

The foundation is knowing what personal data the organization holds, where it came from, why it is processed, where it flows and who receives it. A data inventory lists data sets and their attributes; data flow maps show movement between systems, processors and jurisdictions; and the record of processing activities required by GDPR Article 30 documents, for each activity, the purposes, the categories of data subjects and personal data, the recipients, international transfers and, where possible, the envisaged time limits for erasure and a general description of security measures. A record kept as a spreadsheet separate from the data catalog soon goes stale. The sustainable pattern is to generate it from catalog metadata: processing activities link to data sets, data sets carry classification and retention, and lineage supplies the flows and recipients.

Data protection impact assessments

GDPR Article 35 requires a data protection impact assessment (DPIA) before processing that is likely to result in a high risk to individuals. It names three cases that always need one: systematic and extensive evaluation of personal aspects, including profiling, that drives decisions with legal or similarly significant effects; large-scale processing of special categories of data or of criminal conviction data; and systematic monitoring of a publicly accessible area on a large scale. Supervisory authorities publish further lists.

A DPIA describes the processing and its purposes, assesses necessity and proportionality, assesses risks to individuals and sets out the measures that address them. Where the assessment shows that high risk would remain without further mitigation, Article 36 requires prior consultation with the supervisory authority. Done well, a DPIA is a design review: it is triggered by the intake process for new data products and AI use cases, it draws on the inventory and classification, and its mitigations become backlog items with owners rather than paragraphs in an archived document.

Consent, preferences and data subject rights

Where consent is the lawful basis, it must be freely given, specific, informed and unambiguous, and as easy to withdraw as to give. Consent and preference management therefore needs a system of record that captures what was agreed, when, through which version of the notice and for which purposes, and that propagates changes to every system that processes the data. GDPR Articles 15 to 22 give individuals rights of access, rectification, erasure, restriction of processing, data portability and objection, and protection against decisions based solely on automated processing that have legal or similarly significant effects. Article 12 requires a response without undue delay and within one month, extendable by two further months where the complexity and number of requests make it necessary.

Data subject request fulfillment at scale requires an intake channel, identity verification, an automated search across the inventory, workflow to data owners for review, redaction of third-party data and an audit trail. The erasure right in particular tests the secure deletion capability described earlier: an organization that cannot find every copy cannot honor the request.

Retention and processor management

Storage limitation requires that personal data be kept in identifiable form no longer than necessary for its purposes, which in practice means a retention schedule that assigns a retention period and a legal basis to each record class, enforced by the platform and reconciled with legal holds. GDPR Article 28 governs the use of processors: a controller may use only processors that provide sufficient guarantees, under a contract that obliges the processor to act only on documented instructions, ensure confidentiality, implement security measures, assist the controller with its obligations, delete or return the data at the end of the service, submit to audits and engage sub-processors only with the controller's authorization. Vendor management for data therefore needs a processor register linked to the inventory, due diligence proportionate to the sensitivity of the data, and monitoring of sub-processor changes.

Privacy-enhancing technologies for analytics and sharing

Privacy-enhancing technologies (PETs) allow value to be extracted from data while reducing the exposure of individuals. Beyond the pseudonymization, anonymization and differential privacy already described, the main families are data clean rooms, controlled environments in which two or more parties compute agreed aggregate results over their combined data without either seeing the other's records; secure multi-party computation and homomorphic encryption, cryptographic methods that compute on data while it remains protected, practical for narrow workloads; federated analytics and learning, in which computation is sent to the data and only aggregated results or model updates return; and trusted execution environments, the hardware basis of confidential computing.

Choosing among them means matching the use case, the threat model and the acceptable loss of accuracy, and a PET is a mitigation to be recorded in the DPIA, not a reason to skip it. The artifacts described in this section, from the record of processing and the DPIA log to the consent record, rights workflow, retention schedule and processor register, each need a named owner and belong beside the catalog rather than in separate spreadsheets.

Security and privacy for analytics and AI

Analytics platforms and artificial intelligence (AI) systems concentrate the organization's most valuable data in environments designed for exploration rather than control. Training sets are assembled from many sources, feature stores serve the same attributes to many models, notebooks make copies by default, and generative systems add prompts, retrieved documents and model outputs as new data flows. The by-design principles apply unchanged; what changes is the number of places where data can leak and the difficulty of seeing them.

Training data and feature stores

Data does not lose its classification when it becomes a training set. Personal data in a feature store is still personal data, and a model trained on special category data is subject to the same purpose limitation as its source. The controls follow from that fact. Training and evaluation sets are registered as data sets with an owner, a classification inherited from their sources and a documented lawful basis for the training purpose.

Raw sensitive attributes are minimized or pseudonymized before data enters widely shared training environments, with re-identification keys held separately. Feature stores enforce column-level access and dynamic masking like any other serving platform, so that a model team sees the features it is entitled to and nothing else. Golden evaluation sets are access-controlled, because a leaked test set contaminates every future evaluation.

Prompts, retrieval and vector stores

Generative systems create flows that traditional controls do not see. Prompts carry whatever the user typed, including customer data and confidential material; retrieval-augmented generation pulls documents into the model's context; and outputs may reproduce sensitive content. Three controls matter most. A logging policy decides what is retained from prompts and outputs, for how long, who may read the logs and whether they may be used to improve the system, aligned with the retention terms of any external model provider.

Permission-aware retrieval enforces the asking user's access rights at query time, so that an assistant never surfaces a document the user could not open directly; indexing content into a shared vector store without its permissions is among the most serious, and most avoidable, design errors in enterprise AI. Vector stores are classified at the level of the content they encode, because embeddings can reveal information about the text behind them, and they need deletion procedures so that a withdrawn document or an erasure request reaches the index.

Model-specific risks

Models introduce risks that have no equivalent in a database. Membership inference attempts to determine whether a particular individual's record was in the training set by observing how confidently the model responds. Model inversion attempts to reconstruct sensitive training attributes from model outputs. Training data extraction coaxes a generative model into reproducing memorized text. Prompt injection plants instructions in content the model reads, for example a document in a retrieval corpus, to make it disclose data or take actions the user did not intend.

None of these is fully solved. The practical mitigations are minimization of personal data in training sets, differential privacy for models trained on sensitive populations where accuracy permits, rate limiting and output filtering on inference endpoints, treating retrieved content as untrusted input, and recording the residual risk in the DPIA or AI risk assessment so that it is accepted knowingly rather than discovered later.

Operating model and metrics

Security and privacy by design fails most often at the organizational seams. Security reports to a CISO and thinks in controls and threats; privacy reports to a DPO or general counsel and thinks in obligations and rights; data management reports to a CDO and thinks in assets, definitions and flows. Each depends on artifacts the others own, so the operating model assigns ownership of each control explicitly and connects the three functions through shared artifacts rather than through committees.

Control ownership

Each control has an accountable owner who sets its policy, an operator who runs it and a source of evidence that it works. The accountable owner is rarely the operator: the DPO owns the retention policy, the platform team enforces it, and the data owner decides the retention period for a specific data set. Writing this down in a control ownership matrix, kept in the data catalog beside the classification scheme, ends most arguments about who should have prevented an incident.

The Three Lines Model

The Institute of Internal Auditors' Three Lines Model (an update of the earlier three lines of defense) gives shared ownership a shape that boards and auditors recognize. First-line roles own and manage risk: data owners, stewards and platform teams apply the controls and fix issues at source. Second-line roles provide expertise, monitoring and challenge: typically the privacy office, whose DPO must be able to act independently under GDPR Article 38, and an information security risk function. Where the CISO's teams run controls day to day, they sit in the first line, and a separate security risk function provides the second-line view. Internal audit, the third line, gives the governing body independent assurance that controls are designed and operating effectively. The CDO office usually sits in the first line as an enabler, although regulated firms sometimes add a second-line data risk function. Whatever the placement, no function should both operate a control and provide the independent assessment of it.

Metrics

Metrics show whether the design is holding. The useful ones measure coverage and timeliness rather than activity, and they are reported together to the governance council and the risk committee, because a strong result in one metric that hides a weak result in another is how programs drift.

Pitfalls and the maturity path

The failure patterns are consistent across industries, and most of them are organizational before they are technical. Recognizing them early is far cheaper than discovering them in an incident review or a supervisory finding.

Maturity path

Capability is built in stages, and each stage depends on the artifacts of the previous one. The path below describes observable states rather than aspirations. Organizations are usually at different stages for different domains, which is acceptable as long as the most sensitive domains lead.

The reactive stage is where most programs start: controls exist, but they were chosen by the last incident. The defined stage establishes the classification scheme, the policy set, the control ownership matrix and the inventory, usually with partial coverage. The embedded stage is the turning point, when classification labels actually drive encryption, masking, access and retention across operational, analytical and AI platforms. The automated stage expresses policies as code that platforms enforce and generates evidence from the platforms themselves, so that audits read reports rather than interview people. The adaptive stage tunes controls to measured risk and uses privacy-enhancing technologies to enable uses of data that a blunter regime would have forbidden.

Where to start

The first moves are similar in almost every organization, and each produces an artifact that the next depends on.

1. Name the owners. Agree the control ownership matrix among the CISO, DPO, CDO and business data owners, and publish it in the catalog beside the classification scheme.
2. Classify the most sensitive domains first. Settle a small scheme with personal data markings, label the domains that hold restricted and special category data, and measure coverage from the start.
3. Bind labels to controls. In the main analytics platform, make labels drive masking, row and column policies and encryption under managed keys, including in test environments and AI pipelines.
4. Put privacy at intake. Capture purpose and lawful basis when a data set or data product is specified, and make the DPIA a design gate for high-risk processing and new AI use cases.
5. Build breach readiness. Connect the inventory, lineage, key registry and processor register, then run a tabletop exercise with the owners of the most sensitive domains and fix what it reveals.
6. Report coverage, not activity. Publish the metrics above to the governance council and the risk committee on a fixed cadence, starting with classification, masking and deletion evidence.

Conclusion

Security and privacy by design is not a new set of technologies; it is a decision about when protection is decided. Organizations that classify data as it is created, derive controls from classification, keep purpose and consent as metadata that travels with the data, log access against real identities, rehearse their response and share ownership deliberately among security, privacy and data management will find that most of the obligations they worry about are met as a by-product. Those that defer the decisions will keep meeting them in incident reviews.

Protect the data, not the container; decide at design time, not at incident time.

Glossary

Privacy by design

Embedding privacy protection into the design of systems, processes and data practices from the outset, rather than adding it afterward.

Data protection by default

The GDPR Article 25(2) requirement that, by default, only personal data necessary for each specific purpose is processed.

Zero trust

An architectural posture in which no implicit trust is granted on the basis of physical or network location; every request is authenticated, authorized and evaluated against policy.

Classification

The assignment of a sensitivity level and regulatory categories to a data set or field, expressed as metadata that controls can read.

Customer-managed key

An encryption key whose policy, rotation and revocation the customer controls in a key management service or hardware security module.

Tokenization

Replacing a sensitive value with a surrogate token that has no exploitable meaning, with the mapping held in a protected vault or service.

Dynamic masking

Transforming sensitive values at query time according to the requester's entitlements, while a single copy of the data is kept.

Pseudonymization

Processing personal data so that it can no longer be attributed to an individual without separately held additional information; still personal data for any party that holds or can reasonably obtain that information.

Anonymization

Rendering data such that individuals are no longer identifiable by any means reasonably likely to be used; anonymous data falls outside data protection law.

Differential privacy

A mathematical guarantee, achieved with calibrated noise, that including or excluding any single individual changes the probability of any released result by at most a bounded factor.

Record of processing activities

The register of processing activities, purposes, data categories, recipients, transfers and, where possible, retention limits and security measures required by GDPR Article 30.

Data protection impact assessment (DPIA)

The GDPR Article 35 assessment of necessity, proportionality and risk for processing likely to result in a high risk to individuals.

Break-glass access

A controlled emergency access path that pre-stages credentials, requires a reason, alerts the control owner and triggers a mandatory review.

Data loss prevention (DLP)

Tooling that inspects data in motion and at rest for sensitive content or labels and blocks, quarantines or alerts on policy violations.

Three Lines Model

The Institute of Internal Auditors' model distinguishing roles that own and manage risk, roles that provide expertise, monitoring and challenge, and internal audit, which provides independent assurance.