Enterprise data architecture identifies the data an organization needs and designs the blueprints that describe how that data is captured, stored, moved, protected and served. Done well, it lets a delivery team find the authoritative source for the data it needs, choose a sanctioned pattern and inherit the right controls. Done badly, or not at all, it leaves a landscape that grew one project at a time and that every change makes more expensive.
This paper positions the discipline within enterprise architecture, drawing on DAMA-DMBOK, TOGAF and the Zachman Framework, and sets out nine architecture principles with their rationale and implications. It describes the artifacts an architecture function must maintain, compares ten reference patterns from the enterprise data warehouse to data mesh and data fabric, and explains how to select and combine them. Architecture governance, cloud and security considerations, metrics, common pitfalls and a maturity path complete the picture.
Written for Enterprise and data architects, chief data officers, data platform leads, and solution architects who set or apply the blueprints for how an organization stores, moves, protects and serves its data.
Executive summary
Most large organizations did not design their data landscape; they accumulated it. Each application brought its own database, each analytics initiative its own copy of customer and product data, and each integration project its own point-to-point interface. Nobody can say with confidence where the authoritative version of a critical data element lives or what will break when a source changes. Enterprise data architecture is the discipline that replaces accumulation with design.
Following DAMA-DMBOK, this paper treats data architecture as identifying the enterprise's data needs and designing and maintaining the blueprints that meet them. Those blueprints are a connected set of artifacts rather than one diagram: an enterprise conceptual data model organized into subject areas, a map of systems of record and data flows, a small number of reference architectures, a standards catalog and a log of architecture decisions.
- Principles come before patterns: nine principles, each with a rationale and implications, give architects and review boards a stable basis for decisions as technology changes.
- No single pattern serves every need: the paper compares ten patterns, from the warehouse and lakehouse to the streaming backbone, data mesh and data fabric, on purpose, strengths, weaknesses and fit.
- Targets are reached through transition architectures: a roadmap that sequences change, retires technical debt deliberately and records decisions matters as much as the target state.
- Architecture governance and data governance are distinct but inseparable: the design authority decides how structures and flows are built, while the data governance council decides who owns data, what it means and which source is authoritative.
The paper closes with the metrics that show whether an architecture is working, such as reuse, redundancy, time to provision data and standards adherence; the pitfalls that recur in practice, including ivory-tower architecture, pattern-of-the-year adoption and the absence of a transition plan; and a maturity path from an undocumented landscape to largely automated architectural compliance.
Why architecture must be intentional
An enterprise does not have a data architecture because somebody drew one; it has one because thousands of decisions were made about where data lives and how it moves, whether or not anyone wrote them down.
The question is therefore not whether an organization has a data architecture but whether that architecture is intentional. An unintentional architecture is the by-product of project decisions taken one at a time, each reasonable in isolation and collectively expensive. Its symptoms are familiar, and because they are structural, tactical fixes do not remove them.
An intentional architecture does not eliminate complexity; a large organization's data landscape will remain plural. What it provides is a shared frame for that plurality: agreed principles, a vocabulary of subject areas, a designated authoritative source for each kind of data, a small number of sanctioned patterns and a visible route from the current state to a better one. This paper describes how to build and operate that frame.
The treatment stays at the enterprise level. Schema design is covered only as far as architecture needs it, and integration techniques, data mesh and data fabric appear here as patterns within the overall architecture rather than in full operational depth.
An architecture that is not written down still exists; it is simply unmanaged.
Definitions and scope
DAMA-DMBOK describes data architecture as identifying the data needs of the enterprise, regardless of structure, and designing and maintaining the master blueprints to meet those needs; the blueprints then guide data integration, control data assets and align data investment with business strategy. The definition has two halves that should not be separated. Identifying needs is a business activity: which subject areas matter, which decisions depend on them, and what quality, timeliness and protection they require. Designing blueprints is a technical activity whose value comes entirely from answering those needs.
Within DAMA-DMBOK, data architecture is one of the knowledge areas arranged around data governance, alongside data modeling and design, data storage and operations, and data integration and interoperability, among others. It is neither the sum of the platforms an organization owns nor a synonym for data modeling. It is the layer of decisions above both: which data the enterprise needs, where each kind is authoritatively held, how it flows, and which structures and standards every solution must respect.
Place in enterprise architecture
In TOGAF, data architecture is developed in Phase C of the Architecture Development Method (ADM), Information Systems Architectures, together with application architecture, between the business architecture of Phase B and the technology architecture of Phase D. The data and application work within Phase C may run in either order or in parallel, but both follow the business architecture. The sequence encodes a discipline: data architecture is derived from business capabilities and information needs, and it constrains rather than follows the choice of technology. TOGAF also supplies apparatus that data architects borrow: a template that states each principle with its rationale and implications, an architecture repository that records the standards new architectures must comply with, architecture compliance reviews and an architecture board.
The Zachman Framework plays a different role. John Zachman describes it as an ontology rather than a methodology: a classification scheme that arranges descriptive representations of an enterprise in a grid of six interrogatives (what, how, where, who, when and why) against six rows, which correspond to the perspectives of successive audiences, from the executive's view of scope to the operating enterprise itself. For data architects it is a completeness check: the what column reminds the team that an inventory of business things, a conceptual model, a logical model, a physical design and the deployed data are different artifacts for different audiences.
| Framework | What it is | What data architects take from it |
|---|---|---|
| DAMA-DMBOK | A body of knowledge for data management, with governance at the center of its knowledge areas | The definition and scope of data architecture and its links to modeling, integration, storage and governance |
| TOGAF | An enterprise architecture standard built around the Architecture Development Method | The position of data in the ADM, the principle template, transition architectures, compliance reviews and the architecture board |
| Zachman Framework | An ontology that classifies the descriptive representations of an enterprise | A completeness check across audiences and levels of abstraction, especially in the what column |
Levels and boundaries
Enterprise data architecture sits at the top of a hierarchy of increasingly concrete descriptions, shown in the figure below. Solution data architectures apply the enterprise blueprints to a business area or program, data models define the structures of specific stores and interfaces, and the platform of storage, compute, network and security services carries everything above it. Each level constrains the one beneath it and feeds lessons upward.
Enterprise data architecture
Needs, principles, subject areas, systems of record
Solution data architecture
Patterns applied to a business area or program
Data models
Conceptual, logical and physical structures
Platform and infrastructure
Storage, compute, network and security services
Two boundaries deserve explicit statement. Data architecture covers both the operational plane, where transactions are captured and processes run, and the analytical plane, where data is integrated, historized and analyzed; organizations that architect only the analytical plane inherit every inconsistency of the operational one. It also covers unstructured content, streams and externally sourced data, not only relational tables.
Architecture principles
Principles are the stable core of an architecture. Technologies and patterns change every few years; the reasons for preferring one design over another change much more slowly. Following the TOGAF convention, each principle is stated with a rationale and implications, because a principle without implications is a slogan. Nine principles can be remembered, debated and applied in a review; thirty cannot.
| Principle | Statement | Primary implication |
|---|---|---|
| Data is a shared asset | Data belongs to the enterprise, not to the capturing system or team | Access by default within policy; no unjustified private copies |
| Authoritative sources | Each subject area has designated systems of record and distribution points | Other copies are derived, traceable and refreshed |
| Separate storage and compute | Persist data independently of the engines that process it | Open storage formats; elastic, workload-specific compute |
| Decouple through interfaces | Systems exchange data only through published, versioned interfaces | No cross-boundary database access; contracts and change notice |
| Metadata-driven design | Metadata describes structures, flows and controls and drives automation | Catalog as a build-time dependency; lineage captured, not drawn |
| Security and privacy by design | Protection follows classification and is built into every pattern | Classification before provisioning; controls inherited from the platform |
| Fit-for-purpose storage | Match the store to the workload rather than using one store for all | Polyglot persistence within a sanctioned list; integration budgeted |
| Open formats and standards | Prefer open, documented formats and interfaces over proprietary ones | Portability tested; proprietary features used knowingly and recorded |
| Design for change | Assume sources, volumes, regulations and tools will change | Schema evolution, versioning and transition plans are mandatory |
Shared asset and authoritative sources
The first principle addresses the most common failure of accumulated landscapes: data trapped in the system that captured it. Treating data as a shared asset means that the default answer to a legitimate request within policy is yes, that the platform bears the cost of sharing rather than each project negotiating it, and that producing teams are accountable for the usability of what they publish. Ownership is assigned to business domains through data governance; architecture provides the channels through which owned data is shared.
The second principle corrects a persistent misunderstanding. The goal is not a single physical copy of each piece of data, because replication is inherent in distributed systems, caching, analytics and resilience. The goal is that each subject area, or each segment of it, has a designated system of record where data is created and corrected, and designated distribution points from which consumers obtain it. Every other copy is derived, carries lineage to its source and is refreshed rather than independently maintained.
Decoupling, metadata and protection
Separating storage from compute is the common shape of cloud data platforms: data persisted in open formats on durable object storage can be read by several engines, scaled independently of them and retained when an engine is replaced. Decoupling through interfaces applies the same logic between systems. Data crosses a boundary only through a published, versioned interface, whether a service, an event stream or a governed data product, and never through direct access to another system's tables, so interface contracts and change notice periods become architectural standards.
Metadata-driven design recognizes that structures, flows and controls cannot be managed by hand at enterprise scale. Schemas, lineage, classifications, ownership and quality rules are recorded in a catalog that pipelines and platforms read at build and run time, so that lineage is captured by tooling rather than drawn afterward. Security and privacy by design follows directly: when classification is metadata the platform can read, masking, encryption and access policy are inherited by every pattern. That is the architectural form of the obligation in Article 25 of the General Data Protection Regulation (GDPR) to implement data protection by design and by default.
Fit, openness and change
Fit-for-purpose storage accepts polyglot persistence: a relational store for transactions, a document store for flexible product data, a graph store for relationships, a columnar store for analytics and a log for events. The discipline lies in keeping the list of sanctioned stores short and in budgeting the integration that each additional store creates.
Open formats and standards protect the organization's options. Data held in open file and table formats and exchanged through standard protocols can move between engines and vendors, so the cost of leaving is known rather than discovered. Design for change means schema evolution rules, versioned interfaces, replayable pipelines and a transition plan for every significant component, together with a record of why each decision was made, so that successors can tell which constraints still apply.
Architecture artifacts
Blueprints are useful only if they exist, are current and are consulted. An architecture function therefore maintains a small, named set of artifacts, each with an owner, a refresh cadence and a place in the architecture repository. Organizations that attempt to maintain dozens of artifact types find that none of them is trusted.
| Artifact | What it contains | Maintained by | Typical cadence |
|---|---|---|---|
| Enterprise conceptual data model | Subject areas, core entities and relationships in business terms | Enterprise data architect with domain stewards | Quarterly review; change by decision |
| System-of-record map | Authoritative sources and distribution points per subject area | Data architecture with governance sign-off | On every system change |
| Data flow map | Movement of data between systems, planes and parties | Domain architects; generated where possible | Continuous from the catalog |
| Reference architectures | Sanctioned patterns, components, interfaces and controls | Design authority | Annual review |
| Standards catalog | Approved formats, interfaces and products with lifecycle status | Design authority | Semiannual review |
| Architecture decision records | Context, options, decision and consequences of each significant choice | Deciding architect; indexed centrally | At decision time |
Conceptual model and subject areas
The enterprise conceptual data model names the things the business cares about, such as customer, product, contract, location and event, and the relationships among them, in business language and independent of any system. Its first product is the set of subject areas, which become the organizing scheme for ownership, the system-of-record map, catalog navigation and domain boundaries. A workable enterprise model fits on one page; detail belongs in the subject-area and logical models that sit beneath it.
System-of-record and data flow maps
The system-of-record map records, for each subject area and where necessary for each attribute group within it, the system in which data is created and corrected and the points from which it is authoritatively distributed. It is among the artifacts most often requested by auditors, regulators and integration teams, and among those most often missing. The data flow map complements it by showing how data actually moves between systems, planes and external parties; where a catalog captures lineage, the map should be generated rather than drawn.
Reference architectures, standards and decisions
Reference architectures describe the sanctioned way to assemble components for a recurring need, such as an analytical platform, an integration backbone or an external data exchange. Each names the components, their interfaces, the controls that must be present and the situations in which the pattern should and should not be used. The standards catalog, which corresponds to the standards held in a TOGAF architecture repository, lists approved formats, protocols, products and conventions, each with a lifecycle status such as emerging, approved, contained or retiring.
Architecture decision records (ADRs), a lightweight practice popularized by Michael Nygard, are short, numbered documents that record the context of a significant decision, the decision itself and its consequences; many templates also record the options considered. They are the memory of the architecture: principles say what the organization values, and ADRs show how those values were traded off in specific cases. A decision that is not recorded will be relitigated, usually by people who lack the context in which it was made.
Reference patterns
A reference pattern is a proven arrangement of components that solves a recurring data problem. Each pattern below is described by its purpose, strengths, weaknesses and fit. None is universally superior and most enterprises run several; a table at the end of this section summarizes them, and the figure that follows it shows how they combine into one layered architecture.
Operational and analytical planes
The first distinction is between the operational plane and the analytical plane. The operational plane runs the business: transactional systems, customer-facing applications and the integration between them, optimized for low-latency reads and writes of current data with strong consistency within each system. The analytical plane helps the organization understand the business: integrated, historized data organized for query, reporting and machine learning, optimized for throughput and for change over time. Much architectural harm comes from ignoring the boundary, for instance by running analytics against transactional databases or by using the warehouse as an operational integration hub.
Warehouse, lake and lakehouse
The enterprise data warehouse integrates data from many sources into a consistent, historized store for reporting and analysis. Two lineages shaped it. Bill Inmon's approach, extended in the corporate information factory, builds a normalized warehouse that is subject-oriented, integrated, time-variant and non-volatile, and derives data marts from it. Ralph Kimball's bus architecture builds dimensional marts around business processes and integrates them through conformed dimensions. The warehouse's strengths are consistency, mature tooling and well-understood governance; its weaknesses are the lead time of modeling before use and a poor fit for semi-structured content. It fits regulated reporting and any domain where agreed definitions matter more than speed of onboarding.
The data lake stores data of any structure in raw form on low-cost storage and applies structure on read. The term is generally credited to James Dixon, then chief technology officer of Pentaho, who contrasted the lake's water in its natural state with the cleansed, packaged bottled water of a data mart. The lake's strength is the speed and cost at which new sources can be landed and explored; its weakness is governance, because without catalog, ownership and quality controls raw storage becomes a swamp in which nobody can find or trust anything. It fits exploration, feature engineering and the retention of raw history, provided it is operated with warehouse discipline.
The lakehouse adds warehouse capabilities to lake storage by placing an open table format, such as Apache Iceberg, Delta Lake or Apache Hudi, over files in object storage, which brings ACID transactions, schema enforcement and time travel. The term was popularized by Databricks, and the architecture is set out in a research paper by Michael Armbrust, Ali Ghodsi, Reynold Xin and Matei Zaharia. Several engines can read and write the same tables, which realizes the separation of storage and compute, and the medallion convention of bronze, silver and gold layers organizes progressive refinement. The lakehouse's strengths are openness and one copy serving engineering and business intelligence alike; its weaknesses are the operational maturity it demands and the temptation to treat the layer convention as a substitute for modeling. It fits organizations consolidating lake and warehouse estates.
Operational data store and master data hub
The operational data store (ODS) is an integrated, subject-oriented store of current or near-current data from several operational systems, used for operational reporting and for serving integrated views back to operations. Unlike the warehouse it is volatile and holds little history. Its strength is a consistent, current view across systems without burdening them; its weaknesses are the cost of another integrated store and the risk that it quietly becomes the warehouse or the integration hub. It fits customer service views and operational dashboards.
The master data hub manages golden records for core entities such as customer, product, supplier and location, matching and merging records from contributing systems and distributing the result. Its implementation styles, from a registry that only cross-references identifiers, through consolidation and coexistence, to a centralized hub in which master data is created, trade intrusiveness for control. Its strength is a governed, de-duplicated view of the entities every other pattern depends on; its weaknesses are organizational effort, matching complexity and synchronization latency. It fits wherever the same entity is created in several systems.
Streaming, virtualization and data services
The event streaming backbone moves data as a durable, ordered log of events to which producers publish and from which consumers subscribe; platforms such as Apache Kafka are examples, and change data capture (CDC) tools feed database changes into it. Its strengths are decoupling of producers from consumers, low latency, replayability and a single path of change that both planes can consume. Its weaknesses are the skills it requires, the difficulty of expressing complex joins and history as streams, and schema governance across many topics. It fits event-driven integration and real-time analytics.
Data virtualization presents a logical, queryable layer over distributed sources without moving the data, translating a federated query into source queries. Its strengths are speed of delivery, avoidance of copies and usefulness where data must not be moved for legal or contractual reasons. Its weaknesses are performance that depends on the weakest source, load pushed onto operational systems, limited history and difficult transformation. It fits cross-source access to moderate volumes and prototyping before physical integration.
The data services layer exposes data through managed application programming interfaces (APIs), typically behind a gateway that handles authentication, rate limiting and versioning. It is the operational counterpart of the analytical data product: a contract-bound, discoverable interface through which applications and partners obtain data without touching the source. Its strengths are decoupling, reuse and policy enforcement at one point; its weaknesses are unsuitability for bulk movement and the governance of hundreds of endpoints. It fits application integration, partner exchange and synchronous retrieval of master and reference data.
Data mesh and data fabric
Data mesh, introduced by Zhamak Dehghani, is an organizational and architectural approach rather than a platform. Its four principles are domain ownership of analytical data, data as a product, a self-serve data platform and federated computational governance. Its strength is scaling data delivery with the number of domains by removing the central team as a bottleneck. Its weaknesses are the organizational maturity it assumes, duplicated skills across domains and the difficulty of interoperability without strong federated standards. It fits large organizations with many autonomous, engineering-capable domains.
Data fabric, which Gartner frames as a design concept, uses active metadata, a knowledge graph of the data landscape and automation to integrate, govern and deliver data across heterogeneous platforms with less hand-built pipeline work. Where mesh distributes ownership, fabric centralizes intelligence about data while leaving the data where it is. Its strength is leverage on metadata the organization should be capturing anyway; its weaknesses are dependence on metadata quality and coverage, the immaturity of fully automated integration, and vendor definitions that vary widely. It fits heterogeneous estates with a strong catalog foundation. Mesh and fabric are complementary: a mesh needs a metadata backbone, and a fabric needs owners for the data it connects.
| Pattern | Primary purpose | Strengths | Weaknesses | Best fit |
|---|---|---|---|---|
| Enterprise data warehouse | Integrated, historized data for reporting | Consistency, mature tooling, governance | Lead time; weak with unstructured data | Regulated and financial reporting |
| Data lake | Raw data of any structure at low cost | Fast onboarding, flexibility, scale | Swamp risk; weak governance by default | Exploration, data science, raw history |
| Lakehouse | Warehouse capabilities on open lake storage | Openness; one copy for many engines | Operational maturity; evolving tooling | Consolidating lake and warehouse estates |
| Operational data store | Current integrated view for operations | Near-current data; relieves sources | Another store; scope creep | Service views, operational reporting |
| Master data hub | Golden records for core entities | Governed, de-duplicated entities | Organizational effort; synchronization latency | Entities created in several systems |
| Event streaming backbone | Durable, ordered movement of change events | Decoupling, low latency, replay | Skills, schema governance, complex joins | Event-driven integration, real time |
| Data virtualization | Logical access without moving data | Speed of delivery; no copies | Source load, performance, limited history | Cross-source access; movement constraints |
| Data mesh | Domain-owned analytical data products | Scales with domains; clear ownership | Assumes maturity; interoperability effort | Many autonomous, capable domains |
| Data fabric | Metadata-driven integration across platforms | Leverages metadata; less hand-built work | Depends on metadata quality; varied definitions | Heterogeneous estates with a strong catalog |
| Data services layer | Managed APIs for application access | Reuse, contracts, policy at one point | Unsuited to bulk; endpoint sprawl | Application and partner integration |
Consumption
- BI
- Applications
- Data science
- Partners
- AI
Serving and access
- Semantic layer
- APIs
- Data products
- Virtualization
Integrated storage
- Lakehouse or warehouse
- Operational data store
- Master data hub
Ingestion and movement
- Streaming backbone
- CDC
- Batch pipelines
Sources
- Transactional systems
- SaaS
- Devices
- External feeds
In the figure above the patterns are layers of one architecture rather than alternatives. Streaming and batch coexist in ingestion; integrated storage holds the lakehouse or warehouse, the operational data store and the master data hub; and a serving layer exposes data through a semantic layer, data products, APIs and virtualization. The catalog, security, governance and observability run across every layer. Mesh and fabric are not layers: mesh organizes ownership across the layers, and fabric automates the cross-cutting foundations.
Selecting and combining patterns
Pattern selection is a matter of fit rather than fashion. Five criteria dominate: the latency consumers need, the consistency they can tolerate, the governance obligations attached to the data, the total cost of the pattern over its life, and the skills the organization has or can acquire. The table below sets out the question to ask under each criterion and the direction in which the answer pushes the choice.
| Criterion | Question to ask | What the answer changes |
|---|---|---|
| Latency | How fresh must data be when consumed: days, hours or seconds? | Seconds push toward streaming, an ODS and APIs; days allow batch loading |
| Consistency | Must every consumer see the same value at the same moment? | Strict consistency points to the system of record or a transactional hub |
| Governance | How regulated, sensitive or definition-critical is the data? | Regulated data needs modeled, controlled stores and inherited policy |
| Cost | What are storage, compute, movement and staffing costs over several years? | Open formats and shared platforms lower lifetime cost; count movement and staffing |
| Skills | Can the organization build and run the pattern over time? | Mesh, streaming and polyglot estates demand skills; otherwise prefer fewer, managed patterns |
The matrix below crosses the first criterion, freshness, with a structural factor the table does not capture: how many autonomous domains produce the data. Together they indicate where an architecture's emphasis should lie.
Freshness and latency requirement
Central real-time platform
Streaming backbone feeding a shared lakehouse
Event-driven mesh
Domain data products published over a shared backbone
Central warehouse
Batch integration, conformed dimensions, one team
Federated platform
Domain-owned products on a shared batch platform
Autonomy and number of producing domains
Most organizations need more than one quadrant, because freshness needs and domain autonomy vary across subject areas, so the matrix guides emphasis rather than exclusive choice. Combining patterns is normal and should be explicit: a streaming backbone commonly feeds the lakehouse, the operational data store and the master data hub from one stream of change; a lakehouse is the storage beneath a mesh of domain-owned products; and a fabric's metadata layer spans warehouse, lake and operational systems alike. The combinations to avoid are those that duplicate a function, such as two integration hubs for the same data, or that hide a missing decision, such as virtualization used instead of designating a system of record.
Transition architectures and roadmaps
A target architecture that cannot be reached is a poster. TOGAF's later phases, Opportunities and Solutions (Phase E) and Migration Planning (Phase F), exist because the route matters as much as the destination, and their device is the transition architecture: a described, stable intermediate state that delivers value on its own and reduces risk for the next step. A data architecture roadmap should be expressed as two to four transition architectures, each naming the systems it changes, the data it migrates, the interfaces it retires and the capability it enables.
- Document the baseline honestly. Inventories, the system-of-record map as it is, data flows as they run, and known defects.
- Describe the target in the same artifacts. Use the same models and maps for the target, so that gaps are visible item by item.
- Group gaps into transition architectures. Each should leave the landscape consistent and deliver a measurable outcome.
- Sequence by dependency and value. Attach funding and owners, and record the rationale in an architecture decision record.
- Re-plan at each transition. The target will move, and the roadmap should move with it rather than be defended.
Managing technical debt
Technical debt in data landscapes takes recognizable forms: undocumented pipelines, duplicated copies of critical data, point-to-point interfaces that bypass the backbone, hand-maintained reporting stores and platform versions out of support. Managing it starts with a debt register that names each item, its owner, the interest it charges in effort and risk, and the transition architecture in which it will be repaid. Not all debt should be repaid, because some systems will be retired before their debt matters. The strangler fig pattern, named by Martin Fowler, in which a new component incrementally takes over an old one's responsibilities behind a stable interface, repays large items without a risky cutover.
Architecture governance
Architecture governance is the set of structures and processes through which an organization ensures that solutions conform to its architecture, that principles are applied consistently and that exceptions are deliberate. It should be as light as the organization's risk tolerance allows, because heavy governance is bypassed rather than obeyed. The elements below are common to most workable models.
Design authority
The accountable architecture leadership, typically the enterprise data architect with domain architects, which owns principles, reference architectures and standards and decides contested designs.
Architecture review board
A standing forum that reviews significant solutions against principles and standards at defined gates, records decisions and tracks exceptions.
Principles compliance
Assessment of a design against each principle with a stated finding: compliant, compliant with conditions, exception requested or non-compliant.
Exceptions
Time-limited waivers with an owner, a rationale, a compensating control and an expiry date, after which they are renewed, remediated or escalated.
Fitness functions
Automated checks, a concept from evolutionary architecture, that continuously test an architectural property, such as no direct cross-domain database access or every table registered in the catalog.
Review should be concentrated where decisions are expensive to reverse: a new system of record, a new integration pattern for a subject area, a new store type, an external data exchange, or any design that moves regulated data across a boundary. Routine solutions should pass through automated checks and self-assessment against a checklist, with the board sampling rather than inspecting everything. Reviews that arrive after a design is built are theater; the board's calendar should align with program gates so that it sees designs while options are still open.
Relationship with data governance
Architecture governance and data governance are distinct and interdependent. Data governance decides who owns data, what it means, which policies apply and which source is authoritative; architecture governance decides how structures, flows and platforms are built to honor those decisions. The system-of-record map is the clearest shared artifact: data governance designates authoritative sources, and architecture encodes and enforces the designation. Where the two are not connected, architecture designs channels for data nobody owns, and governance issues policies no platform can enforce.
Data governance decides whose data it is and what it means; data architecture decides how it is kept, moved and served. Neither works alone.
Cloud, security zones and environments
Cloud platforms have changed the economics of data architecture without changing its principles. Storage is cheap and elastic, compute is rented by the minute, managed services replace much platform engineering, and separation of storage and compute is the default. Cloud has also introduced constraints that architects must design around explicitly: data gravity, egress charges, residency obligations and the differences between providers.
Data gravity, egress and residency
Data gravity, a term coined by Dave McCrory, is the observation that large bodies of data attract the applications and services that use them, because moving data is slow and expensive while moving processing is not. The location of the primary integrated store therefore determines where most processing will end up, and that choice should be made deliberately rather than inherited from the first migration. Egress charges reinforce gravity: providers typically charge to move data out of a platform or region, so architectures that shuttle large volumes between clouds or back to on-premises systems incur recurring costs that should be modeled before the pattern is chosen.
Residency and sovereignty requirements determine where data may be stored and processed and under whose jurisdiction. The GDPR's rules on international transfers in Articles 44 to 49, sector regulation in financial services and health, and national data localization laws can each constrain region choice, replication and the location of support staff. The EU-US Data Privacy Framework provides a lawful basis for transfers to certified US organizations, but its two predecessors were invalidated by the Court of Justice of the European Union and the framework itself has been challenged in the EU courts. Residency should therefore be treated as a metadata attribute of each data set that drives placement and replication rules, so that a change in the legal position becomes a change of configuration rather than a redesign.
Many large organizations operate more than one cloud and retain on-premises systems through acquisition and history rather than design. The sensible posture is to accept a hybrid estate while refusing gratuitous duplication: choose a primary location for each subject area's integrated data, use open formats so that engines can change without moving data, place workloads with the data they consume, and reserve cross-cloud movement for services genuinely unavailable elsewhere. Portability is a property to be tested, for instance by proving that a critical data set can be read by a second engine.
Security zoning and environments
Security zoning applies the classification scheme to the architecture. A common arrangement separates a landing zone, where raw data arrives under restricted access; curated zones, where governed data is served under role- or attribute-based access; and restricted zones for data whose sensitivity requires tokenization, column-level encryption or approved-purpose access. Zones are implemented through network segmentation, identity and access management, key management and policy enforcement in the platform, and they should be defined once in the reference architecture so that every solution inherits them. ISO/IEC 27001 and the organization's control framework supply the controls, and GDPR Article 32 requires security appropriate to the risk for personal data; the architecture decides where each control is enforced.
Environments complete the picture. Development, test and production environments must be isolated at the identity and network level, and production data must not flow into non-production environments without masking, pseudonymization or substitution by synthetic data. Pipelines, schemas and policies should be promoted between environments as versioned code through the same deployment mechanisms as application software. An architecture in which production data is copied to developer workstations has a zoning design only on paper.
Metrics and pitfalls
Metrics
An architecture function that cannot show its effect will be judged by the volume of its documents. The metrics below are chosen because each reflects a principle and can be measured from artifacts the function should already maintain: the catalog, the system-of-record map, the standards catalog and the review records. Targets should be set per domain and trended over time rather than compared with external benchmarks, which rarely share definitions.
| Metric | How it is measured | What it indicates |
|---|---|---|
| Reuse | Consumers per governed data set, API or data product | Whether shared assets are shared in practice |
| Redundancy | Copies of each critical data element outside designated distribution points | Progress toward authoritative sourcing |
| Time to provision data | Elapsed time from approved request to usable access | Friction in the platform and governance path |
| Standards adherence | Share of reviewed solutions compliant without exception | Fitness of standards and quality of guidance |
| Exception age | Open exceptions by age and by expired status | Whether waivers are temporary or permanent |
| Interface conformance | Share of cross-boundary interfaces using sanctioned patterns | Decoupling versus point-to-point growth |
| Decision coverage | Significant changes with an architecture decision record | Whether the architecture is remembered |
Common pitfalls
The failures of architecture functions are as consistent as the symptoms of accumulated landscapes, and most have a structural remedy that this paper has already described. The table below pairs each pitfall with its cure.
| Pitfall | What happens | Remedy |
|---|---|---|
| Ivory-tower architecture | Blueprints are produced away from delivery teams and reviewed only at the end | Embedded domain architects, early clinics and decisions recorded in the open |
| Pattern-of-the-year adoption | Each new pattern is adopted as a replacement rather than a tool, so the estate gains a layer with every cycle | Explicit selection criteria and retirements that accompany every addition |
| No transition plan | A target state without transition architectures, funding and retirements leaves teams to improvise the route, usually by building a parallel landscape | Two to four funded transition architectures, each with named retirements |
| Single-copy fundamentalism | Pursuing one physical copy of enterprise data discredits the architecture when copies inevitably appear | Designated systems of record and distribution points, with every other copy derived and traceable |
| Platform before principles | A platform is selected and the architecture derived from its feature list, which makes the vendor's roadmap the enterprise's architecture | Principles and reference architectures agreed first; platforms chosen against them |
| Analytics-only scope | The analytical plane is architected carefully while operational systems exchange data through undocumented point-to-point interfaces, so every inconsistency is inherited downstream | Architecture that covers both planes, with interface standards for operational integration |
| Governance by inspection | A review board is relied on to catch everything, which produces queues and bypasses | Standards encoded in platform defaults and fitness functions, with review reserved for costly decisions |
A staged path forward
Architecture maturity is reached in stages, and attempting to skip stages produces documents without adoption. The path below is indicative; domains within one organization will sit at different stages, which is acceptable provided the enterprise-level artifacts exist and the direction is shared.
- Stage 1
Undocumented
Landscape known only to individuals; copies uncounted
- Stage 2
Documented
Inventories, system-of-record map, baseline data flows
- Stage 3
Standardized
Principles, reference architectures, standards, reviews
- Stage 4
Managed
Roadmap, debt register, metrics, exceptions with expiry
- Stage 5
Optimizing
Automated compliance, fitness functions, live roadmap
Each stage creates the conditions for the next. A roadmap, a debt register and metrics become meaningful only once principles and reference architectures exist, because until then they have nothing to measure against. The final stage, in which compliance is largely automated and the roadmap is revised continuously, is reached by organizations that treat the architecture as a product with users rather than as a standard to be enforced.
Where to start
- Document the baseline. Inventory stores and interfaces, record the system-of-record map as it actually is, and trace the flows that carry critical data.
- Agree principles and reference architectures. Adopt a short set of principles with rationale and implications, publish a few reference architectures, and start reviewing significant decisions against them.
- Build a credible roadmap. Express the target as two to four transition architectures, open a technical debt register, and record each significant choice in a decision record.
- Measure and adjust. Report reuse, redundancy, provisioning time and standards adherence by domain, and revise any standard that attracts repeated exceptions.
- Automate compliance. Encode the most important standards as fitness functions and platform defaults, so that review effort shifts to the decisions that are expensive to reverse.
Conclusion
Enterprise data architecture succeeds when it changes decisions. The artifacts, patterns and governance described here exist to make the right design the easy design: to let a solution team find the authoritative source for the data it needs, choose a sanctioned pattern that fits, inherit the controls that classification requires and record what it decided for those who follow.
The route is unglamorous: principles specific enough to exclude something, designated authoritative sources in place of the myth of a single copy, patterns chosen for fit rather than fashion, transition architectures that retire as much as they add, and governance concentrated where decisions are expensive to reverse. Organizations that reach that state spend less on rediscovering their own data and more on using it.
Glossary
- Architecture decision record (ADR)
- A short document recording the context, decision and consequences of a significant architecture choice, often with the options considered.
- Authoritative source
- A designated system or distribution point from which consumers are expected to obtain a kind of data.
- Data fabric
- A design concept that uses active metadata, knowledge graphs and automation to integrate, govern and deliver data across heterogeneous platforms.
- Data gravity
- The tendency of applications and services to locate near large bodies of data because moving data is costly and slow.
- Data lake
- A store that holds data of any structure in raw form on low-cost storage and applies structure when the data is read.
- Data mesh
- An organizational and architectural approach in which business domains own analytical data and publish it as products on a self-serve platform under federated governance.
- Enterprise conceptual data model
- A business-level model of the enterprise's core entities, relationships and subject areas, independent of systems.
- Fitness function
- An automated test that continuously checks an architectural property of a system or platform.
- Lakehouse
- An analytical architecture that adds warehouse capabilities, such as transactions and schema enforcement, to lake storage through an open table format.
- Master data hub
- A system that matches, merges and distributes golden records for core entities such as customer, product and supplier.
- Operational and analytical planes
- The two broad regions of a data landscape: systems that run the business and systems that analyze it.
- Operational data store (ODS)
- An integrated, subject-oriented store of current or near-current operational data, volatile and holding little history.
- Polyglot persistence
- The deliberate use of several storage technologies, each chosen for the workload it serves best.
- Reference architecture
- A sanctioned, documented arrangement of components, interfaces and controls for a recurring class of solution.
- Subject area
- A high-level grouping of related business entities, such as customer or product, used to organize models, ownership and maps.
- System of record
- The system in which a particular kind of data is created and corrected and which holds its authoritative version.
- Transition architecture
- A stable, described intermediate state between the baseline and the target architecture that delivers value on its own.
References and further reading
- DAMA International, DAMA-DMBOK: Data Management Body of Knowledge, second edition.
- The Open Group, The TOGAF Standard, 10th Edition.
- John A. Zachman, A Framework for Information Systems Architecture, IBM Systems Journal, vol. 26, no. 3.
- W. H. Inmon, Building the Data Warehouse, fourth edition, Wiley.
- Ralph Kimball and Margy Ross, The Data Warehouse Toolkit: The Definitive Guide to Dimensional Modeling, third edition, Wiley.
- Michael Armbrust, Ali Ghodsi, Reynold Xin and Matei Zaharia, Lakehouse: A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics, Conference on Innovative Data Systems Research (CIDR).
- Zhamak Dehghani, Data Mesh: Delivering Data-Driven Value at Scale, O'Reilly Media.
- Neal Ford, Rebecca Parsons, Patrick Kua and Pramod Sadalage, Building Evolutionary Architectures, second edition, O'Reilly Media.
- Martin Fowler, Strangler Fig Application, martinfowler.com.
- Michael Nygard, Documenting Architecture Decisions.
- EDM Association (formerly the EDM Council), Data Management Capability Assessment Model (DCAM).
- ISO/IEC 38505-1, Information technology: Governance of IT, Governance of data, Part 1: Application of ISO/IEC 38500 to the governance of data.
- ISO/IEC 27001, Information security management systems: Requirements.
- General Data Protection Regulation (GDPR), in particular Articles 25, 32 and 44 to 49.
This paper offers general guidance and is not legal, regulatory or professional advice. Summaries of laws, regulations and standards are for orientation only; confirm the obligations that apply to your organization with qualified advisors. Product and vendor names are used for illustration and imply no endorsement.
© ByteCI, LLC. All rights reserved.