Data Mesh vs. Data Fabric: How Are They Different and Which Should You Use?

In a recent article, I provided an introduction to the concept of data mesh, a relatively new, decentralized approach to data architecture, first identified by Zhamak Dehghani in 2019. Another recent approach to data architecture is data fabric, which, despite its similar name, is quite different. Read on to learn more about how the two approaches differ and whether they would be helpful to your organization.

What is Data Fabric?

Believed to have been first described in a 2016 white paper, data fabric is an approach that focuses on avoiding discrete data platforms in different parts of the organization, instead “weaving” them together based on logical relationships. The exact data fabric in a particular organization varies, but these six layers tend to be present in a data fabric: 

1. Data Management is a security-focused layer.
2. Data Ingestion is a layer that looks for connections between structured and unstructured data from the cloud, in order to “stitch” the data together based on these connections.
3. Data Processing filters out irrelevant data.
4. Data Orchestration is perhaps the most critical layer, because it is responsible for shaping the data into a clean and usable form for stakeholders in different parts of the organization.
5. Data Discovery finds connections between data sources whose connections were not initially found.
6. Data Access controls permissions and compliance, as well as providing dashboards and visualizations to make it easier for users to find the data they need. 

This data fabric with its many layers is a centralized approach to data architecture that ensures that separate data sources can all be accessed in one place, in an easily usable form, and organized in a logical way.

What is Data Mesh?

As noted above, data mesh is an approach to data management that has only been a part of industry vocabulary since 2019, but it has already had a big impact. It emphasizes decentralization and self-service. Key to data mesh architecture is the idea that data is owned by the domain that collects it and is responsible for its quality and accessibility. Each domain’s data is considered its data product. This approach aims to empower teams to own and govern their own data, rather than relying on a central storage location and one team of data professionals to provide access to all data to everyone else in the organization. 

Which Approach Should Your Organization Take?

Data mesh and data fabric both have useful applications depending on an organization’s needs, and both are fairly new approaches to data architecture. The most salient difference is that data mesh emphasizes decentralization and self-service, while data fabric is more centralized and focused on connecting and stitching data sources together. When it comes to what to use in your organization, it's also important to be aware that these approaches are not mutually exclusive and can be used in combination to achieve different goals.

What To Think About When Choosing Between Data Mesh and Data Fabric

As you examine the merits of data fabric and data mesh in relation to your organization’s architecture needs, here are some factors you should bear in mind. 

Consider the size of your organization and, in particular, who collects, uses, and maintains the quality of your data. For example, if you have a lot of separate teams collecting data and teams across the organization needing to access it, a data mesh approach might make more sense than data fabric for you. On the flip side, if your organization is small and doesn’t have a lot of separate teams collecting their own data, the more centralized data fabric architecture might be a better fit.

When it comes to data governance, keep in mind that it may be easier to enforce data policies and standards in a centralized data fabric architecture. On the other hand, within a data mesh, each domain owner will have to comply with data governance requirements with its own data product. Similarly, think about data privacy and security concerns. Data fabric can provide centralized security measures, while data mesh may require domain-level security controls.

Determine how much importance you will place on consistent data quality. The centralization of data fabric means that data quality can all be controlled at once. On the other hand, with decentralized data ownership distributed to domains, a data mesh architecture relies to a large extent on domain owners to maintain the quality of their own data. This leaves the rest of the organization relying on those domain owners to comply with quality standards. Understandably, there may be inconsistencies in quality of the data depending on how well a given domain owner maintains the quality.

You should also think about your employees’ capacity to manage their own domain’s data. If you use the data mesh approach, each domain of your organization will need at least one employee who understands how to maintain a high-quality data product that is accessible to the rest of the organization. This means they need to be knowledgeable both about their own domain and about policies and best practices for controlling a data product. If you do not currently have such a setup, you should consider the investment in training or hiring that will be required for data mesh, and decide whether this is a reasonable investment at this time.

This ties in well to another consideration, business objectives. Your organization’s goals should always remain at the center of any major initiative, particularly when it comes to something as essential as how you store and access data. In addition to ROI, think about whether agility, speed, or centralization of data management is more critical for achieving your business goals.

How to Combine Data Mesh and Data Fabric

Combining data mesh and data fabric within an organization can provide a powerful approach to managing and utilizing data effectively. If you decide this combined approach is best, there are ways your organization can use both concepts together.

Here is one example of how you could combine data fabric and data mesh in your organization. You could use data fabric's centralized governance and data quality capabilities to keep your data consistent and reliable all throughout your organization. To accomplish this, set up your data governance policies and data quality control within the data fabric layer. You can bring in data mesh to decentralize data product discovery and ownership. Every domain team will be required to ensure that its data products comply with the centralized data governance and quality rules. This will give you the best of both worlds, with the centralized policies and quality control of data fabric and the decentralized data ownership of data mesh. This is just one example; many other combinations of data fabric and data mesh are possible, and a data architecture consultant can help you to create the best configuration for your organization.

How To Get Help Determining The Best Setup For Your Data

After learning about data fabric, data mesh, and other approaches to your organization’s data, you may feel overwhelmed with choice. Hiring a technology consulting firm can help you to create the ideal data management strategy specific to your organization's unique needs. With deep industry knowledge and experience, consultants can perform a comprehensive assessment of your current data architecture, your business objectives, and your data-related challenges comprehensively. They can determine if data mesh, data fabric, or an alternative strategy suits your organization best, via techniques such as thorough evaluations and feasibility studies. This guidance can help ensure that you choose the most suitable data management approach for maximizing the value of your data assets.

Contact Square Peg Technologies to get assistance with your data architecture needs. Our data expertise coupled with our ability to understand and tailor solutions to your needs will help you to store, analyze, and use data as effectively as possible. Reach out to us to get started.