Data Mesh – Starting Point

Off late, I got opportunities to interact with multiple data experts and analytics leaders from different industries. Data Mesh, featured in all these discussions. What I learned from these conversations, while almost all of them understand the concepts of Data Mesh very well at a strategic level, when it comes to the point of implementation, it is pretty much the old school. Example, many of them think, they have a Data Mesh in place by just having a Data Catalog, that facilitates “Discoverability”, “Understandability” and “Security” to a certain extent. This could be a good starting point but not everything about Data Mesh.

As an advocate and influencer of implementing Data Mesh in my current organization, I think it make sense to pen down my thoughts around Data Mesh and get some feedback from the wider data experts here.

Me and my team after having several workshops, both internally & externally, especially with Microsoft who already helped and currently helping many organizations implementing Data Mesh, concluded to strategize the Starting Point of our Data Mesh journey first.

As Data Mesh is a sociotechnical approach, it makes sense to plan the Starting Point around the Social aspect of Data Mesh. We started with ODS (Organizational Structure & Design) of Data Mesh. Domain ownership is the first-class citizen in Data Mesh. Domain-Driven-Design is complex and it’s an art rather than science. While identifying and designing a domain model that perfectly represents the business process, is a tedious task, you always can start from a base line and keep on improving it gradually (with newly identified domain, sub-domains) as the Data Mesh matures.

Post identifying the domains (crude design though), the next big challenge was, continuous maintaining the harmony of Data Mesh. To tackle this, we came up with an idea of having a centralized Data Mesh Council function focusing on the centralized concerns of data governance, data product integration, self-serve data platform. The council is headed by a chairperson, typically a person in position of power, who can nullify any divergence from Data Mesh principles with influence. The other key personnel are data domain representatives, self-serve data platform architect, legal & compliance expert (e.g., GDPR SME), Data Integrator SME, who enables interoperability across data domains say between CRM and Supply Chain in a manufacturing entity, typically a data architect. The Data Mesh council further supplemented by a Data Mesh Architect & Data Mesh Consultants.

The Data Mesh Architect & Data Mesh Consultants are driven by the vision of Data Mesh council with a goal of enabling the individual business domain to own and develop their own data products, with a consulting approach. Remember the Data Mesh Consultants owns nothing. It is the individual Domain’s Data Product Developer & Data Product Owner responsible for developing, maintaining, and owning the Data Product.

While this is a good starting point, there are many miles to go, to have a full-fledged Data Mesh solution, that scales with the growing organizational complexity. In my subsequent posts I will keep on discussing how we are implementing different principles of the Data Mesh.

Thanks for reading & your feedback.

Ashok K Sahoo

Strategist Analytics Data & Emerging Technologies

Note:- The principles and concepts discussed are influenced by "Zhamak Dehghani", the ideator of Data Mesh.