> ## Content Index > Fetch the complete content index at: https://www.ratneshmishra.com/llms.txt > Use this file to discover other available public pages before exploring further. # Part 1 - THE ENTERPRISE METADATA STRATEGY - Don’t Centralize the Catalog. Centralize the Control Plane. - URL: https://www.ratneshmishra.com/part-1-the-enterprise-metadata-strategy-dont-centralize-the-catalog-centralize-the-control-plane/ - Published: 2026-09-06T20:01:24.000Z - Updated: 2026-09-07T02:37:08.000Z - Author: Ratnesh Kumar Mishra ### Metadata stopped being documentation Most metadata programs do not fail because the team lacked discipline, or because the vendor underdelivered. They fail because of a decision made in month two, by people acting on a reasonable instinct, that quietly guaranteed the outcome. Here is that decision, and what to do instead. ### Three forces ended the documentation era For twenty years, metadata was a compliance chore. It lived in wikis nobody updated and data dictionaries that were accurate for about a week after the consultants left. Success was measured by how many rows had a description filled in. That era is over, and three forces ended it. **Regulation got specific.** GDPR, CPRA, HIPAA, DORA and the EU AI Act do not ask whether a policy exists. They ask whether you can prove, for a named data element, where it came from, what is permitted, who is accountable, and when it must be destroyed. That is a metadata question, not a legal-drafting question. **Architecture fragmented.** Data no longer lives in three warehouses. It lives across object stores, lakehouses, operational databases, SaaS applications, streaming topics, feature stores and vector indexes. No single platform owns it. No single platform's built-in catalog can govern it. **AI arrived without institutional knowledge.** A large language model does not know that `cust_rev_adj_final_v3` is the only revenue table finance trusts, that `region` means sales territory and not legal entity, or that the marketing database contains third-party data that may not be used for training. Every one of those facts is metadata. > **An AI system without a metadata layer is a confident stranger.** So here is the reframe that should anchor everything that follows: **Metadata is no longer documentation about the data. It is the control plane for governance and the context layer for intelligence.** Both have to be true at once, because the same classifications that block a policy violation are the ones that tell an agent what it may see. Which brings us to the decision. ## The instinct that destroys programs When metadata is scattered across a dozen systems, the instinct is to pull it into one catalog. Buy the platform. Run the crawlers. Populate the inventory. Declare a single source of truth. That instinct is half right. The half that is wrong is fatal. Physically centralizing metadata fails for exactly the reason physically centralizing data failed. Metadata is generated continuously by systems that will not surrender local authority. Snowflake, Databricks, BigQuery, Kafka, dbt, Airflow, ServiceNow, Salesforce and the identity provider each hold metadata they consider theirs — and each is right. A program that tries to become the sole system of record for all of it becomes a copy-and-reconcile machine, permanently one crawl behind reality. The workable pattern is **logical centralization with federated authorship**. One graph. One query surface. One identity scheme. Many authors, each authoritative for what they actually know. ![](https://storage.ghost.io/c/bf/09/bf092b96-7c12-4ea4-b204-9fb1e04f4342/content/images/2026/09/urrJg.jpg) ## What the control plane must actually provide **1\. One resolvable identity per asset.** Every table, topic, column, model, dashboard, feature, prompt and API contract gets a durable global identifier that survives renames and platform migrations. This sounds like plumbing. It is the highest-leverage decision in the entire architecture. Lineage, classification propagation, policy evaluation and access control all key off it. Without stable identity, everything downstream degrades into best-effort string matching — and you discover this eighteen months in, when a table rename silently breaks your masking policy. **2\. One graph, many sources.** The plane ingests from source systems without pretending to be their master. Every fact carries provenance: which system asserted it, when, and at what confidence. When two sources conflict, precedence rules resolve it deterministically rather than by whichever crawler ran last. **3\. Event-driven ingestion, not nightly crawls.** Batch harvesting produces a catalog that is always a day behind — which is exactly enough to be untrustworthy. Prefer change-data-capture on system catalogs, hooks in CI/CD, and events emitted at deploy and run time. Where a system only supports polling, poll frequently and publish the freshness lag so users can calibrate their trust. **4\. Open interfaces over proprietary lock-in.** The metadata plane must outlive any individual platform choice, and platform choices change every four years. OpenLineage for lineage events, Iceberg REST catalog semantics for table metadata, and open catalog implementations give you portability. Treat the vendor catalog UI as replaceable. Treat the graph, the identifiers and the event format as permanent. **5\. A single query surface.** One API over the whole graph. If a compliance analyst, a platform engineer and an AI agent each need a different interface to answer the same lineage question, nothing has been centralized. ![](https://storage.ghost.io/c/bf/09/bf092b96-7c12-4ea4-b204-9fb1e04f4342/content/images/2026/09/OUFoB.jpg) ## Centralize the schema of governance. Federate the content of it. Federation is not an excuse for anarchy. A small set of artifacts must be authored centrally and enforced globally: - The **classification taxonomy** and tag vocabulary - The **policy definitions** those tags bind to - The **ownership model** and its resolution rules - The **business glossary** and canonical metric definitions - The **identifier scheme** and lineage event format Everything else — descriptions, domain terminology, local quality rules, operational annotations — belongs to the domain that owns the data. Central teams that try to author domain meaning become a bottleneck first and a scapegoat second. ![](https://storage.ghost.io/c/bf/09/bf092b96-7c12-4ea4-b204-9fb1e04f4342/content/images/2026/09/DfQKu.jpg) ## The vanity catalog, and how to avoid building one The classic failure is a beautifully populated inventory that no operational system depends on. Nothing breaks when it is wrong, so it drifts. Once it has drifted, people stop consulting it. Once they stop consulting it, it dies quietly at renewal time — and the post-mortem blames adoption. Adoption was never the problem. **Dependency was.** The countermeasure is architectural, not cultural: > **Make something important depend on the metadata.** If the retention job reads its schedule from the catalog, the catalog stays current. If access is denied when classification is missing, classification gets filled in. If a deployment fails when a data contract has no owner, owners get assigned within the day. Metadata on the critical path stays true. Metadata that is merely descriptive rots. This principle recurs in every part of this series, and it is the single best predictor of whether a metadata program is still alive in year three. ## Three questions to ask your own architecture this week - **Can you name the durable identifier for a given column — and does it survive a table rename?** If the answer involves a fully-qualified name string, you are carrying a fragility you have not priced. - **What is the p95 lag between a schema change in production and its appearance in the catalog?** Hours means you are in good shape. Days means your users have already learned not to trust it. - **Name one production system that would break if the catalog were wrong.** If you cannot, you have an inventory. Not a control plane.