Google Cloud unveils Governance Agent for Dataplex

Google Cloud has introduced a Governance Agent project for Dataplex and BigQuery to automate the propagation of data governance metadata through data pipelines.

The software uses column-level lineage to carry descriptions, glossary terms, policy tags, and trust indicators from upstream data assets to downstream tables and views, reducing the manual documentation burden on data teams.

The project addresses a common problem in large data estates: well-documented core datasets often lose context as they are filtered, joined, and reshaped into new views for different teams. As a result, metadata such as definitions, privacy labels, and quality indicators often fails to follow the data into later stages of use.

Instead of requiring staff to re-document each downstream table by hand, the Governance Agent traces where columns originated and recommends governance metadata based on what is already known upstream. If a field has been transformed rather than simply passed through, the system reads the SQL used to create it and generates a description that reflects the change.

How it works

The project also uses semantic matching to connect technical column names with business glossary terms. Where lineage is incomplete or missing, it can draw on internal documents such as policy papers, product specifications, design documents, and spreadsheets to support its recommendations.

The software is intentionally conservative when handling privacy-sensitive labels. A column is marked as personally identifiable information only when there is clear evidence from upstream lineage or explicit language in supplied documentation. If the evidence is weak, the metadata is not applied automatically.

The same approach extends to access and masking controls. When sensitive data appears downstream, the agent recommends the relevant policy tag and provides a summary of current read access and masking rules. It also tries to distinguish between direct use of sensitive data and transformations that may have aggregated or anonymised it.

Another element of the project is a trust score derived from upstream data quality and profiling results. Google Cloud said this allows downstream tables to inherit a measure of reliability rather than being treated as unknown assets, while also recognising transformations that improve data quality through steps such as deduplication or null handling.

Second signal

The Governance Agent does not rely solely on recorded lineage. When tracked job history is missing, it can invoke Knowledge Catalog Data Documentation scans to infer relationships and column meaning, then feed those findings into the same traversal process used for lineage-based propagation.

Those inferred relationships are kept distinct from recorded lineage so stewards can see whether a recommendation came from a logged transformation or an inferred link. The order is deliberate: standard lineage is checked first, with inferred documentation used only to fill gaps.

That structure reflects the limits of automation in data governance. The system is designed to propose and preview changes before they are applied, rather than silently updating tags and descriptions across a data estate without review.

Operational use

The project includes both a dashboard built with Gradio and a command-line interface. This gives data stewards a visual review process for a small number of assets, while allowing platform teams to integrate scans and metadata propagation into scheduled jobs or continuous deployment pipelines.

This combination is intended to make governance work easier to automate without excluding the staff still responsible for judgement calls. Google Cloud said those calls are especially important in areas such as ambiguous mappings, unusual joins, and edge cases that do not fit standard definitions.

The broader premise behind the project is a shift from governance as a periodic audit task to governance that updates as data moves through a system. In practice, that means new views can inherit context at creation, sensitive data can be identified earlier, and stewards can spend less time adding repetitive field descriptions.

Even so, Google Cloud acknowledged the limits of the approach. Lineage confidence scoring can produce errors, especially where columns have been renamed or linked through more complex joins, and semantic checks remain heuristic rather than definitive.

A user of the system said the model could materially reduce manual cataloguing work in a large environment.

"As custodians of the VodafoneThree UK Datahub, one of our biggest challenges is that a significant proportion of our data estate remains undocumented or inconsistently labelled. This creates friction for data discovery, slows down delivery teams, and limits the value we can unlock from AI solutions built on top of our data. The Data Steward Agent changes that. By combining cataloguing, lineage, and automated metadata propagation, it enables us to focus governance effort where it adds the most value while automatically carrying trusted context downstream. Rather than manually reviewing thousands of tables, we can concentrate on governing source datasets and allow lineage to scale that knowledge across the platform. We estimate this approach can reduce cataloguing effort by up to 75%, while significantly improving data discoverability, trust, and AI readiness across the UK Datahub," said Radina-Paola Ivanova, GenAI engineer, VodafoneThree UK Datahub.