Apache Iceberg with Unity Catalog: Multi-Engine Governance

0:03All right, thanks Mai. In this demo, I'm going to show just how easy it is to access Unity Catalog managed tables from a wide range of tools. I'll demonstrate how attribute-based access control allows you to easily mask sensitive data such as PII. And finally, I'll show how governance also extends all across the ecosystem. Let's hop into the demo.

0:27We'll start here in the catalog explorer. This is where I can view all of the data and AI assets in my ecosystem. Here I already have an iceberg manage table set up which contains data from a new marketing campaign that my team is working on. For all managed tables, whether iceberg or delta, unity catalog automatically handles performance tuning, data layout,

0:49and predictive optimizations. All managed tables are also accessible from external tools. I'll show just how easy this is using some of the tools that mate just mentioned and DuckDB I already have it configured with catalog's iceberg rest catalog APIs

1:12catalog has a full implementation of the spec meaning that any iceberg engine can read and write to it let's see this in action first I'll query that same table that we were just looking in catalog. So, let's go ahead and change that. And now it

1:38just inserted 10,000 more rows from EMR to the same. Let's head back into Databricks and I'll give this page a quick refresh. And here you can see that the changes are automatically reflected in Unity catalog. Again, this table is governed and tracked just like any other managed table.

2:03It doesn't stop there. I can also access tables from tools like Snowflake and DB. In the interest of time, I'm going to go through these pretty quickly here. I already have Snowflake with Unity Catalog's iceberg rest catalog API. Again, let's query that same table. I also have DuckDB running locally. So again, let's query that same table.

2:28In just under two minutes, I've now accessed and DuckDB. And that barely scratches the surface of supported tools in the ecosystem.

3:02those aren't locking complex rules. Behind the scenes, this tag automatically applies our mask email function. So, if I give this a quick refresh,

3:24you'll see that the data is now masked out. I can no longer see those email addresses. What makes this seamless is attribute-based access control. Admins can define ABAC policies at scale and data producers can apply protections themselves to stay aligned with governance standards.

3:48Now you might be thinking with so many engines and tools in my organization how is my poor data admin supposed to keep up? Unity catalog is the only catalog that centralizes governance across engines. Let's see this in action. And to start, we'll head back into DuckDB.

4:10I'll query that same table that we were just looking at. And if everything is working as expected, I should expect to see that those email addresses are now masked out. Great. I've now verified that customer data is protected no matter what tool I'm using, even if I'm using something

4:32like DuckDB. To conclude, Unity catalog makes it easy to manage data access, security, and governance across a wide range of tools. No matter what engine or format you are using, it all stays governed, searchable, and secure in one place. Thank you so much.