Amperity

Chuck Data from Amperity

0:04I have the pleasure to have kebab, chief hacking officer at umperity. I think you have a surprise for us.

0:12>> Yes, today we are launching Chuck data.

0:14Uh Chuck data is a agent for doing data engineering work in data bricks built natively on data bricks. So chuck runs uh as a python command line tool. You can see uh you know here it it literally

0:26directly interacts with your data bricks tool toolkit. So it Chuck has a library of tools that it can use. You can see that there are catalog tools here um SQL warehouse tools to run SQL modeling. You

0:39can see that it has the ability to profile and tag PII data as well as run imper machine learning stitch algorithm.

0:46And then you can do all this by natural language. So the idea here is to instead of writing ETLs, you can interact with a a a natural language with an agent and have it configure and implement things

0:57in your data bricks account. So in this example, you can see I just asked it at the top, what customer data do I have?

1:03And it looks at my Unity catalog and says, here's what tables are available uh uh you know, and it looks at each of them and profiles them and says, oh well, these are the different types of

1:12PII in there. And you apply those tags to Unity catalog to power governance workflows, that kind of thing. Um and then if you want you can even say hey I want to build an identity graph and so

1:22you can see here I said hey run stitch on this data and it came through and constructs a job for me. It gives me an opportunity to give feedback and adjust it and then it actually will launch that

1:33job. It'll say hey I'm going to create this notebook to show you the results.

1:36I'm going to create these tables with your links and your your nodes and I'm going to run it with this job and you can open that directly in data bricks.

1:43this whole this whole idea is that instead of writing ETLs, you can just use this tool um and and you know it'll operate on your infrastructure in your security bubble. And so you can see if I

1:52go to data bricks, it'll actually run that job. You know, you don't have to understand this this gibberish right now, but um but once that's done, it it has a stitch report that will let you

2:02evaluate the resulting data. You can see how those data sets overlap. You can see, you know, duplication rates by different data sets. you can see uh you know how many people went from anonymous

2:12to known. So like in this example you can see these were partially known and after the the new clusters have more linked data so there's more people that you can actually target and and build

2:22analytics on. Our whole idea here is to flip the concept of ETL on its head and let people interact with a uh an agent that will implement data engineering use cases using data bricks. So um we it's

2:34free to use. It's a Python uh uh library that we are, you know, excited to get as much user feedback on as we can. Uh you know, contributors welcome and and we we just want to see how people uh respond

2:46to this and as a new way to do data engineering. So, I'm excited to see what people think.

2:52>> Thank you so much, Caleb.

I have the pleasure to have kebab, chief hacking officer at umperity. I think you have a surprise for us. >> Yes, today we are launching Chuck data. Uh Chuck data is a agent for doing data engineering work in data bricks built natively on data bricks. So chuck runs uh as a python command line tool. You can see uh you know here it it literally directly interacts with your data bricks tool toolkit. So it Chuck has a library of tools that it can use. You can see that there are catalog tools here um SQL warehouse tools to run SQL modeling. You can see that it has the ability to profile and tag PII data as well as run imper machine learning stitch algorithm. And then you can do all this by natural language. So the idea here is to instead of writing ETLs, you can interact with a a a natural language with an agent and have it configure and implement things in your data bricks account. So in this example, you can see I just asked it at the top, what customer data do I have? And it looks at my Unity catalog and says, here's what tables are available uh uh you know, and it looks at each of them and profiles them and says, oh well, these are the different types of PII in there. And you apply those tags to Unity catalog to power governance workflows, that kind of thing. Um and then if you want you can even say hey I want to build an identity graph and so you can see here I said hey run stitch on this data and it came through and constructs a job for me. It gives me an opportunity to give feedback and adjust it and then it actually will launch that job. It'll say hey I'm going to create this notebook to show you the results. I'm going to create these tables with your links and your your nodes and I'm going to run it with this job and you can open that directly in data bricks. this whole this whole idea is that instead of writing ETLs, you can just use this tool um and and you know it'll operate on your infrastructure in your security bubble. And so you can see if I go to data bricks, it'll actually run that job. You know, you don't have to understand this this gibberish right now, but um but once that's done, it it has a stitch report that will let you evaluate the resulting data. You can see how those data sets overlap. You can see, you know, duplication rates by different data sets. you can see uh you know how many people went from anonymous to known. So like in this example you can see these were partially known and after the the new clusters have more linked data so there's more people that you can actually target and and build analytics on. Our whole idea here is to flip the concept of ETL on its head and let people interact with a uh an agent that will implement data engineering use cases using data bricks. So um we it's free to use. It's a Python uh uh library that we are, you know, excited to get as much user feedback on as we can. Uh you know, contributors welcome and and we we just want to see how people uh respond to this and as a new way to do data engineering. So, I'm excited to see what people think. >> Thank you so much, Caleb.

Open on YouTube

Transcript