Vector Search and RAG Chatbots with Databricks at Microsoft Ignite

0:00and one of our most strategic Partnerships is with data breaks we are proud of the co-innovation that we've done with data breaks Azure data breaks and Microsoft fabric are highly interoperable as both products are based on a common open Lakehouse architecture so now I'd like to welcome Scott back on stage and with him Ali godsy co-founder and CEO of datab breaks to share more about the incredible work Microsoft and

0:22datab breaks are doing [Music] together welcome thank youon thank you Ali for being here we're very excited about our partnership and all the work we're doing together to help our customers accelerate their data and AI initiatives thank you so much arone I'm super excited to be here uh it's so awesome because now we have the same

0:44foundation for fabric and Azure datab brick so it's all built on this open source project called Delta lake so it'll be really seamless to go between these two projects awesome and I think you have some exciting stuff to show us yeah yeah super excited let's take a look all right okay so basically datab bricks is a unified platform where you have one copy of the data in Delta which is now shared and one place to do governance in

1:08unity catalog so let's start here all right so um we can see here there's a SQL data warehouse here um there is data engineering ETL for Delta and machine learning but let me go over to the catalog here and this is Unity catalog and inide unity catalog you have all your tables you can actually even Federate into other systems so we have

1:30table that we Federated from Snowflake here you can Federate bigquery or other systems um but what I want to really do is go look at this one table that we have in the system uh which is a churn table okay so churn features you can see here if we click on it that we have different columns in this table we have some sample data that's sitting in here um there's permissions so you can set

1:52out attribute based Access Control you can look at the lineage so you can see what are the Upstream Downstream uh table you can even see the whole lineage graph here and what we can do now is using all the semantic information about lineage metadata the AI has data intelligence to actually generate the text for what this table does we can actually click here and for each of the

2:14columns we can see that uh it's describing in English what they each do so we have an English description this now enables us to do semantic search so we can go to search and we can type in English find information about platform churn and it now will actually leverage that English generated data intelligence and give us the table that's important for this so this is super critical

2:37because it's really really hard in organizations to um find the right data okay so let's look at here in SQL uh I'm going to actually do some queries here show me the number of turned users uh for premium platform uh in the past fiscal year okay so it's going to generate SQL for us here but now it actually understood the data

3:00intelligence for these words premium platform um or fiscal year so let's run this query right we got the results uh so actually it generated a pretty good SQL for us um so um now what we're going to do um is use this uh to do some more uh intelligence with this so super exciting

3:22it's awesome to see how you're using uh you know Ai and specifically the Azure open AI service directly in the experience that's awesome yeah so let me show you what we can do actually with azzure open Ai and do some build some rag chat bot together so let's start here okay so um what I'm going to do is I'm going to go here and I'm going to upload a bunch of PDFs on documentation

3:44in datab bricks here uh to Unity catalog volume it now takes each of these and it actually creates Delta tables from these automatically so if we go click here um on databas documentation we can see that now it has the sample data uploaded it's actually chunked it it understands the contents uh and we can actually now say let's create a vector search database

4:06with this we give it a name and we have to then pick a primary key so that the vector search can use and we use an endpoint and then we use an embedding model so I'm actually just going to use Azure open AI adaa model here and I'm going to say that continuously update this uh Vector database okay so we can click here we can see that now the vector database is up and running um so

4:30what we're going to do next is we're going to create uh an endpoint so we're going to serve this model let's create an llm using Azure open AI chatbot um and create the serving endpoint okay so it's off now and it's creating an nlm endpoint that we can call into and I prepared a notebook here that we can see this notebook has a chat bot in it and we can actually ask a

4:52question that calls that endpoint now so how can I publish my Unity catalog schema to powerbi you can see this hits the endpoint that goes to the vector search and it answers it uh so this is awesome this now also gets automatically stored in a Delta table so we can see here all the requests all the monitoring and we can easily just push this now to

5:14powerbi so it pushes the data set into powerbi and we can go here and say automatically create a report uh based on the Delta table that we have and we can see here that we can see the toxicity or all the responses that got recorded so it's really really tightly integrated now into powerbi so you can see here with a few clicks we were able to build our own chatbot leverage Azure

5:36open AI uh and get the insights published into powerbi that's fantastic I mean it's it's awesome to see how how easy it is to create use Azure Databricks uh to create a modern geni application on Azure yeah we're super super excited we're super excited about doubling down on the partnership uh and see what we can do together to integrate this even more and more with fabric now

5:58thank you Al thank you so much Al thank [Applause] you