Databricks + Salesforce: Unified CDP with ML Churn Prediction

0:00visualizing data from Databricks into Salesforce you have your data in your lake house uh where you as a data engineer can collect everything and centralize the contents how do we expose data from Salesforce into Data breaks to either retrain those models or to conduct any other analysis of Downstream in data hey folks uh so today I'm with youf

0:24he youf and we will be talking about uh Salesforce integration with the bricks with the Canadian dream team so with us we have Vincent today and Jag hey guys hey hello thanks for taking the time um so maybe we could do like a quick introduction Vincent do you want to start and you know kind of explain us what you're doing at the bricks and and

0:46with Salesforce sure uh my name is Vincent I'm in Montreal Canada uh I've been with Databricks for five years now and I'm a manager of field engineering so my team covers all the public sector accounts in Canada and uh my name is Ja Jani I'm a senior solution engineer at Salesforce I cover our American accounts Vincent and I met about a year ago and we started working on you know what's

1:09the possibilities between Databricks and Salesforce and uh and here we are we're excited to show the four pillars of what that looks like that's great so we see a lot of you know sales first Plus Databricks and I know we had tons of updates uh to make sure the product work better together and customers can can get the best out of them uh I Know Jack do you want to get us you know started and and just give us an overview

1:33of what's available with d bricks and sales yeah sure um and I'm going to share my screen but before I do that really quickly the four pillars of this integration this partnership between Databricks and Salesforce are one uh the longstanding pillar between Tableau and Databricks as you know right Tableau and Databricks have been working together much longer actually than Salesforce and Databricks um and there are some updates there from the

1:55Delta sharing connector but on the other three pillars they're more mostly FOC on Salesforce which is uh bring your own Lake meaning bringing data and visualizing data not bringing it visualizing data from Databricks into Salesforce for a sales rep marketing teams service teams to activate on that data then you have bring your own model so you know data breaks is traditionally

2:18a data science platform so a lot of data Sciences have built some really good models there how do we infer those models and make use of those models in the flow of work and that's bring your own models so be able to call those models via API into Salesforce and then be able to act on that and then the third pillar which is something they just released at the summit which is what you call lak house Federation for yall right which is how do we expose

2:40data from Salesforce into Data breaks to either retrain those models or to conduct any other analysis um Downstream in Databricks so those are the four pillars primarily focused on two pillars today and then the other two pillars we'll showus at a later time but we're going to show bring your own model and bring your own Lake today so for those that haven't worked um with Salesforce before Salesforce is a u is is

3:04considered you know one of the best crms out there and it's essentially a way for you to be able to manage your customer interactions and your relationship with your customers through various touch points that's everything from sales to service to marketing to analytics and that and commerce and that keeps growing every day right just because customers expectations have changed um and as a result you know we understand at

3:26Salesforce that a single source of Truth has many definitions a single source of Truth in Salesforce may be something that's uh within the scope of those Realms but a single source of Truth for the company might be something more and therefore we introduce data cloud data cloud is the ability is our own Lakehouse that helps you um bring data together from various sources and then activate on that data for a sales

3:49service marketing Commerce use case so this is data Cloud here um how you access data cloud is just by clicking on the waffle icon you'll see all the different applic we have so if you're a sales user you typically use something like a sales app which is something like this like a homepage give your contacts accounts opportunities Etc and data

4:11cloud is that layer beneath that that basically brings all that data together and it's actually just accessed right from the same place as you would for all the other apps and here's when you start configuring your different data streams so you know datab brakes is very familiar with Medallion architecture right it's actually quite similar in Salesforce as well what you'll see at

4:33the very top here are the bronze layer the silver layer and the gold layer right so data streams it's just raw data coming in from various sources so I have all my data streams coming in here and this means data also from Salesforce so this doesn't discriminate between Salesforce data versus data brakes data data is data and data cloud and then you activate that back in Salesforce so starting things off you

4:57would start to configure your data streams in here from various sources and you'll see here that I have some data coming from Databricks as well as some data coming from Salesforce we have a plethora of other connections as well via moft um as well as some of our own pre-built connectors but you'll notice a few things here so um you have your inje

5:19connectors and then you have your direct access connectors direct access connectors def are defined by connectors that are just pushing down queries to The Source in this case data so you're just virtualizing data you're not actually porting data over and replicating it and the value there is you know Unity catalog is a good example governance is managed from one place

5:41the moment you replicate this data you lose the ability to govern that data and this way that data still is governed in datab breaks and only what's what's available to see in Salesforce will be shown so you see here I have my uh data coming in from data break so I have some order data coming in some lifetime value data as well as contact engagement data and there's no records in here and that's because it's just pushing down

6:03that query once I've created those data sources and uh Forge that connection between Databricks and Salesforce then I'm starting to basically move that over to the silver and gold layers and you'll see that here as I start mapping it to a canonical model right so I've got data coming in from the left side here which is my external Doos as we

6:25call them data Lake objects and my dmos here so I can have various Doos being mapped to One D this is essentially to make it efficient for analytics right so you have um all the different columns that you may need to slice and dice your data so your DMO is what you're going to use to expose that back to a user and

6:49you'll see here my DMO here is that order data and now if I fast forward to a sales user what that looks like for a sales user is my customer as an example is Lauren Bailey Lauren Bailey has been a longtime customer of mine I'm not a database administrator I'm not a data analyst I am just a sales user that wants to sell

7:11right and so I am interested in the things that are happening in other systems but that relate to Lauren Bailey so we'll see here as Lauren Bailey is a customer of mine I want to keep track of all the orders she may be purchasing making on uh the POS system that POS system is not going to connected to Salesforce it's connected to Databricks which makes sense and then with data

7:34bricks is lak house Federation into Salesforce I'm able to visualize that data in Salesforce you'll see here all the orders this data actually does not reside in Salesforce it's just a window into Data from data braks and the same data can also be visualized using CRM analytics which is essentially Tableau Mary Salesforce and the product you know is CRM analytics

7:56there but um I'm able to visualize that data as well now as a sales rep that's great but how do I actually do something about this so um we have our own um llm generated email solution we call it prompt Builder and it's and it's powered by Einstein Einstein is the

8:19Gateway between open aai and all the other llms and Salesforce uh that basically creates a trust layer between the llms and our CRM data so here as a sales rep I want to email Lauren and when I click on draft with Einstein I can use one of my prebuilt custom prompt templates that helps me

8:41upsell based on information coming from data breaks this is important to note the information's actually in data brakes but I'm able to infer that into the prompt template and in my prompt instructions say hey refer to data in data breaks and do something about it based on that data so you'll see here the last order that was made was order 999 I'm able to call that order

9:05information the amount any other details that are residing in data breaks in this email and then Define next steps for the customer either provide her with a discount or a coupon or perhaps invite her to like a um you know a members only event or something like that right so now I've never moved the data over I've maintained governance in Databricks but

9:29activated on that in Salesforce all right that that's amazing so let let me let me recap everything because that's I think I think that's a lot to impact so you you have your data in your lake house uh where you as a data engineer can collect everything and centralize the contents on top of that you can create the connector from Salesforce to their bricks and run direct queries uh

9:52on all the data you might have from any data sources that you might collect in your lake house and then on top of that any business user so non- technical can see what's available U but also use all the AI llm like einin suits um so that you can for example craft these email space and templates and everything and because you have all the contents you reduce all the kind of hallucination and

10:15everything and make sure you trust the like the agents wow that's precisely precisely and with the prompt template it's uh let's suppose uh the order value was like 4,000 and let's say her average order value is 6,000 so this is well below her order order value we can input that logic into the prompt

10:38instructions and then therefore change the kind of template that's generated as a result right so there's a lot of cool things you can do with it but honestly the value is being able to activate on that data and data breaks because there's so much data and data breaks now I can visualize it and act upon it and I don't have to be you know bound to a bunch of data engineering pipelines and it's just zero copy connection right

11:00very very low on ETL as well and you can just generate this the text and then customize it a bit and make sure to drive the business so let's see what's in the background of that thank you ja for showing us you know the nice front end but in Databricks we were holding all that data and making sure that it was available for cell Force so let's go

11:23in Databricks and see what was happening in the background so basically we can see the table directly in the catalog uh the table that you were doing a query uh directly that we saw all the orders was this table here and we can see the simple data that we had all the order numbers the order value and that's how you were creating the nice graph for

11:46that you know person uh Loring so uh basically this is where the data resided also you had something prediction at the top and we needed to create a model for that prediction so basically we had a training data set created so that we had all the information and I am not a data

12:08scientist I only play one on TV so I you know created the uh the model with Jag uh using our feature called automl so basically a few clicks and you create a multiple models you pick one at the end so let me show you how we created that we just clicked on new automl experiment basically selecting one

12:31cluster that was running so a compute on our side the ml problem that we wanted to uh to do basically classification or regression or forecasting uh let's just pick a classification the training data set that I showed you this one here this one here and the training data set and then the target which was the turn

12:53you know was the person going to turn or not and basically give it a name and a few options if you want to customize it a bit but as soon as I click on auto start automl it will generate multiple models and try to um to have the best model for a problem currently I won't go into and click it because you know it's

13:15a bit too long but I'll show you the end result of it so basically the end result is you know fully open source fully customizable um it's all the experiments with all the notebooks and you know everything that you need to recreate that experiment basically and it also gave us a a notebook for the best model and also a data exploration notebook so

13:39let's just review the best model right away as you can see it's a complete notebook with all the code it's not a blackbox it's like you can see everything and modify it if you want and rerun it if you want but that gave us the opportunity after that to take that experiment create a model and then serve

14:01it so let's go see how we create a model let's close that and basically you just have to take one of the experiment and we already created it but it's a few clicks just to create and register that model in the model registry and you can see that we have multiple inputs and output for that model and that's what

14:23Jag is going to send us you know the inputs and we're going to send uh the output basically but after that we can use the model for inference so with just a click I can do real time inference so create an endpoint for a rest API endpoint that will be up and down depending on on you know the queries coming in and I can select the version

14:45of my model take a name and specify which compute size I want if I click on create end point it's going to create it if I want to do Delta life table streaming data coming in I can can do that directly here a few clicks it's going to create my my Delta life table and be able to do it in a streaming fashion or if I have a table that I want

15:09to do a batch inference on it I can also do that so I can just take the input table and apply the model directly on it so three options for this use case we selected the real time one and it's already created so I won't create another one but basically if you go in serving here you will see that this one was created there's one model on it and there's not that much because it's a

15:31test but you can see that it was used this morning a bit and there was a few predictions this morning since we recorded this video and you can see that it was not using anything and it's it went up in a few seconds so that you know it was live so pretty easy to create the model with automl and also provision the model and then Jag was

15:54able to use it and show that there was either a turn or not for this specific end users Jack I have a question for you uh how can we integrate the model uh the arrest endpoint or the model serving endpoint in Salesforce yeah so um with data Cloud there you you don't just have the

16:18ability to bring your own data you also have the ability to bring your own model um so I'm going to share my screen and show you what that looks like going back to that record page we saw earlier right going going into Data Cloud you'll see here as we talked about The Medallion architecture your your gold silver and bronze tables we also have Einstein Studio Einstein studio is where you will

16:42host and store all your models the ones that are built on Einstein or the ones you're bringing to Einstein so Einstein Studio acts as a broker between data breaks in this case and Salesforce so you'll see here that let's say I want to create a new predictive model or a foundational model model Einstein studio is where I go for that in this case I want to create a new prediction model so

17:05here I have two choices either I can build my own model from scratch just on the Einstein platform it's declarative it's simple similar to automl in that case or if I've already got a model built out in data breaks I can call that model and this is where Usef that connection is broker between data breaks and Salesforce right so I would authenticate in um important to know

17:29here it's very very case sensitive so your token actually and this is for the folks that want to implement this when you go into your secret key start with Bearer so b a r r space and then input your token information uh Vincent and I spent two weekends trying to figure this out and we got it figured out but the syntax was very sensitive um and so

17:54you'll see here this is the prediction format right very similar to what you saw here it's basically a Json payload right um so as long as your schema matches with the schema expected in Databricks the um as well as the metadata so the type you know the uh the types of fields matches back to Salesforce you're ready to start using it so when I go

18:18back here I connected the endpoint I review the schema and I match the schema then for an end user I have a few options so I can either create batch predictions for all my data all at once and create maybe a nightly prediction job or I can stream predictions in so you'll see here I've

18:40already built out uh a model here that's the one that's calling the one Vincent just created right so in the Integrations you can set up these prediction jobs to run in batch or streaming and this is how it's going to go back onto the user record so now that we've seen the back end of it let's see what the front end experience looks like

19:02so going back to Lauren Bailey's page right back to you know wearing my sales reps hat not only is it important for me to be able to sell to Lauren as much as possible but I also want to manage her loyalty I want to make sure that she's a happy customer right and thanks to data breaks now that prediction of churn metrics is actually appearing to me in my on my homepage so I look at Lauren's

19:25record and I see okay the customer is likely to turn I should do something about this here it's a it's a very Boolean yes or no answer but you can have predictions that give you some sort of like likelihood to turn so if it's 70 80% whatever that may be but for the sake of this demonstration We've Got U you know this here that hey the customer's likely to turn we should do something about this so I'm going to

19:47send an email to Lauren again and hey you know let me try to fix my fix this issue before it becomes an attrition problem right once again with Ein I'm able to generate an email that's going to take into Factor what's happening with Lauren as a customer and if she's a happy customer or not so here in this case I'm trying to

20:12pitch something I'm trying to sell a product to her but Einstein's going to intercept and say hey hold on a second she's likely to turn so we still want you to send the email but let me suggest perhaps as a token of appreciation let's offer her 10% discount and let's send her maybe a survey to collect her feedback on why she's likely to turn so

20:35Einstein once again is smart enough in taking that prompt information from Databricks seeing that from Databricks we know that she's going to turn and then send an email accordingly so Lauren calls us and says hey you know I'm not very happy with the monthly charges I'm paying way too much per month with you guys um I would like to be at this like the $10 sweet spot for example okay and

20:57so here we agree you know what Lauren you've been a customer of ours for like four years we're happy to give you 33% off right and then go ahead and change that monthly charge as soon as I do that it's taking all the information on this record page and sending that back to that model to confirm that the prediction is still true so now she's

21:21not likely to turn based on new data that we just shared with the model we reduced her monthly charge and that had a positive impact on the customer and now we know she's not going to turn so this is how we could first create that brokerage between Salesforce and Databricks for models and then apply that in various ways I just showed

21:43you one use case but imagine for a marketing user um based on the prediction you can set up marketing campaigns right so you can use the prediction as a way to separate the folks that are going to churn versus the ones that are loyal customers right so a lot of different use cases you can apply to it yeah yeah yeah and I imagine the

22:06example is just you know prediction yes no but you could do much more advanced thing like uh maybe next best action based on the CH risk um so for example 10% discount could be an actual you know recommendation from some from some model um and I imagine also uh like the model could even be some kind of llm AI P implementation giving you the actual

22:28recommendation running on bricks you get the information on sales first you usein to you know send the email with all the suggested action and like you you almost don't have to do anything you just you know click Send that's it exactly um so I'll I'll give you guys an example of one of the customers what they're doing is they're applying external models to product usage so

22:54let's suppose you know you're trying to buy a product and it's an e-commerce store and they're figuring out based on um how much they're selling and the models you know what seems to be the root cause of it so in this case is it pictures of the actual merchandise on the storefront is it the description and using llms to recreate new descriptions based on that

23:16feedback from the model right so now you're really closing the loop between your data science teams your data engineering teams and your front end teams right yeah and just wanted to add something I think it's interesting to share that uh you talked about bring your own uh bringing or bring your own data I talked about lake house

23:38Federation we're going to also have the possibility to inest data with Lake flow which means you just go to Lake flow and your credentials you're going to inest data as part of the um data Engineering Process I know prepare some some some some data and then serve this data data in in Tableau for example can

24:01be this that can be used to serve all dashboards in Tableau or maybe train uh ml models that can also uh send back to or be used in the in the in the platform and you can track everything like which and that's how you see sales Forks is working super well with uh data braks and for the records I know it's going to be funny but I spent 10 minutes

24:25understanding the logo and now I just see two hands in know which means data breaks is working well with Salesforce I thought it was a third third logo I I I you know that's good feedback I should find a different handshake Emoji I don't think that's a good one but yeah you're absolutely right I with lak house Federation it will close

24:47the loop so well Yousef right so you're constantly getting new data that you're feeding into the model you're refining the model then you're calling that model back in Salesforce so yeah I'm really looking for to that as well and I like the Tableau angle as well and you can also for the records you can track the queries coming from Tableau if you use system tables

25:10and the um query uh if you go to query uh you can track the down uh let's say tool used and you can filter by Tableau see all the queries who done this query the duration of the execution if the data com in from the cache so it can be super super inform cool information in order to optimize or see who who's consuming more data in U

25:36in your T dashboards yeah that's a good point because every time a page refreshes here it's pushing down a query to data braks so yeah how you can optimize the data brakes will be very valuable you're absolutely right about that yeah it's going of tracking the usage but also tracking the cost can be also very important for for customers for the

25:59phobs part like how much does it cost to update this dashboard in Daily base or weekly base or monthly base and also uh for troubleshooting for the lineage this is also something important data is coming from Salesforce and ingested in in I don't know uhing this table training this model so in case you have a problem you know from where from where this problem is coming

26:21you can to fix it easily or quickly yeah I I would love to see that um on the I'm guessing it's coming up on a future next gen lak House episode right yeah nice so thank you Vincent and jack for this uh demo it was super interesting to see this how those two

26:45platform uh works together it was very very useful to see this integration of the models integration of the data and how this works seamlessly uh together thank you again for uh this demo and I hope to see you soon in next lak house awesome thanks everyone appreciate the call thanks a lot bye