Databricks AI Assistant: Auto-Fix Errors, Generate Code, Free

0:04hey folks today I'm with uh Weston Hutchins head of product applied Ai and developer experience and we are here to talk about dat breaks assistant this feature has been uh unveilled almost I think one year ago and here I have the pleasure to have Weston with us hi

0:26Weston hey thanks for having me excited to be here how you doing I'm doing good yeah things are exciting uh assistant is uh is you know we got a lot of cool features coming out so it's just been a really fun space to be a product manager for yeah and I remember I talked with you the first time I think one year ago I I had Rel like when I released the

0:49recording about getting started with data breaks assistant and a lot of people were requesting this feature because uh it was something exciting and and I've seen like the number of users keep growing and I've seen so many people using it so can can you tell us more about the assistant how it gets how it did start and and all the changes

1:14that did happen in for in for the last 12 months yeah absolutely uh we started our assistant Journey about a year and a half ago and uh this was about the time when chaty PT came out uh co-pilot was seeing a lot of adoption and you know we had a lot of requests from our customers to build AI productivity features inside Databricks

1:37and we uh yeah we spun up a team and started prototyping and what we found really quickly is that you know by bringing AI into Databricks and into Unity catalog we were able to offer some pretty unique experiences um so most of the original Databricks assistant has been like really really focused on authoring COD authoring query authoring

2:01dashboard authoring um the types of features that that customers hit every single day in fact our our most use feature by far is diagnosed error um you know everybody hits errors inside uh inside their their coding environment uh the assistant is extremely good at at resolving some of those and fixing those automatically and we actually have some improvements that I can uh showcase that

2:23are are new coming to that that even make it better but um yeah you know I think when we when we uh we started at for authoring and and then we unveiled it at data and AI Summit uh last year uh the we had an overwhelming amount of interest in it and uh I think it's been the fastest growing product in datab Brick's history uh there's over 150,000

2:47monthly users on it um we're seeing uh a ton of pickup from across all of our accounts uh and we really you know the exciting thing that we've done in the last year is one just you know continue to improve the quality of the responses uh make sure that we are pulling in you know the most accurate data braks docs and samples into these experiences but also uh you know a big

3:09goal for us is bringing the assistant outside of just the authoring surface so you know one of the things if you've been using it for a while you'll probably notice is it moved from the left side as like a panel inside you know your your notebook um and SQL editor off to the right and uh you know a big reason for this is we wanted the assistant to be available able throughout Databricks uh and so you

3:33know one of the first experiences that's going to that we launched is diagnose job errors um where you can actually use it on a jobs error diagnosis you can use it on jobs page and see if there's an error it will help you diagnose those but we're also integrating it into search we're integrating it into our catalog Explorer page uh eventually we'll start bringing it to other Pages

3:55um uh across data rck so we're really excited to see the types of AI experiences that other teams and product managers start to design around this like assistant is really turning in from a you know a productivity coding authoring tool to a more general purpose platform uh across Databricks powered by you know a lot of it data intelligence engine you know our

4:17compound AI system that that takes in all of this uh all this metadata and actually generates our responses and and it and it did help me a lot and trust me the fact that like they moved the uh um the assistant from the left to the right was massive because whenever I wanted to use assistant I needed to go either to a notebook or data break SQL workbench

4:39like to ask some questions and I needed to switch from like from those two let's say uh NV two tabs to have the assistant now it's now that that it's on the right so it can it it tracks all my let's say questions and I can switch from different notebooks and I will still have the same context which is very important in my case I use it a lot for

5:02documentation because I know whenever we have a new release uh we all those details are there and I ask questions about I don't know volumes about uh fine tuning and I will have the right answer but also improving the productivity because I I do a lot of copy pasting and I forgot a lot to add commas at the end as as

5:24everyone or maybe forgot the group buy at the end for some reason because I'm thinking about something else and they assist can help me fix this uh problem quickly or maybe when I become lazy asking okay can you translate this squel to python or the opposite you know just to make sure or maybe because I have some issues with some I don't know I sometimes I can have I had I remember

5:46weird uh error messages it was hard for me to troubleshot they just copy just copy paste now we don't need you no longer need to copy paste it captured the errors and troubleshoot it for you and this is also something something interesting yeah you know just to like interrupt you know you mentioned something about asking for help uh you know one one of the things we ran into that's sort of interesting with data

6:09bricks assistant is a lot of the llms that we were leveraging both like you know internally hosted ones and you know partner powered ones they didn't know about new Databricks features they hadn't been indexed right so if we were launching something and you were ask hey you know tell me about you know AI bi dashboards it wouldn't know so like one of the things we learned pretty early on is we had to build our own uh Vector

6:31databases and other retrieval systems that were constantly being fed into uh these models as as rag so that they knew about the latest and greatest data brick features so that was like a big learning for us is we've improved the quality of our health responses quite a bit over the last year indeed and I I couldn't agree more with you and I've also seen like uh

6:53several changes since the first release can you tell us more about this trading and features that has been recently added yeah so um as we move the assistant out of like a a docked notebook or docked SQL editor panel um and into uh you know a Global Experience across all of Databricks you know one

7:16of the things we saw when people were using it is this need for you know a different a different user experience pattern of interacting with the assistant um mainly because if you're in a notebook uh you want to retain a lot of the context that you had from your last uh editing flow like you probably already have a bunch of prompts that you

7:38saved maybe you've you know worked with the assistant a lot of assistant interactions are pretty iterative um so you'll go and say you know maybe ask it to generate some regular expressions for you spend some time tweaking that um get the prompts just how you like it you want to save that state you don't want to have to recreate that every time you open up Databricks um as you're doing work for the rest of the rest of the week so we found this need for an

8:02interrupt driven way of starting new conversations um similar to what you would find in a lot of the other uh uh llms out there but you know you can hit a new thread uh it will allow you to create and start a new conversation so for instance if you're you're working and you get an email that a job has failed you can open up Databricks uh rather than losing the work that you're

8:24currently doing you can start a new thread it will help you diagnose those job failures you can check that back in and then when you're ready you can go back to your previous thread uh and resume your work um and so that was you know a big part of you know this experience of moving to not just a single purpose notebook authoring assistant and something that allows you to do more tasks that aren't necessarily

8:47related to coding that that's interesting and I'm curious to know how did you work with the uh notebooks team because I know that we have a awesome integration for example now you can just go and write down SL explain and it will automatically trigger the assistant to explain the query how how does it work well you know I so I my team works on

9:10the notebook we work on our SQL editor we own all of our authoring surfaces so it's actually quite easy because they uh they sit right next to me I'm able to really get them to think about ways that this assistant uh interacts with all of our editing surfaces so yeah a big part of us you know a big part of our philosophy with the assistant um especially on authoring surfaces we want

9:32to remove toil we want to help um resolve problems that customers hit every single day faster better um you know maybe you aren't familiar with the latest syntax or maybe have an error in a Databricks run time that you need to resolve these are things that people hit all the time which slows them down and so a big thing for us is trying to identify those those like targeted pain

9:56points uh that people hit over and over and over again and really imizing around those flows so yeah there there we have a lot of you know new things coming out but really we're about just making data data scientists machine learning engineer SQL analysts a lot more productive uh writing code inqueries inside data BRS yeah and and just this month I think

10:18released some features like about possibility to find tables from the assistant or find queries um like improve the um uh or prey I'm I'm more struggling struggling with this word like improve the format of for format code readability renaming and also the uh the settings so can you like before we move

10:42into the road map can you show us a demo for folks who have never used the assistant yeah absolutely let me uh let me pull up my screen here and we'll kind of walk through you know I have a demo that showcases um more than just you know the assistant kind of demo some of our other AI powered features um in data bre but they're they're all related and so um

11:06you know we'll we'll jump into this right now this is you know I have a table I I I downloaded the top 1,000 IMD movies uh in a CSV file I uploaded it to Databricks and uh the first thing you'll notice here is that you know when I uploaded this file uh all of the table descriptions and the comments uh they hadn't been added yet and really metadata is a really key part to how all

11:31our assistant works and how a lot of these AI systems um uh really provide more accurate relevant answers is you really need quality metadata so one of the things that I clicked over here um is this AI generate comments and I actually used it to generate all of the comments here and rather than accept one I just kind of wanted to show a few examples but um this makes documenting your data assets extremely easy um you

11:55hit AI generate this is actually running off of a fine-tuned uh datab bricks hosted model uh we used Mosaic AI tuning we used a bunch of synthetic data uh to uh add descriptions for common column names and what's really neat about this whole feature is that once you use these AI generate comments you save them in there they're indexed by our search system and so I can start to go up here

12:18and ask questions like you know where is my movies table and um we'll click over here this is our new full page search results um but you know something that we've added for tables is that we actually support natural language queries um and so uh when I when I actually type this in um a few things are happening one we aren't just doing a

12:40keyword match like movies will show up here but honestly if I go up here and I type you know film that will also return this result and uh we use uh relevancy we use like a lot of signals the fact that I use the word my in here um gives you personalized search results so uh this really like the assistant in

13:02a lot of these AI systems end up boiling down to search and retrieval problems uh so uh we're really excited to continue advancing this and and display relevant information but what's what's nice about this is that you know we've integrated search as you mentioned directly into the assistant so if I open it up here I don't actually need to leave my authoring experience um in order to find

13:24the tables I can ask the same sort of question like where is my movies table and this calls the same semantic search API on the back end you can see that my table is returned here and then I can ask it to query um uh ask it to specify this table and you can go in here and say you know things like uh query the

13:46first 10 rows and it will uh it will use context from the table that I've selected um it will uh send it off to our models and it will turn accurate text to SQL and what's really actually need about our assistant is it is itself a notebook so results that come back here um are actually executable code um so this runs

14:09as part of your cluster it runs as part of your compute um and you can see results in here it also makes it really easy to go put it into a cell over here if you wanted to and run it there but the fact that you can use the assistant like a notebook makes it really really really easy to iterate um especially as you start to um you know find questions and refine queres and get more

14:31accurate text to SQL so you know one of the other things you can do is rather than even using the side panel um you can come in here and we have an inline assistant you can click this button here um or you can uh uh click the generate Tab and you can just ask it directly in line now and this is a you know code first or a SQL first experience and you can say things like um you know what are the top uh

14:54genres and our assistant will go and generate query you can actually run it before uh before accepting just to test that the results look good and yes this looks great I can accept this this query now and you can see that drama is the most popular uh movie genre that we have in this this sample data set probably

15:16not a huge surprise a lot of the top 1,000 movies or dramas um and now we can start asking more interesting questions uh so I can do things like uh what are the top 10 longest runtime movies um and then I you know runtime is actually string and like one of the things I can tell it is you know runtime is a string that looks like 99 minutes

15:40and the assistant will generate the appropriate casting code for me um and what's really cool is I can put this in my metadata it will get this correct let's go ahead and run this example um and you'll see that it actually pars it correctly so when it does the order buys and the sorts um it's actually using the integer values for the strings rather than doing string comparison so this

16:02just makes it really easy um to go and uh iterate over code and like cast it um date manipulations is a really common use case that we see from customers as well um so I want to demo a few other things um about uh a few months ago we launched a feature called assistant autocomplete uh and for those that are familiar with you know co-pilot style autocomplete as you type that's the

16:24experience we were trying to build uh and this is uh you know it worked for pip Pyon uh it works for SQL as well uh and so I can ask it a question so I can say like write a query to find um drama movies with a rating greater than 8.0 and uh you'll see that the assistant starts to auto fill in my answers for me

16:49um as I am typing uh this is a a really powerful form of autoc completion that takes into account code Snippets that you've used uh popular tables maybe ones that you favored um and we feed this all to a Databricks host in a Databricks tuned model uh that runs really close to our customers so like a key tenant of

17:12this whole system is that everything has returned in less than a second we have to have really really fast latency for this uh so we're really excited about this a lot of customers have loved this feature it just really AIDS um in the speed you're able to write code and write queries and site data RS um and then finally you know I just wanted to sort of show we're doing more than just code authoring you know a big part

17:34of using the assistant is actually coming in here and seeing other ways in which we can leverage AI to help you let's say document your code cells so maybe you want to clean up your notebook before uh sending it off to other folks at uh at your company and so now you can come in here um and see that you have a table of contents we've also added um a lot of these slash commands

17:58so these are really simple you were mentioning pry earlier but we have a number of these now um where these are just quick shortcuts like common tasks that we see developers using every day um so a common one that that we see is like people want to add uh comments uh to their code and queries maybe clean it up before they check it in to get um so that these are like really helpful

18:20utilities that you can invoke um through the assistant and we're we're working on you know a number of more things there so that that's a quick demo um I did actually want to jump you know you had mentioned like what's coming next I I have some slides on road map uh these are features that we'll be launching over the next uh you know the next quarter so the next couple months or so and things that I'm pretty excited about

18:42um the first is that you know we're moving this assistant to other pages one of the first uh user Journeys that we're working through is diagnosing job failures uh this is a common case we see people hit syntax issues see see people hit runtime issues we really want to make our assistant helpful so um diagnosing jobs will uh let you open up an assistant on a job's failure page

19:04you'll get a fix for it you can go back into your notebook or change your cluster configuration to go and fix that and we're really you know hoping that people get a lot of value out of automating uh job failur uh through the assistant and then uh assisant Quick Fix so this is it should be rolling out actually now and so customers will probably start seeing this pretty soon um what we noticed when people were

19:27hitting diagnos a is that the vast majority of them were just simple typos uh maybe you had a trailing comma maybe you misspelled the column name that you were trying to use uh so we we really tuned our prompts and offer this very low latency Quick Fix experience that picks up things like syntax errors missb built table column names things like missing Imports Group by statements uh

19:50you were mentioning that as well and it's really optimized for Speed uh so we want this to be fixed in you know less than 1 to 3 seconds you shouldn't have to take your hands off the keyboard uh this is really just about you know an autoc correct as you type experience and we're we're really uh excited to set that out there and see how people react to it um this is something we're also working through uh is you know

20:13optimizing queries now this llms are not the best tool for you know really deep query optimization but they can help with uh a bunch of simple fixes that we see lots and lots and lots of SQL users hitting um the biggest one for us is uh you know winding over uh you know making sure that you have wear Clauses for for winding and partition Keys um those

20:36types of things and so um we're the goal of this feature is to really you know let users run it if we notice a query is taking a long time we can suggest autocomplete we can suggest um optimization fixes maybe limiting the number of rows or maybe reducing the amount of data that's queried as part of it so um another one that's going to be coming out in the next couple months or so and uh we're really working through

20:58and trying to figure out those use cases that work well for this uh assistant usage reporting so a lot of customers have asked for um just information about how it's being used inside their company um so we now have uh assistant integrated into our system tables so you can go and query usage across your account um or workspaces we also have pre-built dashboards it says

21:20Lake View I should update it to aibi dashboards that provide insights um uh about who is using the assistant um across your works bace and then uh lastly you know the big thing for us is um you know a lot of customers want to use only datab bricks hosted models um and so we've been we've been really working hard on uh spinning

21:43up uh open source models that are fine-tuned uh using data break samples and queries um and running these on data brakes infrastructure within your governed estate so um for for customers that don't necessarily want to use um Azure open AI or Sy cross Cloud traffic this is a really good option for it um this is going this is in private preview right now and will be going to public

22:05preview pretty soon uh so yeah that's like a quick overview of uh of what we have coming up in the next couple months um I always love to hear feedback from from customers and users about the types of things that we can help with the most I think half of the 10 you mentioned uh are something that speaks to me because I do a lot of mistakes

22:27while cing and I've been using those features because they're enabled for us before uh the customers I've been benefiting from them for a very very long time and to be honest I got used to them so those are really tiny tiny let's say features but they do change the uh a lot or they improve a lot the productivity so maybe for now you're not

22:49used to them so for you it's going to be just something on the top but trust me it's going to change your a lot the way you see your perceived notebooks and that's why trust notebooks are very interesting I know a lot of not everyone is fan of notebooks sometimes others prefer V vs code but trust me Assistant Plus notebooks are I think the mo the product that are getting the fastest

23:14improvements you've seen I think if you're familiar with my with the channel you've you've probably seen Jason who is also working on the notebook experience and how they added the debugging experience the interaction with the assistant so this is where we need to go and last question for you Weston what about the pricing for the assistant now

23:36I'm talking about the current situation so the assistant is free it's available at no additional charge um and you know a big reason for that is we wanted these features to be available for everyone um everyone we've talked to that's use them even internally it's it's really funny to hear you you uh you know speak so so fondly of the features it's really hard to go back when to use

23:59them um I've been in Works spaces where the assistant's been turned off and I was like please like how do I turn it back on um and so a key tenant for us is that we wanted this to be free and available at no charge for all of our customers um we really want everyone to be more productive inside data BRS I think that was uh click clear so

24:22and if you have if you're not using the assistant go enable it and start using it and trust me there is no going back you're GNA love it thank you so much W Weston for all the uh interesting information you shared with us and maybe we're g to see you in six months to Showcase all the new features yeah I'd love to be back we

24:44have a lot more stuff uh that we're Brewing up right now so I would love to come back and show all the new stuff in about 6 months