Enterprise Dashboards with Metric Views & Genie Analytics

0:05Hey folks, today I have the pleasure to host for the second time Een, product manager at the AIBI team. He's back. He will walk us through the amazing improvement that has been added to AIBI dashboards for the last couple months. Hi E. Hey, how's it going? I'm always happy to have you. So, please showcase

0:30everything your amazing you and your amazing team have built for the last couple months. >> You're too kind, but but happy to. Okay, let me let me share my screen and then and then we'll get into the actual demo. So, at a high level, uh if you're not yet familiar with AIBI, the quick introduction is that AIBI is Databriggs' native BI offering. So, there's two key products here, uh Dashboards and Genie. Um, and both don't require any extra

0:53licenses. They're natively governed and secured by Unity Catalog. And dashboards is sort of our new AI powered approach, canonical reporting with your rows and columns of charts and so on. And then Genie is a conversational interface. And you'll see in the demo today that we're really bringing these two experiences together. So you can do a lot of the follow-up questions you traditionally had to add more charts to on a dashboard

1:16now in Genie. And we think that's a more modern and more scalable way to do BI going forward, you know, with all the new tools that are available. And you know, if you're wondering why should you consider AIBI as an option in your in your BI toolkit? Well, first of all, you know, uh we just wanted to make sure that the their governance story was just a lot simpler than maybe if you had to go procure other tools to do BI in, right? So that means if your data is

1:38already in Databricks, your users are in Databricks and you've set up all of your governance to unity catalog, there's no separate licenses to procure like I mentioned, there's no kind of shadow data warehouse you have to manage. everything is done in one platform and we find that admins, you know, have a lot easier time managing all of that when they only have one tool to take care of. Um, it's also good at analyzing data scale scale data, right?

2:00So, you know, there's no limitations on volume. You don't have to set up extracts. You don't have to purchase live query performance packs. And then lastly, you know, I know a lot of organizations now are looking at, you know, how can they leverage AI to be more effective. We are, you know, I think in a unique position at data bicks to give you, you know, a really highly powerful and accurate agent to help you do your text to SQL and analysis task because, you

2:23know, unlike other tools where you kind of give them a subset of the data that you want to analyze on, you know, Databricks can actually look at your entire platform and say, hey, like what are the most common queries you run, you know, what are your jobs and and all of that as context for the for agents to understand, for example, how you define revenue, right? or like how is the cadence of data being updated in your platform. Okay, so that's the preamble

2:45and you know we're here to talk about dashboards, right? So, you know, the team has been growing. Um, and we've done a lot of things in the past year to make dashboards kind of richer for the viewer but also easier to build for the author. the big items that you see on top there that I'll show in a second. Multi-page reporting, the ability to drill through from page to page, the ability to embed not just for your

3:07internal users, but also for completely external use cases, right? So, if you have downstream partners and customers you want to put dashboards in front of, you can do that with AIBI. You can write custom calculations. Uh, so you can write a SQL query once and then do all of your aggregations as on top of that. And then you can also define themes. So look and feel really matters. And now you can set up, you know, fonts and colors that are consistent through your

3:30whole dashboard. And the team is actually working on bumping those up to the workspace level as we speak. Right now, there's a longer list of items that I'll show you. Uh, but I think one of the things I wanted to do you today is to take a step back actually and, uh, kind of talk about the other surrounding work that we're doing in AIBI to make dashboards and Genie just more consumable and maintainable in the long run. The first thing we wanted to call out is this new experience we call

3:52Databricks one. So, if you haven't heard of Databricks one, uh there's a good chance today you're building dashboards in Databricks and you're inviting users into the workspace and and you might be a little nervous about that, right? Because your dashboard viewer might not be somebody who should be have access to notebook. They maybe don't care what ML toolkits are, right? They're really here to just look at the dashboard you shared with them. And so, pair of Databricks one is a new

4:15consumer entitlement. uh which means now you can invite users into Databricks and only give them consuming ability uh on dashboards genie and apps and they get this nice home screen it's super userfriendly you can directly ask questions that I'll show in a second look at your dashboards and then move on with your day right and this really changes kind of perception on Databricks being hey like this is a pretty technical product for for your data

4:37science and data engineers of the world now Databricks is also catering for kind of the business user who's just only here to consume insights before moving moving on what they're actually need to do in their day. And the other thing I wanted to show was this concept of Unity catalog business semantics. And what we mean by that is, you know, conventionally today pre-business semantics, you were defining and doing your data modeling within the dashboard layer. Um, and one of the key trade-offs

5:00of that is that while it's flexible, um, it made it really hard to share common calculations and SQL expressions between dashboards. And that context is also kind of hard to transfer to a genie space. And you know if you're coming from a PowerBI or a Tableau or another tool you might have had a semantic layer in those tools that your dashboards and those tools shared. But you know in you know large organizations you know

5:23typically we see that different departments have different tools and they all have their own business semantics embedded in those tools and then as time goes on definitions start to drift. So one team makes a tweak to what a monthly active user means but another team doesn't pick that up in their tool and now you're reporting on kind of slightly different dimensions and and slightly different numbers. the business semantics in Unity catalog is designed to tackle all of those issues right so one it means that within the Databricks platform in unity catalog

5:47you can define you know your measures your dimensions your synonyms your certifications and then you can directly use them in dashboards and genie but we're also working on making these available in our part in our uh you know other tools that you know connected via third party as well so the goal is hey like let's give you guys a centralized place to manage all these semantics and then you know hopefully drive more consistency in the future in your organization

6:10Okay, so that that's kind of the all the slides I wanted to show, but what I figured I'd do now is actually start with Databricks one to show you guys what that experience looks like. So let me do this. Here is a datab bricks one environment. So I'm a consuming only user at this point. Um when I come into Databricks one, I only have you know I don't have a few options to choose from

6:32because the the workflow is actually pretty simple, right? I can search for dashboards. I can search for genius spaces. I can directly ask a question. So for example, how much is revenue and then this will actually triage and find the right uh genie space to ask that or I can look at my common you know most access dashboards what's hot in my workspace or I can come into dashboards and get you know thumbnails and this much more userfriendly presentation of

6:54what dashboards are available to me and I think that domains are coming very very soon and this will simplify life of business users because they can just choose whether they want to access assets from finance team or sales team or marketing team and this would narrow down the assets they can view and thus find the right one quickly on DB1.

7:16>> Yeah, definitely when whenever that comes out uh which is very soon uh we should do another video uh to walk through that. >> And so if you imagine you've now you know opened a dashboard as a Databricks one user and this is intentionally zoomed in a bit so things are a little bit easier to see as a Databricks one user right I can just come in here now and use the dashboard like I normally would. So, you know, I have my global

7:38filters on the side here. I can filter by industry, you know, things just like AIBI you would imagine load super quickly. Um, looks like we don't have a ton of aerospace opportunities in this dashboard. So, I'll clear that. But I can also, you know, hide this, right? And then come down here. I can do things like, hey, you know, if you notice like this was unavailable a year ago. Like we had now we have spark lines. We have percentage change. So now we can look at, hey, like we have $83,000 of backup

8:02opportunities now, but a month ago we had 6,000. So we're up $6,000 in active opportunities. Like you can visualize trends. One thing we didn't show last time was now we have forecasting. You know, now it's, you know, as easy as one click to go through a line chart and say, "Hey, can you forecast this out another quarter and then, you know, Databricks will do that for you uh and with our native uh uh native

8:27function >> forecast model." Exactly. >> Yeah. Um, and to show you guys what drillthrough looks like, for example, if I wanted to understand, hey, you know, just for, you know, what are my trends and and losses, I can right click here. I can say drill to trends. Um, and because I rightcicked on close uh on close opportunities, you can see that we've pre-filled here as a drill through. And then now we can look at annual revenue by stage, but now my

8:49stage only has, you know, closed loss opportunities. So now I am looking at kind of the trends of loss opportunities over time. And then obviously that's the same here where you know like we have all of our regions but we're looking at lost opportunities to revenue for for loss for closed. And now if I reset all those to default I get all my other data points back. And then now I can continue my analysis. We've also added things

9:12like hey now you have waterfall charts to kind of visualize things like net cash flow changes over time. Um and the other things you might have noticed is we've done a lot of little tweaks to make presentation a lot nicer. So things like, hey, now you can customize the way that your x-axis labels are formatted, right? You can now conditionally color uh your your counter visualizations. You can now bold them when they're red, for

9:35example. Um, and there's just a lot more control and and a lot more expressibility in your data like this. So from here, I actually want to jump over to our edit view uh to kind of show you what what new things an author is now able to do. So this this is edit mode and and so imagine earlier what we saw was kind of business user een and business user een just came and looked at that dashboard and moved on. If

9:57you're a user, uh there's a lot more uh things you can do now, right? So, um I actually want to start with data modeling. So, if you come into uh the sidebar here, so I've already done the work ahead. So, to show you guys what it looks like, this dashboard that you're seeing here is actually powered by a Unity catalog metric view. So, when you pull that in, you know, all you do is click add data source and metric views are not demoted by this little ruler to

10:20show that it's a metric view, not just a normal table. And when you come in here, what you get is a set of measures you can immediately start using like, hey, what is my total amount? What is my average probability of winning, right? And these are defined via SQL in the in the metric view layer that I'll show you. But then you also get the rest of your fields. And it's as easy as, hey, like from here, if I wanted to visualize a new metric, for example, I can just

10:43drag one onto the canvas, right? I'm already looking at my view. And then from here, I can just grab, say, a counter. And then I want to grab a value. And in here like I have all of my pre-counted measures and then I can look at you know average probability. So that's going to run. It's going to tell me 081. And just to show you some of the formatting things that we've done. You can now go into format go into custom and say hey this is actually a percent.

11:06Right? And I actually only want zero decimal places. I can do all those things super easily. I can click title and I can say hey this is my, you know, average opportunity. And because everything is linked to one data set, right? Like I can come in here and say, "Hey, I actually only want to filter for the last 30 days." All my visualizations update. Well, in this case, looks like uh we don't actually have the last 30 days of data here. So, let's go to

11:28maybe last last 365. Um you know, we're back to 81%. Right? So, so this this is the kind of things that you can do with just simple point and click now in dashboards. Um and for this reason, you know, I I can I can delete that. But that was just an idea to show you guys what what working with metric views feel like in dashboards. Now you look at metric views, you may be thinking, you know, Een, is there an easy way to make metric views? And

11:50definitely is the answer. Uh, so there's a couple ways to do it. Uh, but one of the popular ways that we see right now are, you know, users actually come in. Um, and typically the data modeling starts at a dashboard level. Uh, so you might have a SQL query you want to run. You want to join a couple of tables and you you've basically ended up at this kind of state where this is a standard table. you've added custom calculations,

12:13you know, with this button here. For example, here I have a weighted win likelihood measure. Um, and if you click edit, you can look at the expression and it's, you know, it's weighted by the amount. So, you're multiplying probability by amount, you know, every time. And then down here, you also have your regular regular everyday fields. Right now, say you're happy with this data model, but you want to share this with other users. It's as simple as going to here and promoting it to a

12:35metric view. So you can hit export to metric view and what we do is we convert it to a metric view definition. So up up top here you get your SQL. Here you define your dimensions and then down here you get your measures. Um and let's just say you know opportunity for team right and then I can hit create. And what this does is one you know we pull it back into the dashboard for you. So

12:57that win likelihood measure that we just talked about is now added as a as a canonical measure that is now kind of promoted to Unity catalog. But you can directly start using this and if you wanted to get a sense for kind of what that looks like in Unity catalog. So we added it to the to this folder and then let's just open up what let's see yeah super helpful because you know a lot of times I think people start from a dashboard they get to a point where they

13:20like the data model and then they can promote it up to UC govern it right and then from there they can add more measures and things like that. So this is the unity catalog view of the same metric view that we just created. Um and then from here you know for example you can double check and see okay like even these definitions came through and then if I wanted to edit it I wanted to add more measures I can do that you know

13:43first via the writing straight YAML if I wanted to. uh but I can also use the UI right so I can come in here and say actually I want to add a measure I can ask the assistant to do that for me or I can write it directly from here and then obviously these changes as I add more measures will be propagated back to dashboards and genie spaces using the metric view so this is a pattern that we really start to you know recommend to

14:05users instead of defining a lot of your business measures locally and keeping them there in dashboards you can really start promoting them up and it's incredibly helpful in Genie as well right so um you might have defined the weighted moving average in a dashboard. You promote it up to Unity catalog. You build a genius based off of that and then the user comes in and goes, "Hey, what is my weighted moving average from the past two months?" And then Genie will immediately know what that means

14:27and have actually the right SQL expression to calculate what you're looking to do. >> And I think also something something it is worth adding that I think it was it was hidden. I discovered by uh by chance last time. You can also build a genie with uh and add all the um all the things you want to know like the knowledge base and so on. And then you can just grab the ID of that ID of that

14:50genie and then put it inside the uh dashboard. So people when they enable genie they can use the one that you have created for them. >> Yes. Yeah, definitely. Let let me actually show show people what what you mean by that. It's it's a pretty cool feature but it's very it's hard to find it. we've been moving it around, but one of the things we're doing at least now is making it default on so that it becomes

15:13less of a decision an author has to make. But the way you find it today is you go into, you know, three dots here. Uh you go into settings and actually this is a good sneak peek at our themes uh feature. So now you can for example define a custom theme which will be done here in this dashboard and then you know you can pick your different color palettes and so on. To get back to the point now in general you can have this enable genie icon and then when you turn

15:37it on you have two options right. So one is you can autogenerate a genie space which means hey like based on the context of this dashboard based on my visualizations and my calculations autogenerate for me a a conversational assistant or you can link an existing genie space. In this case, this is the ID. You can just grab the URL, put it in Yousef, like you said, and then when you uh talk to Genie, you actually talk directly to that space, and there would

16:00also be a link to open it natively in your full screen mode. So, actually before we jump over it, cuz I'd love to show you that that Genie experience. The last thing I wanted to mention, you know, we talked about, hey, more charts, right? You can you have more adjustability. The thing that we did is that, you know, tagging a certification is also available in dashboards, right? So, now you can tag dashboards as dashboards team, for example. You can make it certified and you know really

16:23soon all of these will also be bubbled up into search experiences and so you can start filtering dashboards by team only looking for certified assets and you bet these things are going to be in Databricks one right so in addition to domains you know if you have more granular things you want to organize your assets by all of that will be available very soon okay so let's publish this dashboard and then I'll show you guys kind of what what we've uh

16:45what we've done with the Genie space experience here last time we looked at dashboards you might remember that there might have been uh ask Genie button on the top left corner. Uh now we moved that to the bottom. So to make sure you know it's kind of always available kind of a little bit easier to find. Um when you click Ask Genie because we linked an existing Genie space, this actually has a lot more context that I was able to manually give it, you know, by curating

17:07the space and I'll show you that in a second. But for now, you know, if I wanted to say, you know, what was uh total revenue for 2023, Genie went out try to understand the question. finding the right data and as it's thinking about it um kind of the one thing I want to highlight here and and this might be a little bit of a departure from how folks have thought about dashboards before is you know I

17:29think there are two types of dashboards that are typically built today right there's one that is you know meant to go on the TV it's meant to be an executive reporting interface it might be your team's KPI tracker for the quarter and that's a dashboard that everyone logs in and looks at maybe once a week once a month or maybe maybe even every day and for those dashboards you know dashboards make a ton ofense That's uh you know, you're putting the insights that you

17:51need to know every time onto this predictable interface and people come in, they get the big picture and then maybe they have follow-up questions, but they move on, right? And that that's what we call monitoring dashboards. And those those are great, but there's another type that's not so great. Uh that's what we call kind of oneoff ad hoc dashboards you build to answer one person's question. So you if you and I were working together and you were like, "Hey, I want to understand how things

18:13are going, you know, uh, in Asia for a particular product line and I build you a dashboard, you know, you might have only needed that for a deck you were making one time to do a report, right? But then that dashboard lives in the workspace forever. We might have scheduled it, forgotten about it, right? And then somebody finds it a year down the road and starts reporting off of the same thing, but things have really changed and nobody cared to go back and verify, right? And so, you know, in

18:35those worlds, what we see is organizations ends up with thousands of dashboards where they really only want to keep like 50 to 100. Um, and Genie integrated into dashboards is sort of a big bet we're making to make that easier. So, now if you had a follow-up question on this dashboard, you'll know like, hey, I want to actually break down a specific region or I really want to zoom in on like a particular year, right? You can ask all those questions

18:58to Genie instead of having to rebuild a whole new dashboard because, you know, once this interface is, you know, used, you know, obviously it's a chat history. You can save these results, but if you don't need them anymore, you know, they don't stay in your workspace as clutter. And obviously, like I mentioned before, you can actually open the Genie space. So, that opens the full page experience, right? And now you get things like, you know, because I'm an author back to benchmarks and monitoring. I can come in here. I can actually look at the

19:20instructions. Um, I can continue asking questions as well. So, I hope I gave everybody like a big picture on kind of all the key improvements we made in dashboards this year. You know, again, going back to what I started with, it's all about kind of one, making things more consumable with Databricks one, making your data models reusable via metric views and then just all the bells

19:42and whistles that you'd expect from an enterprise dashboard tool, right? Like drill through theming now with an assistant to do Q&A, you know, more more UI configurability. Um, and and like I mentioned last time, you know, we really do welcome feedback. We'd love to hear what folks are using dashboards for and what um kind of other features they like to see. We'd love to talk to the community about that sort of thing. >> That's perfect. So, I guess I'm going to see you maybe in a few weeks so you can

20:06walk us through the uh domain.