Serverless Workspaces: Deploy Databricks in Under 2 Minutes

0:04Hey folks, today I have the pleasure to host for the second time Retana Das a product manager at data race. Hi Rita. >> Hello. So great to be back. Thank you for having me again. Um just for everyone I'm Rea product manager at Databricks. Been here for about three and a half years and I am super excited to be talking to you all about serverless workspaces.

0:26>> And you literally spoiled my question. tell us more about serverless workspace. >> Okay, perfect. So, let's talk about serverless workspaces. So, they are a new capability that was recently introduced as a public preview for Databricks customers on AWS and are coming soon to Azure and GCP on public preview. But today, if you're an Azure and GCP customer and you love what you hear,

0:48reach out to your account executive and we can get you set up on the private preview. Um, but yeah, so let's get into it. So with a serverless workspace, customers can simplify the process of creating new workspaces to unblock their teams and get them started on new data and AI projects faster. So say that again, gets you started on workspaces faster. So it accomplishes this by utilizing serverless compute and default

1:11storage to reduce the amount of infrastructure administration that that you have to configure upfront when creating a workspace. What's the difference between what you're showing right now and the Databricks express setup? >> Great question. We get this a lot. So, they're very similar. Um, express setup is mostly for new customers. I believe you get credits. This is really for existing customers. So, any existing

1:34customer today can go and create this serverless workspace that all of our new customers have been fortunate enough to be able to be trying out for the last I think year or so through various channels. Um and it's yeah so it's bringing it to our existing customers. >> That's amazing. Customers can just go and spin up a serverless workspace and they have everything ready without any

1:56configuration. >> Nope. Yeah. So just to go in a little bit more into how it works. So this really is a new way to create workspaces that shifts the underlying infrastructure provisioning monitoring and maintenance responsibilities. All that comes to Databricks. We're going to handle it all for you. Um so you can create a workspace in seconds with serverless compute. Unity catalog and workspace storage are also provisioned which are backed by default storage. So

2:19you can retain familiar governance and workspace controls. Um you can important thing to know which might be your next question. You can attach your existing oh sorry you can attach your existing cloud data through unity catalog just like in classic workspaces. So you can bring in your own external storage at any time that you want to a serverless workspace. So even though you start with

2:42Databricks managing that storage using default storage, you can always bring in your own external storage. And another thing that's really great about serverless workspaces is this simplified network security experience where instead of operating that gateways or firewalls or private link endpoints or security egress zones, you just define serverless egress policies and serverless private link and enforce them for all serverless workloads in the

3:04workspace. Pretty simple. Yeah, I mean removing the burden of handling gateways, firewalls, private link, I think the that's just amazing. It will simplify life of many many people. >> Oh yeah. >> And can you show us how we can create this uh workspace? >> I thought you'd never ask. Let's go into it. So I am in the an account console as an admin. Um I'm in AWS where this is

3:28currently in public preview. Um so what I would do is I would go to workspaces. I'm going to create a workspace. I'm going to give it a really creative name like Rea's test. And you'll see I already have serverless selected here. You can still use you can still create a workspace using make a classic workspace using your own compute, your own storage, but now you have this option to use serverless compute with default

3:51storage. So I'm going to click that one. I'm going to select a region. Unfortunately, this test account actually limits the regions I can see. But for all other customers that are using serverless, they'll see every region available that serverless is in, which is pretty cool. If you want to try out a new region, it'll automatically collect connect to the meta store you have in that region. And then I'm going to just click create. And that is it. So

4:13this workspace should spin up in under a minute usually. >> 1 hour later, >> it's running. So here, all right, great. So it finished running. I can see my test right here. I'm going to open up my workspace. And here we are. Look at that. In under a minute, we have a workspace where I can do SQL, data engineering, ML, anything I

4:35want in seconds. And so, just to give you a little tour of what's going on in here and show you how powerful it is. I, as you can see, do have access to the cataloges in my system. I also have some sample cataloges. I'll, as always, let's look at how the weather's going. Pick something in the catalog. Let's look at some sample data. I do need to push this just for the first time because I spun

4:57up the workspace. But here I'll get my my cluster started, my compute started. Here we go. All right. And then I can go right into ask Genie on this data set. So I can already start by saying show sample data. So think about it. In less than two minutes, we are already in a catalog. I'm already in a genie. I'm already asking questions and I didn't

5:20have this workspace two minutes ago. Um, so that's some of the really cool stuff we can do here. Another thing I'll show you just to highlight which customers really like is a lot of them have a use case that they want an isolated workspace. They want to spin it up fast and so it does give you great isolation off the bat. So you'll see that when you create a catalog and I'll just show you. Let's pretend we're going to create a new catalog. >> The catalog um catalog segregation or

5:43catalog binding with by workspace. >> Yes. So unlike with classic workspace where cataloges are by default bound to all workspaces, these are only bound to the workspace to this one workspace. You can obviously very easily go and change that and open it up. But what's great is that you know that anything you create in here is all going to be just bound to this workspace. But you can update that. And as you can see here, I could always

6:06go and change the type of storage I want. So if I clicked off default storage, I can now add an external location that we discussed. It's that simple to bring in external storage. That's amazing. The deployment can fit in a real or YouTube shorts because they're like two minutes. They can up to two minutes. >> We're all gonna get so much time back. We're gonna be able to finally have hobbies. It's going to save so much

6:28time. But >> and can you go click on compute? So yeah, >> since everything is serverless, they will be able to find only the SQL warehouse will be serverless only. And for notebooks, they will be using serverless notebooks. Same thing with the workflows. Databricks will choose the right size. We'll have this horizontal and vertical scalability

6:50which brings reliability, performance and of course cost efficiency. >> Yes. And what's really cool is there are a lot of things today that are only available in serverless like model serving, vector search, AI runtime. Those are some great things that you'll get automatically with serverless workspaces that you can only use with serverless. People are probably thinking, "All right, serless sounds great. Should I just go all in on serverless the next time I create a

7:12workspace or are there times I should use classic? And so you should always choose a serverless workspace when you want the fastest path to a govern environment with minimal configuration. Today's serverless workspaces can do most things. There are few edge cases though we are working very fast to fix them. So we say use serverless workspaces for most things especially when you need fast setup with fully

7:34managed infrastructure. It's great for enabling business users. You can do Databricks one AIBI dashboards Databricks app. You can perform exploratory analytics. You can connect to SAS providers. You can use Genie test mosaic AI features prototype ABI features without production setup. A ton of awesome stuff. The things that we're working to close the gap on with classic um and when you should use classic is of

7:57course if you just want a full control over your infrastructure and assets. If you want to connect via on-prem systems or private databases if you want to perform a IML workloads needing GPUs if you want to use scholar or R. If you want to stream data with default base uh timebased default triggers those are just some times you could use you should use um classic but again these gaps were

8:19filling them and the you the reasons to use classic are getting smaller and smaller. >> Keep having a look at the um serverless release notes where you will see everything that has been added to serverless and you can check uh all the improvement that has been done for the last year. >> Yep they are working very hard. Um and then yeah some other things that frequently come up. What is the cost? Consumption is just charged to the

8:41billing mechanism of the account and it's available in the Databricks console. There are a bunch of great thing bunch of great features out there available today to track your spend all with serverless. So that's great. Like we said with classic compute people might say can I add classic compute? You know I can add external storage. What if I want to bring in classic compute as well? That's coming. Hang tight. Next year we'll bring that in. Um can service

9:04workspaces be enabled via API? Yes. And how do you sign up right now? If you're on AWS, go to your account console, click create workspace, and you can just play around with one. You've got one, a serless workspace available today right there. And then GCP or Azure, please reach out to your account executive and they will talk to me and we'll get you signed up for the private preview. >> Amazing. That was Crystal here. Thank you so much, Rita. And see you very

9:27soon, I guess. >> I hope so. It was awesome chatting with you. Thank you all. Fight.