David Meyer: Product Strategy & AI’s Future (Databricks SVP)

0:00doing it and doing it and doing it. Do not whatever you do, do not be a product manager. Engineers have a million things tugging at their time, right? They want to believe in it. So you have to enroll them into the vision and so they agree that this is impactful. Hey everybody. So today we are with David Mayor with uh one of the CPO at

0:24Databricks. one of the two CPO actually 8 two SVP products Ali CEO and the CPO by design all right thanks so much for that time I know you're super busy it's super cool to super cool to have you so I feel like you've been working at Dris like when did you join Dris actually from uh August of uh 2017 okay so almost from

0:48the beginning uh since like 2013 but I you know it was 200 people when I joined It's a little bigger. Yeah, it was 2000 when I joined. So, well, maybe maybe we can start if we can tell if you can tell us a little bit more about what you are doing at Dicks. Uh like you mentioned, you're SVP of the

1:10product. What does that mean exactly? Like what what do you do? Yeah. So, you know, [Music] um Databricks, you know, uh we build a lot of our innovation working with our top customers. Um, and so the founders are and the, you know, Adam and I, the two heads of

1:32product and a bunch of other people are are constantly working on this strategy. We have a goal to make data and AI simple for every company to run their business on, right? To be kind of the AI operating system or database for every company in the world that or every organization in the world to unlock the data. So, uh, that's the vision and then we break into a lot of pieces. And we

1:54have a lot of strategy reviews of the top like goals and then we figure out the implementation plan. There's a 100 teams across engineering and each of them are paired with product managers and you kind of distill that into the overall product plan and you just keep turning the cranks on that thing you know. So it's uh it's uh it's a lot of fun and a lot of work and a lot of

2:16debate. So are you more in that case are you more customerf facing like because we also had we've run it different ways you know when I joined I was kind of running and building out product Ali had run product before he became CEO um and then we kind of like I was taking it from there and then Adam joined and we split the job and when we split the job I was

2:38uh I was kind of running the platform and he was running all the capabilities on top the AI machine learning etc. Um, and then we just decided we just give it all to Adam. So, so he he's running the product machine and I'm figuring out how to take it to our tens of thousands of

3:00customers, you know, with our field which is 5,000 people. So, it's think of it kind of inbound over there and outbound through me. But it's all he and I are, you know, interchangeable. To prepare for this talk, I did some research on you and I find out that you have a diploma in civil engineering. So,

3:22how how did you end up being an SVP of a large company? Yeah, it's funny. I was uh I I I I got graduate uh degrees in uh environmental engineering. Um and uh I couldn't get the job I wanted. So, I ended up doing nuclear engineering because that's what my dad did and I knew people that needed help there. But it was just like kind of like math and

3:44stuff like that. Um and uh I was doing like nuclear safety assessment stuff. And then I lived with a guy at a startup back in uh 9697 uh startup called Plum Tree. Uh and he invited me to have lunch with the founder of Plum Tree and the guy offered me a job as like their first QA engineer. So I went into QA engineering

4:06and then engineering and then product and you know it's just fun. Enterprise software is exciting. You just dig in, you learn, you uh really work with customers and you can you can help them achieve a lot. It's pretty amazing. And uh since like what's project management like at data

4:29bricks like maybe what's your day-to-day uh tasks? So it's it's uh it's really kind of being the glue between all the stakeholders and finding a way to move you know Databricks vision forward while helping our customers succeed. Um,

4:53so it always starts I mean basically all these books and stuff kind of describe a product manager as like the mini CEO of a product and that's garbage, right? People that come into a product management role at Databricks thinking they're the mini CEO and they can just decide what happens. That's garbage. You're you're more like the like you're

5:16not the rockstar, you're the roadie, you know. So the vision is the rockstar. what we're trying to achieve and then you're just like plugging in wires and stopping something from smoking over here and like trying to get people in the door sitting to listen to the music and like you're just there's so many pieces to get it done and data is a very

5:39technical company so you have to really understand how the product works really use the product a lot really understand the customers you have to find customers that know what you're they're trying to do better than you so you can learn from them. So, it's like a lot of uh helping make it happen in a way that's coherent with our long-term strategy.

6:01So, so who is the CEO at the end? Because someone still has to decide, you know, I want to put the button here and do that feature. the C [Laughter] Ali's CEO and CPO ultimately. Um, but the CEO of the product is really it it sounds kind of uh piffy or an avoidance, but it's kind of it's it's

6:24the strategy document. The strategy document is personified and anything you do, you have to test against that that decides the customers are going to ask for things. So I I think about like, okay, are you customer obsessed? Yes, we're customer obsessed with a big C, not a little C. A little C would be customer

6:47obsessed about one customer. You talk to a customer, they're going to ask for features that it's not in your interest to build. Why? Because 99 other customers would see that as friction, this specific thing they're trying to do. So you need to you need to be customer obsessed for the whole your target addressable market, the TAM. Um, and that means filling their needs,

7:10fixing what they're trying to do, achieve their goal, but subject to our strategy, which really tries to figure out a long-term durable way to So, how do you get that information? I I feel like I know you're going to say you have to be data driven and, you know, collect the feedback and and track everything, but sometime I I've seen so many, you know, data just being used to

7:33make the points you already know and already want. like it's like so easy to get like a giant bias and collect call weaponizing it. You know, Databricks used to have this uh cultural principle of be data driven, but it did exactly what you just said when everybody weaponized it because I can basically make data prove any point. I'm good with data and I'm I'm convincing. Um so

7:57people were weaponizing it and pushing their own agendas with their data. So we changed that cultural principle from let the data decide to be truth seeking. If you're really truth seeeking, you can't weaponize data because weaponizing data is trying to mislead people with data. That's not truth seeking. So um I I got away from

8:22the origin of your question there. But uh but uh I think the key is if you have a strategy document that's from first principles. So you start with the beginning of time and you lay out the reason why this is the right thing to do. You can use that as a test case against anything someone proposes in a room. Sometimes the strategy needs to be

8:44modified because the uh you know a new technology emerged or a new trend developed or a market crashed or something like that. Um so so they need to be living documents but as long as the principles remain valid, they're tested against the current market. The strategy document should remain true and

9:07should help you decide, you know, whether to do it in the A proposal or the B prop. Yes. So in that case because I feel like as a PM it's really easy to have super strong personal conviction and you know I want to do that because that's what I believe. Very confident and often wrong. Yeah. Yeah. Exactly. Yeah. Yeah. So that's what I was about to say. You have to accept that you will be wrong and you have to pivot really

9:29quickly otherwise it's going to be a mess I guess. Yeah. And it's like so I mean any good product manager uses their product all the time. So you're also a user right? So, you have opinions and you're like, I use this thing. There's so much friction here. If we did it this way, it'd be better. And you're so confident. Um, but the problem is you're one person. It's easy to overfit to yourself. So, you need to test the idea

9:53out with a bunch of pe real practitioners that have been doing it for years. Um, and then the trick is you you're a human, right? So, you're going to you're going to lead the witness without without meaning to. You're going to say, "Here's a great idea. uh is it great? And they don't want you. They're like, "Yeah, that's great." So, it's really about um you know, how how would you solve

10:18this kind of problem? Does this kind of situation ever, you know, make your day harder? Yes. How would you want to solve it? you know, and just kind of you have to tease it out because it's it's really hard to be a a good product manager because it's really hard to step away from your ego and your

10:41ideas and what you've convinced yourself will be good for the customer. I guess especially when you worked on it for for a while. Say that again. I think you just describe you just described the key the key the key skills and threats we're looking for a successful product leader like Yeah. Yeah. I mean and I think

11:04the well the there's all of the nuts and bolts of product management and then there's something in product management about having and it's I hate this term but it's like good product sense or good product taste. Someone wrote a a good uh blog post about this recently where this eusive like uh product

11:27taste is really just the result of a lot of hard work grinding and using the product and watching a lot of people use the product and seeing what works and you do that over years and years and you internalize it until it's like instinct and taste. It's not like people are born with product taste. If you're hungry for the truth and truth seeeking and you use the

11:51product every day, many hours a day, and you talk to customers and you solve the like you you you develop this taste, but that taste is really hard. And then the last part, which is the rarest, is having all of that, knowing the right thing to do, and then being able to execute, right? Because you can't just write it on paper and think it's going to happen. You know, engineers have a

12:14million things tugging at their time, right? they want to believe in it. So, you have to enroll them into the vision and so they agree that this is impactful. Um, and then, you know, there's a million other things people are emailing you constantly. If you're working in an area that people care about, you're getting so much inbound, you have to figure out, oh, this customer needs to talk. There's just a lot of time crunch, which makes

12:38it hard to execute. So, be great and execute. Yeah. Two pieces. And listening to you, I feel like it's somehow very close to being a good uh maybe pre-sales engineer in the sense of you really have to ask the good question, not making any assumption, really trying to listen what's the actual issue so that you can properly,

13:00you know, position the product. Uh but I feel like it's really like I never saw it like that, but like it's closer. Yeah. Know I think I think that I mean the elements of uh a lot of jobs are the same, right? to excel in fast-paced environment. I think the thing that's really the distinguishing aspect of

13:22product management is the long-term principled view. All of the incentives are for short-term thinking. Sales people want to close a deal. Engineer want to plan out the scrum. You know, they want the document all buttoned up for the thing they're starting to develop. So everything is pushing you towards short-term compressed thinking and you

13:45need to hold the line to think about how will this impact the customer Databricks the market four years from now and of course there's going to be very very imperfect information the further you go out. So I think that's that's what kind of distinguishes it in my and maybe you'll disagree but it seems that's a stronger dimension in product than in like pre-sales. Yeah. No, you're

14:08right. It's like it's like balancing the like because all the all the field and all the all the sellers going to ask you are going to ask you for the qu the feature which is going to let you sign the deal and then that's it where you actually have to build for the and one of the like key aspects of the product manager is like to go and discuss with customers capture them their feedback and like build a feature

14:30but how do you balance for example if you have a feedback from I don't know a large large customer like I don't know this customer is spending in hundred of millions and at the end like maybe this customer will will try to cannibalize the features and maybe make them tailored for his need. So how can you balance this this ass? I mean that's that's really the art, right? And so

14:53you've integrated all this experience into this this product sense or product taste. Um but really that's just accumulated, you know, war wounds that you've internalized. Um and so understanding like if a customer asks for you know something to be moved in the UI or some parameter to be in the

15:17API you have to understand what they're trying to solve like the deeper problem like this the kind of the five W's of you know why do you need that why anticedent to that you know why is that important what are you trying to achieve as an outcome and then understand that like is that generalizable what they're trying to do. So you know have an engineering working plan to the system.

15:40Um but then you know a great friend of yours if you're in a place like Databricks that has true scale is we have a lot of telemetry. So there's probably things the person said which you can assess what percentage of the customers do that kind of thing whether it's like you know connect to some system of data. Okay, they're connecting to system of data fu. What percentage of our

16:03customers ever try to connect to FU? So, you can take elements of it and see how how how universal the need is. Um, and but you could do that by interviewing 20 customers. You you'd be shocked. Interviewing 20 customers, you get so much information. Interviewing 10, you get so much information. interviewing five. If you have a

16:25hypothesis and you talk to five customers, you're gonna have a pretty good sense of whether or not you're crazy. So like people often think that you need so much data, but you just need to with an open mind, not you know, not leading questions, talk to five customers, you'll be a lot smarter about. So how do you go from there? So let's

16:47say you have the requirements and you have the need from the customer. And and now you I'm going to going to take for example AI because AI is moving so quickly now and I think it's really like it's a real challenge. How do you know like I feel like if you're a small startup okay you can build something it fails like you know the market is moving and in one year it's like you realize it

17:10wasn't the good thing. So it's okay know you can trash everything just pivot and go build something else. When you operate at the scale of Databricks and you have thousand of customer how do you solve that? It seems like you have to balance going too fast and then potentially, you know, making the wrong product, but at the same time, you don't want to miss an innovation. Yeah, I mean, this is this is why people

17:32kind of love the job because it's such a fascinating intellectual problem. Um, but let me let's just give an example of the DBRX model that we released, you know, a year or so ago. um when we decided to invest in this DBRX model and this is the our own open-source for a while it was like the

17:55best you know foundational model out there um it wasn't clear what models customers could use because then proprietary were well above open source and a lot of the open source ones leading ones had dubious origin in terms of their source data and whether a derivative ative commercial work would

18:18be viable from an intellectual property and legal standpoint for someone to use. So in other words, there wasn't a clear it wasn't clear that customers could build LLM derivatives on their own data and use them commercially. But we believed that the future of AI for businesses was AI on your data. But there wasn't a foundational model that

18:40was clear that they could use. They were all legally murky. So, we created DBRX and we invested a lot in it. I think we published like $10 million, whatever. Um, and for two weeks, it was the world's best model and then Llama 3 came out. Llama 3. Yes. Yeah. But then but then but then what what Zuckerberg said publicly in a lot of forums about the

19:02plan the long-term plan for the Llama series of models and because that was in the market uh we didn't need to keep doing DBRX 2345 because the customers had a solution. Okay. So like let's generalize what happened there. um for us to be investing in something because as you said there's so many demands on our time

19:24so many opportunities AI is moving very fast we have to know that it satisfies a few criteria including it solves it solves a deep customer need that there's not clearly another way another vendor to solve or that we can solve uh in a because of our other context we can solve in a uniquely

19:48a durably differentiated way. Once LMA started coming out, we didn't we, you know, we would have been one of many if we kept investing a lot in DBX. So what did we do? We took the same research team and we started investing in all kinds of techniques that were publishing all the time about, you know, different LLMs as judges, eval models, and then

20:11the new technique for fine-tuning just on the prompt response. It's has some acronym, you know, but so like the the key is if you're gonna like as a product manager, if something's going to take like a person week, yeah, you can work with engineer and make sure you do it and roll it out. If it doesn't work, you can roll it back and stuff like that. But if you're going to do anything of real significant

20:34investment, why is Databricks the best company? Why will we defensively have the best solution in the market for that need right and it has to rely on our durable strengths and then we'll invest it. So looking back at DBRx since you take that example, how do you analyze

20:56that? Uh like was it the good thing like the right thing to do it back in that in that time or do you think maybe obviously you know Databricks has resources we've you know raised historic money and all this stuff. So so um at the time I don't look at a decision and judge it based on the information. Now

21:18could we have saved $10 million? Probably. But hey, it was a great marketing event, right? It showed people that you could build industryleading foundational models on the mosaic stack. So even if with all the current information, I still would have done it, right? And um it actually a lot of customers are using DRX. It's a very good model. Um and we learned a lot

21:43every you know. So, so but the the key was we need to move the market forward in terms of commercial AI on your data and it it you know even Dolly before that fundamentally move the market forward and that's that's good for us because we want to you know help every customer be a data.

22:06I think I have it here. Yeah, that's why I think I brought it upstairs. That's a really nice one. Yeah, I I was the one who gave it to you in Paris. Oh, that's right. That's right. I've used that in some of my videos. And since you like we're talking about AI, like how do you think AI is impacting like the data platform like in

22:29the future? Because I think like um I don't know three years ago it was uh data warehousing. Before that it was ML then it was governance and now we have AI like every I around two years or three years we have a new cycle and so how it will impact the platform and how on on your end like as a Databricks

22:51company like try to balance the those investments because it's like the platform is getting bigger and bigger and like how do you your bet? So you know um there's layers to that question. So ultimat what is what do you think is our overriding goal with everything we're adding to the

23:14platform if you could what would you think it would I think as usual democratizing like access to data and AI to everyone that's a good buzz word but but basically I would say simplicity simplicity will win um the reason like unity catalog is so important is it fundamental ally

23:36simplifies the whole system. Um you know with a lakehouse you don't have to do integration with a warehouse inside the lakehouse is now vector searches feature stores online tables apps all the comput serverless. So the work to get to the business goal keeps simplifying and the irony is with all of

24:00these new efforts underway at Databricks with our you know thousands of engineers. It's really actually making it simpler. There's lots of new capabilities with lots of new names but the all of those things mean you don't have to stitch this lake lakehouse together with a bunch of other infra to do basic data and AI stuff.

24:23And the end goal being to reduce so the difficulty to get access to the data so so that you can and AI is a big part of that. I mean, I I've always been a little skeptical on the like Gen AI in the platform like talking to the data, but it's it it it's shocking how well it works now, you know? So since Unity catalog is seeing all of the traffic and

24:46learning about all the cardality of the data and how it joins and stuff when I'm writing SQL in the product now like it joins against tables that I hadn't even thought of yet in the ghost text and I'm like oh yeah that is the new version of the table I should you know it's it's it's it's spooky how good it is for the plumbing and you know people with less

25:09and less sophistication are going to be able to get the same value in a fraction of the time out of the platform. Do you see the same the same happening on the data engineering side? I feel like it's moving. I'm not even talking about Databricks overall on the data engineering. I don't know. I was expecting to see some big open source project just to magically do all the you

25:32know like some kind of the DBT of the world fully geni but like it's not really happening. What's going on here? It is funny. Um I mean I I would say that the all the code and data pipelines are getting the autocomplete stuff and everything I'm talking about. So the but it's still like what's a good example of that recently?

25:55Um so in uh like as you're you know people are you can create a a data pipeline now like visually like connecting things um but you'd kind of like you know DT is declarative delta live tables where you can say this data from source A should go to destination B with these

26:17transformations and data virtual will spin up the compute but you really want that to be like state the business problem and spit out a DLT, you know. Yeah. Yeah. You should be like, "Okay, I have my idea." Where's that? I don't know. I mean, what are you guys doing? I Well, I think the um the trick is the It's still hard like what what

26:42you're able to do with these data pipelines with like xabytes of data. It's really hard to make those work. Databricks have built a massive company on making it easier for highly skilled data engineers to make that work. And now they don't have to be as highly skilled. But it's still it's really hard

27:05to crunch that much data. So you can't kind of dumb it down into a prompt spitting out a pipeline that can move that kind of data at production scale, low TCO, running your whole business. It's too it's too um core to your business to really trust it to that yet. But it's moving fast. Yeah. Which which means now the AI in the data engineering world will just

27:28increase the productivity like helping you like I don't know autocomp completion fix some issues understanding the logs maybe debugging for you writing some some simple codes for you but then you need to still do the extra mile. Yeah. Exactly. Yeah. Well, I still think it's a little bit too boring. We need to do I don't know, maybe some crazy MCB thing and plug everything together one of these

27:51days. We'll see. We'll see. We'll see. And repeat this interview next year. We'll see where we are. All right. Right. So, look, I also wanted to touch a little bit on the I know you are talking to many uh exec and many customers. So I wanted to see you know do you have any insight on uh what are they doing currently around like

28:15data and AI? Do you think we are at the stage where um they really start to put stuff in production using AI or is it still some kind of PC's or do you really see you know things really being yeah running out to production and having some business? Yeah, you know, there's a wide variety of customers. Like all the big

28:38customers are doing what you'd call boring ML, you know, old school AI where they're doing outlier detection. You know, every time you run a credit card, there's some ML. Don't call that AI or it's AI, not Gen AI. Um but uh so all of our customers are doing you know massive atscale production work around non- Gen

29:01AI but the use cases I'm hearing even now that are actively being built are are light years ahead of what I was hearing 6 months ago like you know large manufacturers of heavy equipment working having AI helped with

29:23the design life cycle of the deep schematic drawings and stuff like that and you're like how is that possible but then you know and I was I was in a in a meeting with the CEO of one of these companies and Ali and you know Ali knows a thing or two if you haven't noticed when you talk to him and he was just talking about all the latest advances in

29:45um creating uh uh synthetic data and then having LLM judge evaluate that synthetic data and sort it into high quality synthetic data and low qual like there's so many things you can do even with deep like CAD drawings three-dimensional spatial stuff that I wouldn't have guessed we'd already be doing with some of these

30:07models but it's such a like if you can underwrite an insurance like plan in 6 hours instead of 6 weeks like the material advantage it gives you in the financial markets is astonishing. So there's like it it what but what this is doing is it's making the highly

30:29skilled plant manufacturing engineers more productive. It's not eliminating them. It it means that they don't have to do the groundwork around finding all the schematics from the previous projects and looking at the commonalities like it's making the super sharp subject matter experts 10 times more productive. It's

30:52not trying to replace them because that's absurd, you know. Yeah. And and I think it's the same on the like on the software engineering piece anyway. Um like it's really making those really good data engineer even even more productive predict like using this metadata to optimize the tables then like coming up with liquid then with this auto liquid that will

31:15automatically optimize the table the table for you then we have genie so when at Databricks they came up with with an idea like they had this vision like this long-term vision because I can still I I kept thinking like how is it even possible but was it like really the uh the objectives that you had first or those are things that came up later like

31:38how can we deal with this? That's a great question and I think it's going to be a little bit of a combination. There's a lot of uh emergent innovation that's made possible by preceding innovation that you didn't think about when you designed the preceding innovation. So, Unity Catalog was

32:00necessary for us to be able to have deeper reasoning capabilities about who had access to what and what was using what for the purpose of governance. But if you look at the principal design around all of this, it was to have these fundamental reasoning capabilities across the entire data estate.

32:22And it's a logical conclusion that these fundamental reasoning capabilities across the data estate can be incredibly useful when you apply AI to it. But that was kind of a second order effect of a good principle design. And it's think so if you think about AI augments

32:45humans the initial kind of unity catalog vision that I saw and mate probably saw around a few corners because he always does but was uh to let humans be able to reason about these things systematically by querying the system and AI just augments humans you know so it it it's kind of a

33:07derivative if someone said what will this allow us to do next and had an hourlong brainstorming. We would have talked about these things. But that wasn't in the original design. But a good design allows for uh uh allows for a lot of multiplicity of like so any good platform people will build things on it that you never would

33:31have imagined. That's kind of like the litmus test for a platform. Do people build apps on it that you're like, I had no idea you could do that with this platform? That means you have kind of good design principles in your platform. So, how do you like what's your advice for someone to have this kind of mindset? Because I feel like I'm a I'm a good PM when it comes to, you know,

33:53bitching about a button not being at the right place or, you know, like it's not working well that I can do. But I feel like going to the next level, which is really having the, you know, like the longer term vision is really hard. How how do you how do you build that? You know, it it's going to be like a super lame answer. It's just uh it's just grinding like doing it and

34:18doing it and doing it and making mistakes. Now, if you're smart, you can accelerate that learning path by actually looking back at what like in every one pager or PR or D, you have hypothesis embedded in them, right? And if you look every year at what you wrote the previous year and where you

34:42were an idiot, like you know, whenever I'm interviewing someone and we're talking about a thing they did and they launched or whatever, I'm like, what happened that surprised you? Cuz it was not in your kind of mental picture when you it wasn't in your hypothesis. what what emergent

35:04behavior conflicted with your mental model going in, you know, and if you ask yourself that question, um I think you can accelerate the learning and and figuring out kind of the durable long-term thinking. Okay, I'm going to have to do that with the demos. And do you think this is the right time

35:26maybe like let's suppose I want to become a product man like join the product team. Is it really the a good time or or AI is going to replace uh or like reduce their amount of work? Like I mean there's always going to be the people that need to grind to make it happen and make it happen the right way. You're going to use a lot more AI but um

35:48I did this podcast recently about product stuff on 20DC and like I'll say the same thing here. Do not be a product manager. Now, do not whatever you do, do not be a product manager. Now, if I say that to you and you're like, I'm going to be a product manager

36:12cuz you have to then you should like it's a thankless job in a lot of ways. the the satisfaction has to be intrinsic of making things happen, like creating the future that's unlikely. Um, and if that feels really good and you're willing to deal with, you know, all the crazy, it could be a good job for you.

36:34But don't try to become a product manager unless you have to do it. And if you have to do it, you're doing it today. If you if if today you're sending emails to people in in groups you don't work for telling them how they have to change their product because you have to cuz you're using it and you can't stand how it's

36:56behaving and you have to let them know then you might be a product manager. But if if you have a different job and you're not also doing product management because you have to you don't want to be a product manager. You just think it's a CEO of a product and you want to be a CEO, you know, that's just as hard in a different way. Much harder, I'm sure.

37:18Makes sense. Do it. That's my advice. But if you you still need to do it, do it. All right. Sounds good. Well, look, I think I think we can close it on this nice, you know, on on that amazing advice. Don't become David Mayor.

37:42Exactly. Don't be me, whatever you do. Well, thanks a lot. That was a lot of fun. Thank you so much. Thank you very much.