NextGenLakehouse on YouTube

Real-Time Data Engineering with Apache Spark Structured Streaming on Databricks

Structured Streaming is Apache Spark's streaming engine, and Databricks runs it for real-time and micro-batch pipelines. This video shows how to build streaming data engineering workloads with it. The Structured Streaming concepts page explains real-time and micro-batch processing modes, stateful processing and monitoring. It is the official reference for the streaming model used in the video. Official docs: Introduces Structured Streaming concepts, real-time and micro-batch modes, stateful processing and monitoring. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/structured-streaming/
🔗 More: https://www.nextgenlakehouse.com/answers/data-engineering

#Databricks #ApacheSpark #StructuredStreaming #Streaming #DataEngineering #RealTime #SparkSQL

— Pinned comment —
Structured Streaming powers real-time pipelines on Databricks. The concepts page and our data engineering hub are linked below. Docs: https://docs.databricks.com/aws/en/structured-streaming/ Hub: https://www.nextgenlakehouse.com/answers/data-engineering

Build Data Pipelines Without Code: Lakeflow Designer Demo

Lakeflow Designer is a no-code Databricks tool for building data pipelines on a drag-and-drop canvas. This demo shows how to build a pipeline without writing code. Lakeflow Designer prepares and explores data through a visual canvas and natural language, with workflows governed by Unity Catalog. The Lakeflow Designer page linked below describes the feature and how it fits with Lakeflow pipelines. Official docs: Introduces Lakeflow Designer, a no-code canvas for preparing and exploring data, with workflows governed by Unity Catalog.

📚 Docs: https://docs.databricks.com/aws/en/ldp/designer
🔗 More: https://www.nextgenlakehouse.com/answers/data-engineering

#Databricks #Lakeflow #LakeflowDesigner #NoCode #DataPipelines #DataEngineering #ETL

— Pinned comment —
Lakeflow Designer builds pipelines without code. The Designer docs and our data engineering hub are linked below. Docs: https://docs.databricks.com/aws/en/ldp/designer Hub: https://www.nextgenlakehouse.com/answers/data-engineering

Databricks' New Secret Weapon for Data Engineers

This video introduces a new Databricks capability for data engineers. The linked Lakeflow Spark Declarative Pipelines page is the closest official reference for building pipelines. Lakeflow Spark Declarative Pipelines lets engineers define batch and streaming pipelines declaratively, with the platform handling orchestration and state. The pipeline overview linked below is the entry point to the feature set. Official docs: Introduces Lakeflow Spark Declarative Pipelines for batch and streaming workflows. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/ldp/
🔗 More: https://www.nextgenlakehouse.com/answers/data-engineering

#Databricks #Lakeflow #DataEngineering #SparkDeclarativePipelines #ETL #DataPipelines #ApacheSpark

— Pinned comment —
A new data engineering capability is introduced here. The Lakeflow pipelines docs and our data engineering hub are linked below. Docs: https://docs.databricks.com/aws/en/ldp/ Hub: https://www.nextgenlakehouse.com/answers/data-engineering

Python Unit Testing in Databricks: Step-by-Step Guide

Databricks notebooks support unit testing of Python functions with pytest, so you can test pipeline code before it runs. This guide walks through setting it up step by step. The notebook unit testing page explains how to write and run unit tests for functions in Python, R, Scala and SQL notebooks, using pytest and related frameworks. Use it as the reference for the test setup shown in the video. Official docs: Explains unit testing functions in Python, R, Scala and SQL notebooks with pytest and related frameworks.

📚 Docs: https://docs.databricks.com/aws/en/notebooks/testing
🔗 More: https://www.nextgenlakehouse.com/answers/data-engineering

#Databricks #Python #UnitTesting #pytest #DataEngineering #DevOps #Testing

— Pinned comment —
Unit testing Python code in Databricks, step by step. The notebook testing docs and our data engineering hub are linked below. Docs: https://docs.databricks.com/aws/en/notebooks/testing Hub: https://www.nextgenlakehouse.com/answers/data-engineering

Photon Engine and Incremental Innovation: Building Engineering Teams

Photon is the Databricks query engine that speeds up SQL, DataFrame and ETL workloads without code changes. This video covers the engine and the teams that build it. The Photon page explains how the engine accelerates SQL, DataFrame, ETL and stateless streaming workloads, where it is enabled, and its limitations. The page is the exact reference for the engine topic. Official docs: Explains the Photon engine, which speeds up SQL, DataFrame, ETL and stateless streaming workloads without code changes.

📚 Docs: https://docs.databricks.com/aws/en/compute/photon
🔗 More: https://www.nextgenlakehouse.com/answers/data-engineering

#Databricks #Photon #QueryEngine #Performance #DataEngineering #ApacheSpark #Engineering

— Pinned comment —
Photon speeds up Databricks workloads without code changes. The Photon docs and our data engineering hub are linked below. Docs: https://docs.databricks.com/aws/en/compute/photon Hub: https://www.nextgenlakehouse.com/answers/data-engineering

Getting Started with the Open Source Synthetic Data SDK

The open source synthetic data SDK generates realistic test data for demos and pipelines. This video shows how to get started with it; the Databricks notebooks overview is the closest official reference. Notebooks are where most Databricks data generation code runs, and the notebooks overview linked below covers editing, running and sharing them. Check the SDK project's own documentation for generator-specific options. Official docs: Overview of Databricks notebooks for data science and machine learning workflows. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/notebooks/
🔗 More: https://www.nextgenlakehouse.com/answers/data-engineering

#Databricks #SyntheticData #OpenSource #Python #DataEngineering #TestData #Notebooks

— Pinned comment —
Getting started with the synthetic data SDK. The notebooks docs and our data engineering hub are linked below. Docs: https://docs.databricks.com/aws/en/notebooks/ Hub: https://www.nextgenlakehouse.com/answers/data-engineering

Lakeflow Demo from the Data+AI Summit 2025

This demo from the Data and AI Summit 2025 shows Lakeflow building data ingestion and transformation pipelines. The Lakeflow pipelines overview is the entry point for the feature. Lakeflow Spark Declarative Pipelines lets engineers define batch and streaming pipelines declaratively, with orchestration handled by the platform. The pipeline developer reference linked below covers the Python and SQL APIs. Official docs: Introduces Lakeflow Spark Declarative Pipelines for batch and streaming workflows. Index of resources for building Lakeflow pipelines in Python and SQL, with local development options.

📚 Docs: https://docs.databricks.com/aws/en/ldp/ https://docs.databricks.com/aws/en/ldp/developer/
🔗 More: https://www.nextgenlakehouse.com/answers/data-engineering

#Databricks #Lakeflow #DataEngineering #DAIS2025 #ETL #DataPipelines #ApacheSpark

— Pinned comment —
The Lakeflow demo from DAIS 2025. The Lakeflow pipelines docs and our data engineering hub are linked below. Docs: https://docs.databricks.com/aws/en/ldp/ Hub: https://www.nextgenlakehouse.com/answers/data-engineering

Lakeflow Designer Demo from the Data+AI Summit 2025

This demo shows Lakeflow Designer, the no-code drag-and-drop tool for building data pipelines. It was recorded at the Data and AI Summit 2025. Lakeflow Designer prepares and explores data through a visual canvas and natural language, with workflows governed by Unity Catalog. The Lakeflow Designer page linked below describes the feature. Official docs: Introduces Lakeflow Designer, a no-code canvas for preparing and exploring data, with workflows governed by Unity Catalog. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/ldp/designer
🔗 More: https://www.nextgenlakehouse.com/answers/data-engineering

#Databricks #LakeflowDesigner #NoCode #DAIS2025 #DataPipelines #DataEngineering #Lakeflow

— Pinned comment —
Lakeflow Designer demo from DAIS 2025. The Designer docs and our data engineering hub are linked below. Docs: https://docs.databricks.com/aws/en/ldp/designer Hub: https://www.nextgenlakehouse.com/answers/data-engineering

Spark Declarative Pipelines Is Now Open Source

Spark Declarative Pipelines, the declarative framework behind Lakeflow pipelines, is now open source in Apache Spark. The Spark programming guide linked below covers the concepts and the spark-pipelines CLI. Spark Declarative Pipelines lets you build batch and streaming pipelines declaratively in Python or SQL, with the spark-pipelines CLI. The Spark guide is the exact reference for the open source release shown in the video. Official docs: Spark guide to building batch and streaming pipelines declaratively, with the spark-pipelines CLI. Introduces Lakeflow Spark Declarative Pipelines for batch and streaming workflows.

📚 Docs: https://spark.apache.org/docs/latest/declarative-pipelines-programming-guide.html https://docs.databricks.com/aws/en/ldp/
🔗 More: https://www.nextgenlakehouse.com/answers/data-engineering

#ApacheSpark #OpenSource #SparkDeclarativePipelines #Lakeflow #DataEngineering #Databricks #Streaming

— Pinned comment —
Spark Declarative Pipelines is now open source. The Spark programming guide and our data engineering hub are linked below. Docs: https://spark.apache.org/docs/latest/declarative-pipelines-programming-guide.html Hub: https://www.nextgenlakehouse.com/answers/data-engineering

Introduction to LakeFusion's MDM

LakeFusion is a master data management (MDM) product built on Databricks. This introduction explains the MDM approach; no LakeFusion-specific Databricks doc was verified, so the link points to the Unity Catalog overview. Master data management consolidates customer and product records into golden records, with governance on the lakehouse. The Unity Catalog overview linked below covers the governance layer that MDM data relies on. Official docs: Introduces Unity Catalog, the governance layer covering access control, lineage and auditing. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/data-governance/unity-catalog/
🔗 More: https://www.nextgenlakehouse.com/answers/data-engineering

#Databricks #MDM #MasterDataManagement #LakeFusion #DataGovernance #DataEngineering #DataQuality

— Pinned comment —
An introduction to MDM with LakeFusion. The Unity Catalog docs and our data engineering hub are linked below. Docs: https://docs.databricks.com/aws/en/data-governance/unity-catalog/ Hub: https://www.nextgenlakehouse.com/answers/data-engineering

Lakeflow Spark Declarative Pipelines (Delta Live Tables) Just Got a Major Update

Lakeflow Spark Declarative Pipelines, formerly Delta Live Tables, received a major update. This video covers what changed for building declarative pipelines. Spark Declarative Pipelines lets you define batch and streaming pipelines declaratively in Python or SQL, with orchestration and data quality handled by the platform. The pipeline overview linked below is the entry point to the updated feature set. Official docs: Introduces Lakeflow Spark Declarative Pipelines for batch and streaming workflows. Index of resources for building Lakeflow pipelines in Python and SQL, with local development options.

📚 Docs: https://docs.databricks.com/aws/en/ldp/ https://docs.databricks.com/aws/en/ldp/developer/
🔗 More: https://www.nextgenlakehouse.com/answers/data-engineering

#Databricks #Lakeflow #SparkDeclarativePipelines #DeltaLiveTables #DataEngineering #ETL #Streaming

— Pinned comment —
The major Lakeflow Spark Declarative Pipelines update is covered here. The pipeline docs and our data engineering hub are linked below. Docs: https://docs.databricks.com/aws/en/ldp/ Hub: https://www.nextgenlakehouse.com/answers/data-engineering

Data Orchestration with Databricks Workflows

Databricks Workflows, now Lakeflow Jobs, orchestrates multi-task data pipelines with schedules, triggers and monitoring. This video shows how to orchestrate a pipeline with it. The Lakeflow Jobs page covers scheduling and orchestrating multi-task workflows, including jobs, tasks, triggers, monitoring and limits. It is the exact reference for the orchestration features in the video. Official docs: Introduces Lakeflow Jobs for scheduling and orchestrating multi-task workflows, with triggers and monitoring. Watch the full walkthrough on the NextGenLakehouse channel. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/jobs/
🔗 More: https://www.nextgenlakehouse.com/answers/data-engineering

#Databricks #Workflows #Lakeflow #Orchestration #DataEngineering #ETL #DataPipelines

— Pinned comment —
Orchestrating pipelines with Databricks Workflows is the topic. The Lakeflow Jobs docs and our data engineering hub are linked below. Docs: https://docs.databricks.com/aws/en/jobs/ Hub: https://www.nextgenlakehouse.com/answers/data-engineering

Getting Started with DQX: Data Quality Checks for Spark Pipelines on Databricks

DQX is the Databricks Labs data quality framework for checking Spark DataFrames against rules before bad records reach downstream tables. This video shows how to get started with it. Compare it with Spark Declarative Pipelines (formerly Delta Live Tables) expectations, which define quality rules inside a pipeline definition. Use the DQX project docs for its Python API. DQX runs checks on the DataFrame you already have, so you can add it to an existing notebook or job without redesigning the pipeline. Start with a few high-value rules such as not-null and range checks, then expand coverage. 📚 Docs: https://databrickslabs.github.io/dqx/ and https://docs.databricks.com/aws/en/ldp/expectations. 🔗 More: https://www.nextgenlakehouse.com/learn (Data Engineering hub). #Databricks #DataQuality #Spark #DQX #DataEngineering #ETL #Lakehouse #DataAnalytics

— Pinned comment —
Official reference: https://databrickslabs.github.io/dqx/ More Data Engineering content lives at https://www.nextgenlakehouse.com/learn

Laktory: Open-Source DataOps Framework for Databricks Pipelines and Infrastructure

Laktory is an open-source DataOps framework that lets you define Databricks pipelines and infrastructure as code. This video explains what it does and why teams use it. Databricks deploys projects with Declarative Automation Bundles (formerly Databricks Asset Bundles), which Laktory can complement. Check the Laktory site for its configuration model. Laktory fits teams that want pipeline and infrastructure definitions reviewed and versioned like application code. Compare its approach with bundles before choosing one for a project. 📚 Docs: https://www.laktory.ai/ and https://docs.databricks.com/aws/en/dev-tools/bundles/. 🔗 More: https://www.nextgenlakehouse.com/learn (Data Engineering hub). #Databricks #DataOps #IaC #OpenSource #DataEngineering #ETL #Lakehouse #DataAnalytics

— Pinned comment —
Official reference: https://www.laktory.ai/ More Data Engineering content lives at https://www.nextgenlakehouse.com/learn

Simplifying ETL with Rivery and Databricks: Ingestion to Lakehouse

Rivery and Databricks can simplify ETL by ingesting data from SaaS and database sources into the lakehouse with low-code pipelines. This video shows how the two fit together. The Rivery partner page covers how to connect Rivery to Databricks for loading data. Low-code tools like Rivery suit teams where source connectors matter more than custom code, with the lakehouse handling transformation afterwards. Check the partner doc for supported destinations and authentication. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://docs.databricks.com/aws/en/partners/ingestion/rivery. 🔗 More: https://www.nextgenlakehouse.com/learn (Data Engineering hub). #Databricks #ETL #Rivery #DataIngestion #DataEngineering #Lakehouse #DataAnalytics #AI

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/partners/ingestion/rivery More Data Engineering content lives at https://www.nextgenlakehouse.com/learn

The Data Engineering Guide You Need: Bilal Aslam on Modern Pipelines

This conversation with Bilal Aslam covers practical data engineering practices for modern pipelines. It is an overview rather than a single-feature tutorial. The Databricks data engineering docs index links to pipelines, jobs, ingestion and Delta Lake guides for building production workflows. The conversation is broad, so use it as orientation and then follow the linked docs for specific pipeline, job and ingestion tasks. It suits engineers choosing where to start with modern pipelines. 📚 Docs: https://docs.databricks.com/aws/en/data-engineering/. 🔗 More: https://www.nextgenlakehouse.com/learn (Data Engineering hub). #Databricks #ETL #Pipelines #DataEngineering #Lakehouse #DataAnalytics #AI #DataGovernance

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/data-engineering/ More Data Engineering content lives at https://www.nextgenlakehouse.com/learn

Future of Data Engineering 2025 and Beyond with Michael Armbrust

Michael Armbrust discusses where data engineering is heading in 2025 and beyond, including declarative pipelines and streaming. This is a forward-looking conversation. Spark Declarative Pipelines (formerly Delta Live Tables) is the declarative framework for batch and streaming pipelines on Databricks, and its docs show the current model. Forward-looking talks help plan skills and architecture, while the docs describe what is available today. Check the pipeline docs before committing to a feature. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://docs.databricks.com/aws/en/ldp/. 🔗 More: https://www.nextgenlakehouse.com/learn (Data Engineering hub). #Databricks #Spark #Streaming #FutureOfData #DataEngineering #ETL #Lakehouse #DataAnalytics

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/ldp/ More Data Engineering content lives at https://www.nextgenlakehouse.com/learn

Data Engineering Pro Tips from a Tech Lead on Databricks

This video shares practical data engineering tips from a tech lead working on Databricks pipelines. It covers practices for building reliable, maintainable workflows. The Databricks data engineering docs index links to the pipeline, job and ingestion guides referenced in these tips. Tips from a tech lead are most valuable on a real pipeline, so pick one job to refactor first and measure the result. Tie each tip to run time, failures or cost so you can tell whether it helped. 📚 Docs: https://docs.databricks.com/aws/en/data-engineering/. 🔗 More: https://www.nextgenlakehouse.com/learn (Data Engineering hub). #Databricks #Pipelines #BestPractices #DataEngineering #ETL #Lakehouse #DataAnalytics #AI

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/data-engineering/ More Data Engineering content lives at https://www.nextgenlakehouse.com/learn

Streamline Your Migration to Databricks with Alchemist: Walkthrough

Alchemist helps streamline migrating existing data workloads to Databricks by converting and validating code. This video walks through that migration process. The Databricks migration guide covers planning and executing a move from legacy data platforms to the lakehouse. Migrations go best when you inventory workloads first, convert code, then validate outputs side by side before cutting over. Check the migration guide for the phases that apply to your source platform. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://docs.databricks.com/aws/en/migration/. 🔗 More: https://www.nextgenlakehouse.com/learn (Data Engineering hub). #Databricks #Migration #Alchemist #CodeConversion #DataEngineering #ETL #Lakehouse #DataAnalytics

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/migration/ More Data Engineering content lives at https://www.nextgenlakehouse.com/learn

Getting Started with Databricks Connect and Serverless Compute in Python

Databricks Connect lets you run Spark code from your local IDE on Databricks serverless compute. This video shows how to set it up and run your first session. The Databricks Connect for Python doc covers installation, authentication and selecting serverless compute from your local environment. Serverless Connect sessions let you iterate from your IDE without managing a cluster. Check the Python doc for supported Python and Databricks Connect versions before installing. 📚 Docs: https://docs.databricks.com/aws/en/dev-tools/databricks-connect/python/ and https://docs.databricks.com/aws/en/compute/serverless/. 🔗 More: https://www.nextgenlakehouse.com/learn (Data Engineering hub). #Databricks #DatabricksConnect #Serverless #Python #DataEngineering #ETL #Lakehouse #DataAnalytics

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/dev-tools/databricks-connect/python/ More Data Engineering content lives at https://www.nextgenlakehouse.com/learn

Getting Started with Amperity: Data Quality for AI and Customer Engagement

Amperity is a customer data platform partner that prepares customer records for AI and engagement, and this video covers getting started with it on Databricks. Data quality is the core theme. The Databricks partner directory lists Amperity and other integrations, and the data profiling docs cover quality checks on Unity Catalog tables. Customer data quality issues often come from duplicate and inconsistent identifiers, so check match rates before using the data for AI or outreach. 📚 Docs: https://www.databricks.com/partners and https://docs.databricks.com/aws/en/data-governance/unity-catalog/data-quality-monitoring/data-profiling. 🔗 More: https://www.nextgenlakehouse.com/learn (Data Engineering hub). #Databricks #Amperity #DataQuality #CDP #DataEngineering #ETL #Lakehouse #DataAnalytics

— Pinned comment —
Official reference: https://www.databricks.com/partners More Data Engineering content lives at https://www.nextgenlakehouse.com/learn

From Legacy to Lakehouse: Essential Tips for Migrating to Databricks

Migrating from legacy data platforms to Databricks is smoother when you inventory workloads, convert code and validate results in stages. This video shares essential tips for that path. The Databricks migration guide covers the phases of moving data applications to the lakehouse, and the Hive metastore upgrade doc covers table migration to Unity Catalog. Start with a small, well-understood workload to build the team's migration process before moving critical pipelines. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://docs.databricks.com/aws/en/migration/ and https://docs.databricks.com/aws/en/data-governance/unity-catalog/migrate. 🔗 More: https://www.nextgenlakehouse.com/learn (Data Engineering hub). #Databricks #Migration #Lakehouse #LegacyModernization #DataEngineering #ETL #DataAnalytics #AI

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/migration/ More Data Engineering content lives at https://www.nextgenlakehouse.com/learn

The Next Generation of Databricks Notebooks: Features and Setup

The next generation of Databricks notebooks adds a new editor, AI assistance and interactive debugging for Python and SQL. This video shows the main features. The notebooks docs cover the editor, cells, and how notebooks run on serverless compute. The new editor changes how you write and debug cells, so try it on a small notebook before moving production code. Check the notebooks doc for features available on your workspace and compute type. 📚 Docs: https://docs.databricks.com/aws/en/notebooks/ and https://docs.databricks.com/aws/en/notebooks/notebooks-code. 🔗 More: https://www.nextgenlakehouse.com/learn (Data Engineering hub). #Databricks #Notebooks #Python #SQL #DataEngineering #ETL #Lakehouse #DataAnalytics

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/notebooks/ More Data Engineering content lives at https://www.nextgenlakehouse.com/learn

Fivetran and Databricks Beginner's Guide: Ingest Data into the Lakehouse

Fivetran can load data from many SaaS and database sources into Databricks, which then stores it in the lakehouse. This beginner's guide covers the connection. The Fivetran partner doc covers connecting Fivetran to Databricks and the destination setup. Fivetran handles connector maintenance so you can focus on modelling in Databricks, which suits teams new to the lakehouse. Check the partner doc for connection settings and the destination schema before the first sync. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://docs.databricks.com/aws/en/partners/ingestion/fivetran. 🔗 More: https://www.nextgenlakehouse.com/learn (Data Engineering hub). #Databricks #Fivetran #DataIngestion #ETL #DataEngineering #Lakehouse #DataAnalytics #AI

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/partners/ingestion/fivetran More Data Engineering content lives at https://www.nextgenlakehouse.com/learn

Orchestrate Your Data with Databricks and Orchestra: Partner Integration

Orchestra is a data orchestration tool that can coordinate Databricks workloads alongside other data systems. This video shows how the two work together. Databricks Jobs schedules and orchestrates tasks, and the Jobs docs cover triggers, dependencies and retries. Orchestra is useful when pipelines span tools beyond Databricks, so one control plane can manage dependencies across them. Keep Databricks job definitions in source control so the orchestrator and workspace stay consistent. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://docs.databricks.com/aws/en/jobs/. 🔗 More: https://www.nextgenlakehouse.com/learn (Data Engineering hub). #Databricks #Orchestration #Orchestra #Jobs #DataEngineering #ETL #Lakehouse #DataAnalytics

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/jobs/ More Data Engineering content lives at https://www.nextgenlakehouse.com/learn

Boost Apache Spark Performance on Databricks: Optimization Techniques

Apache Spark performance on Databricks improves with better partitioning, caching, adaptive query execution and avoiding skew. This video covers the main optimization techniques. The Databricks optimization guide covers adaptive query execution, data skipping and file layout choices that affect Spark jobs. Start with the Spark UI to find the slow stage before changing code, so fixes target the real bottleneck. Check the optimization guide for adaptive query execution settings and their defaults. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://docs.databricks.com/aws/en/optimizations/. 🔗 More: https://www.nextgenlakehouse.com/learn (Data Engineering hub). #Databricks #ApacheSpark #Performance #Optimization #DataEngineering #ETL #Lakehouse #DataAnalytics

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/optimizations/ More Data Engineering content lives at https://www.nextgenlakehouse.com/learn

Rivery and Databricks: Beginner's Guide to Modern Data Engineering

Rivery and Databricks together give you low-code ingestion and lakehouse processing for modern data engineering. This beginner's guide covers the setup. The Rivery partner doc covers connecting the Rivery pipeline to a Databricks destination. Rivery suits teams that prefer visual pipeline building, while Databricks handles transformations and storage. Compare source coverage with your own list before committing, and keep credentials in a secret scope. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://docs.databricks.com/aws/en/partners/ingestion/rivery. 🔗 More: https://www.nextgenlakehouse.com/learn (Data Engineering hub). #Databricks #Rivery #ETL #Lakehouse #DataEngineering #DataAnalytics #AI #DataGovernance

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/partners/ingestion/rivery More Data Engineering content lives at https://www.nextgenlakehouse.com/learn

Anomalo and Databricks: The Future of Data Quality Monitoring

Anomalo detects data quality issues automatically by monitoring tables for unexpected changes. This video explains why that matters and how it fits with Databricks. The Anomalo partner doc covers connecting Anomalo to Databricks Unity Catalog tables for monitoring. Automated monitoring finds unexpected changes that fixed rules miss, so it complements quality checks rather than replacing them. Connect it to the tables that feed dashboards and models first. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://docs.databricks.com/aws/en/partners/data-governance/anomalo. 🔗 More: https://www.nextgenlakehouse.com/learn (Data Engineering hub). #Databricks #Anomalo #DataQuality #DataObservability #DataEngineering #ETL #Lakehouse #DataAnalytics

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/partners/data-governance/anomalo More Data Engineering content lives at https://www.nextgenlakehouse.com/learn

Getting Started with Databricks Lakehouse Monitoring and Data Profiling

Databricks Lakehouse Monitoring tracks table quality and drift over time, and its docs now sit under data profiling. This video shows how to get started. The data profiling doc covers creating a monitor on a table and reading the generated profile and drift metrics. Profiles and drift metrics are most useful when reviewed on a schedule with the data owner. Create the monitor on a key table first, then extend to others, and check the profiling doc for supported metrics. 📚 Docs: https://docs.databricks.com/aws/en/data-governance/unity-catalog/data-quality-monitoring/data-profiling. 🔗 More: https://www.nextgenlakehouse.com/learn (Data Engineering hub). #Databricks #LakehouseMonitoring #DataQuality #DataProfiling #DataEngineering #ETL #Lakehouse #DataAnalytics

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/data-governance/unity-catalog/data-quality-monitoring/data-profiling More Data Engineering content lives at https://www.nextgenlakehouse.com/learn

Run a Databricks Workflows Job as a Service Principal: Step-by-Step

Running a Databricks job as a service principal means the job runs under an identity that is not tied to a person. This video shows how to set it up in Workflows. The service principals doc explains creating and granting permissions to service principals, and the Jobs doc covers jobs and their run identity. Service principals keep scheduled jobs running if an employee leaves the team. Grant the service principal only the permissions the job needs. 📚 Docs: https://docs.databricks.com/aws/en/admin/users-groups/service-principals and https://docs.databricks.com/aws/en/jobs/. 🔗 More: https://www.nextgenlakehouse.com/learn (Data Engineering hub). #Databricks #ServicePrincipal #Workflows #Jobs #DataEngineering #ETL #Lakehouse #DataAnalytics

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/admin/users-groups/service-principals More Data Engineering content lives at https://www.nextgenlakehouse.com/learn

The Secret Weapon for Databricks Deployments: Bundles and CI/CD

This video covers a deployment approach for Databricks projects. The title does not name the tool, so this summary stays general, and the bundle and CI/CD docs are the closest references. Declarative Automation Bundles (formerly Databricks Asset Bundles) package jobs and pipelines for deployment across environments. Use the CI/CD doc for pipeline setup across environments. Confirm the tool and steps against the bundle docs before following the video. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://docs.databricks.com/aws/en/dev-tools/bundles/ and https://docs.databricks.com/aws/en/dev-tools/ci-cd/. 🔗 More: https://www.nextgenlakehouse.com/learn (Data Engineering hub). #Databricks #DeployDatabricks #CICD #DevOps #DataEngineering #ETL #Lakehouse #DataAnalytics

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/dev-tools/bundles/ More Data Engineering content lives at https://www.nextgenlakehouse.com/learn

Getting Started with Unity Catalog and Spark Declarative Pipelines (DLT)

Spark Declarative Pipelines, formerly Delta Live Tables, can write governed tables into Unity Catalog. This video shows how to get started with that setup. The Declarative Pipelines doc describes defining datasets and flows and publishing them to a catalog and schema. Declarative pipelines publish tables to Unity Catalog, so access control is set in one place. Check the pipeline doc for the catalog and schema settings, and test updates in a development environment first. 📚 Docs: https://docs.databricks.com/aws/en/ldp/. 🔗 More: https://www.nextgenlakehouse.com/learn (Data Engineering hub). #Databricks #DeltaLiveTables #SparkDeclarativePipelines #UnityCatalog #DataEngineering #ETL #Lakehouse #DataAnalytics

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/ldp/ More Data Engineering content lives at https://www.nextgenlakehouse.com/learn

Getting Started with Databricks Connect V2 for Python

Databricks Connect V2 lets you run Spark code from your IDE against Databricks compute. This video shows how to get started with it. The Databricks Connect for Python doc covers installing the package, configuring authentication and running code remotely. Connect V2 lets you run Spark code from your IDE, which helps with debugging against real data. Check the Python doc for install steps, and use the same catalog and compute settings as your jobs. 📚 Docs: https://docs.databricks.com/aws/en/dev-tools/databricks-connect/python/. 🔗 More: https://www.nextgenlakehouse.com/learn (Data Engineering hub). #Databricks #DatabricksConnect #Python #DevTools #DataEngineering #ETL #Lakehouse #DataAnalytics

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/dev-tools/databricks-connect/python/ More Data Engineering content lives at https://www.nextgenlakehouse.com/learn

What Is Vibe Data Modeling on Databricks? Agentic Silver-Layer Models Explained

What is Vibe Data Modeling on Databricks? This short video introduces an agentic approach to data modeling in which you describe a business model in plain language and an agent drafts Silver-layer tables, checks them and prepares them for deployment. The official Databricks write-up describes the notebook-based reference implementation, its test harness and quality gates. Genie Code is the separate built-in agent for coding and pipeline work, so the two are related but not the same tool. The notebook reference models and quality gates matter most when you adapt the approach to your own business domain and naming standards. 📚 Docs: https://www.databricks.com/blog/reimagining-data-modeling-lakehouse-introducing-vibe-data-modeling https://docs.databricks.com/aws/en/genie-code 🔗 More: https://www.nextgenlakehouse.com/content#lakeflow #Databricks #DataModeling #AgenticAI #DataEngineering #UnityCatalog #GenieCode

— Pinned comment —
Official reference for this topic: https://www.databricks.com/blog/reimagining-data-modeling-lakehouse-introducing-vibe-data-modeling More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content#lakeflow

Autonomous AI: Less Maintaining, More Building for Data Teams

Can autonomous AI take maintenance off data teams? This video argues that agents should handle routine operational work such as investigating failing jobs and preparing fixes, so engineers can focus on building. Databricks describes this model with Genie ZeroOps, an agent that detects problems, traces them with Unity Catalog lineage and prepares validated fixes for human approval. Nothing is applied to production without approval. Use this framing to decide which operational tasks you would hand to an agent and which still need human review. 📚 Docs: https://www.databricks.com/blog/introducing-genie-zeroops 🔗 More: https://www.nextgenlakehouse.com/content#lakeflow #Databricks #AutonomousAI #GenieZeroOps #DataEngineering #AIAgents #DataOps

— Pinned comment —
Official reference for this topic: https://www.databricks.com/blog/introducing-genie-zeroops More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content#lakeflow

Hiring Junior Data Engineers at Databricks: What Candidates Should Know

What does Databricks look for in junior data engineers? This video discusses hiring junior data engineers, the skills and mindset that help candidates stand out, and how to prepare for the process. Current openings and requirements are listed on the official careers page, so check there before you apply. Build a small data pipeline project to show your practical skills, since hands-on work often matters more than the degree. Junior candidates often stand out by showing how they would design a small pipeline with clear data quality checks and documentation. 📚 Docs: https://www.databricks.com/company/careers 🔗 More: https://www.nextgenlakehouse.com/content#lakeflow #Databricks #DataEngineering #Hiring #Careers #JuniorDataEngineer #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/company/careers More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content#lakeflow

Joe Reis on Keys to Succeed in Data Engineering

What are the keys to succeed in data engineering? Joe Reis, author and data engineering practitioner, shares the habits and skills that help engineers grow, from fundamentals to communicating with stakeholders. The Databricks data engineering documentation gives the technical side of building pipelines on the platform. Pair the advice with the Databricks data engineering docs to turn the principles into concrete pipeline work. Many data engineers find that the basics of modelling, testing and communication matter more than any single tool, so use the advice to set priorities. 📚 Docs: https://docs.databricks.com/aws/en/data-engineering/ 🔗 More: https://www.nextgenlakehouse.com/content#lakeflow #Databricks #DataEngineering #JoeReis #CareerAdvice #DataCareers #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://docs.databricks.com/aws/en/data-engineering/ More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content#lakeflow

Soda at Data+AI Summit 2025: Data Quality Partner Talk

What did Soda present at Data+AI Summit 2025? Soda is a data quality and observability tool, and this partner talk shows how teams monitor and test data quality alongside Databricks pipelines. Use the Databricks data engineering documentation to see how pipelines and tables are built before adding checks. Add data quality checks after the pipeline is built so failures surface early instead of reaching dashboards. Quality checks work best when they run on a schedule with clear owners, so decide who acts on each alert before you add a new rule. 📚 Docs: https://docs.databricks.com/aws/en/data-engineering/ 🔗 More: https://www.nextgenlakehouse.com/content#lakeflow #Databricks #Soda #DataQuality #DataObservability #DataAISummit #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://docs.databricks.com/aws/en/data-engineering/ More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content#lakeflow

Lakeflow Designer Announced at Data+AI Summit: Visual Pipeline Building

What is Lakeflow Designer? This Data+AI Summit announcement introduces a visual, no-code canvas where you build data preparation workflows from operators, preview intermediate results and generate code. Natural-language tools help you describe steps. The documentation covers the canvas, built-in operators, previews, and how to deploy workflows to jobs with Git and Declarative Automation Bundles. Start with a small workflow, preview each step, and deploy it through Git and bundles once it works. Visual pipeline tools suit analysts and engineers who share the same data, so decide early which steps you want to keep as code and which to build visually. 📚 Docs: https://docs.databricks.com/aws/en/designer/what-is-lakeflow-designer https://docs.databricks.com/aws/en/designer 🔗 More: https://www.nextgenlakehouse.com/content#lakeflow #Databricks #LakeflowDesigner #Lakeflow #NoCode #DataEngineering #DataAISummit

— Pinned comment —
Official reference for this topic: https://docs.databricks.com/aws/en/designer/what-is-lakeflow-designer More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content#lakeflow

Parameterize Pipelines in Rivery: Job Parameters for Reusable ETL

How do you parameterize pipelines in Rivery? Rivery is a third-party ETL tool, and this video shows how to reuse one pipeline across environments and sources by passing parameters. Databricks has its own parameter mechanism for jobs, so the job parameters documentation is the closest official reference for doing the same in Databricks. Use job parameters to run the same pipeline across dev and prod without copying code. Parameters let one pipeline definition serve several sources or environments, which cuts maintenance and makes reviews easier. 📚 Docs: https://docs.databricks.com/aws/en/jobs/job-parameters 🔗 More: https://www.nextgenlakehouse.com/content#lakeflow #Databricks #ETL #DataPipelines #Rivery #JobParameters #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://docs.databricks.com/aws/en/jobs/job-parameters More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content#lakeflow

Is Lakeflow the Future of Data Engineering?

Is Lakeflow the future of data engineering? Lakeflow combines managed ingestion through Lakeflow Connect, declarative pipelines and jobs for orchestration on one platform. This video assesses that proposition. The Lakeflow Connect and Declarative Pipelines documentation shows how the pieces fit together and where each one applies. Map your current ingestion, transformation and orchestration tools to Lakeflow components to see what would change. Teams that already run several tools often gain the most from fewer moving parts, but migrate one pipeline first to measure the effect. 📚 Docs: https://docs.databricks.com/aws/en/ingestion/lakeflow-connect/ https://docs.databricks.com/aws/en/ldp/ 🔗 More: https://www.nextgenlakehouse.com/content#lakeflow #Databricks #Lakeflow #DataEngineering #LakeflowConnect #DeclarativePipelines #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://docs.databricks.com/aws/en/ingestion/lakeflow-connect/ More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content#lakeflow

What Is Databricks Lakeflow? Ingestion, Pipelines and Jobs Explained

What is Databricks Lakeflow? This video introduces Lakeflow, the Databricks data engineering offering. It covers Lakeflow Connect for managed ingestion from SaaS and databases, Spark Declarative Pipelines for transformations, and Lakeflow Jobs for orchestration. The data engineering documentation is the starting point for the full set of capabilities. Start with one managed connector and one declarative pipeline to see how the pieces fit in your workspace. Lakeflow is useful when you want ingestion, transformation and scheduling governed in one place, with the same permissions model across them. 📚 Docs: https://docs.databricks.com/aws/en/data-engineering/ https://docs.databricks.com/aws/en/ingestion/lakeflow-connect/ 🔗 More: https://www.nextgenlakehouse.com/content#lakeflow #Databricks #Lakeflow #DataEngineering #ETL #Orchestration #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://docs.databricks.com/aws/en/data-engineering/ More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content#lakeflow

Hugo Lu from Orchestra on Data Orchestration

What does Hugo Lu from Orchestra say about data orchestration? Orchestra is a third-party orchestration platform, and this interview covers how teams coordinate data pipelines and dependencies. For how Databricks schedules and orchestrates its own jobs, the Lakeflow Jobs documentation is the official reference. Compare orchestration options on your own pipelines, since dependencies and retries differ between tools. Orchestration choices affect retries, alerting and cost, so compare how each tool handles failures before you commit to one. Check how each tool handles retries and alerting on a real pipeline before you commit to an orchestrator. 📚 Docs: https://docs.databricks.com/aws/en/jobs/ 🔗 More: https://www.nextgenlakehouse.com/content#lakeflow #Databricks #Orchestration #DataEngineering #Orchestra #DataPipelines #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://docs.databricks.com/aws/en/jobs/ More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content#lakeflow

Large-Scale ETL and Data Transformation on Databricks

How do you run large-scale ETL on Databricks? This video covers transforming large datasets with declarative pipelines and Spark, so teams define the transformations and let the platform manage execution. The Spark Declarative Pipelines documentation covers the concepts, and the data engineering overview shows the wider toolset. Start with one batch pipeline, measure its cost and runtime, then apply the same pattern to other large jobs. Large transformations benefit from clear dependencies and incremental processing, so check which steps can run on a schedule and which need full refreshes. 📚 Docs: https://docs.databricks.com/aws/en/ldp/ https://docs.databricks.com/aws/en/data-engineering/ 🔗 More: https://www.nextgenlakehouse.com/content#lakeflow #Databricks #ETL #DataEngineering #ApacheSpark #DataPipelines #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://docs.databricks.com/aws/en/ldp/ More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content#lakeflow

Catalog Commits with Unity Catalog: Multi-Table ACID Transactions

Catalog commits let Unity Catalog coordinate commits across tables, giving multi-table ACID transactions on managed Delta Lake and Iceberg tables. This video explains how it works. The catalog commits page covers how Unity Catalog coordinates commits for managed Delta Lake and Apache Iceberg tables, the runtime requirements, and how to turn the feature on or off. Limitations are listed on the same page. Official docs: Explains how Unity Catalog coordinates commits for managed Delta Lake and Iceberg tables, with runtime requirements.

📚 Docs: https://docs.databricks.com/aws/en/delta/catalog-commits
🔗 More: https://www.nextgenlakehouse.com/answers/delta-lake

#Databricks #UnityCatalog #ACID #DeltaLake #Transactions #Iceberg #DataEngineering

— Pinned comment —
Catalog commits give multi-table ACID transactions in Unity Catalog. The docs and our Delta Lake hub are linked below. Docs: https://docs.databricks.com/aws/en/delta/catalog-commits Hub: https://www.nextgenlakehouse.com/answers/delta-lake

Atomic Transactions in Databricks: Multi-Table Commits with Catalog Commits

Atomic multi-table commits on Databricks come from Unity Catalog catalog commits for managed Delta Lake and Iceberg tables. This video shows how transactions span multiple tables. The catalog commits page explains the coordination behind multi-table commits and the runtime requirements. The SQL scripting page linked below covers compound blocks that group several statements together. Official docs: Explains how Unity Catalog coordinates commits for managed Delta Lake and Iceberg tables, with runtime requirements. Covers SQL scripting: compound blocks, variables, cursors, condition handling and control flow.

📚 Docs: https://docs.databricks.com/aws/en/delta/catalog-commits https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-scripting
🔗 More: https://www.nextgenlakehouse.com/answers/delta-lake

#Databricks #Transactions #ACID #DeltaLake #UnityCatalog #SQL #DataEngineering

— Pinned comment —
Multi-table atomic commits are covered here. The catalog commits docs and our Delta Lake hub are linked below. Docs: https://docs.databricks.com/aws/en/delta/catalog-commits Hub: https://www.nextgenlakehouse.com/answers/delta-lake

Orchestrating FHIR Files with VARIANT and Lakeflow in Databricks

This video shows how to load FHIR healthcare files into Databricks using the VARIANT type and Lakeflow pipelines. VARIANT stores semi-structured records without a fixed schema. The VARIANT page covers querying semi-structured data, extracting fields, flattening arrays and objects, and casting values. Pair it with the Lakeflow pipelines overview for orchestrating the ingestion. Official docs: Explains querying and transforming semi-structured VARIANT data, including extracting fields, flattening and casting. Introduces Lakeflow Spark Declarative Pipelines for batch and streaming workflows.

📚 Docs: https://docs.databricks.com/aws/en/semi-structured/variant https://docs.databricks.com/aws/en/ldp/
🔗 More: https://www.nextgenlakehouse.com/answers/delta-lake

#Databricks #VARIANT #FHIR #Healthcare #Lakeflow #SemiStructured #DataEngineering

— Pinned comment —
FHIR files with VARIANT and Lakeflow are covered here. The VARIANT docs and our Delta Lake hub are linked below. Docs: https://docs.databricks.com/aws/en/semi-structured/variant Hub: https://www.nextgenlakehouse.com/answers/delta-lake

Delta Lake 4.0: Connect, Clustering, Variant Type, Type Widening

Delta Lake 4.0 adds features such as liquid clustering, the VARIANT type and type widening. This video walks through what changed and how to use each feature. Liquid clustering organizes table data without partitioning or ZORDER, the VARIANT type stores semi-structured data, and type widening changes column types without rewriting files. The liquid clustering page and the type widening page are the exact references. Official docs: Explains liquid clustering, a layout technique that replaces partitioning and ZORDER for many tables. Explains changing column types to a wider type without rewriting data files, with supported changes and limits.

📚 Docs: https://docs.databricks.com/aws/en/delta/clustering https://docs.databricks.com/aws/en/delta/type-widening https://docs.databricks.com/aws/en/semi-structured/variant
🔗 More: https://www.nextgenlakehouse.com/answers/delta-lake

#Databricks #DeltaLake #DeltaLake4 #LiquidClustering #VARIANT #TypeWidening #DataEngineering

— Pinned comment —
Delta Lake 4.0 features are covered here. The liquid clustering and type widening docs and our Delta Lake hub are linked below. Docs: https://docs.databricks.com/aws/en/delta/clustering Hub: https://www.nextgenlakehouse.com/answers/delta-lake

Getting Started with Bauplan Labs

This video introduces Bauplan Labs, a third-party data platform, and how it works alongside Databricks. No Bauplan-specific Databricks doc applies, so the link points to the Delta Lake overview as the closest core reference. Delta Lake is the open table format underneath Databricks tables, covering transactions, streaming and table management. The Delta Lake overview linked below is the reference for the storage layer any table-based workflow relies on. Official docs: Overview of Delta Lake, the open storage layer under Databricks tables, with transactions and table management.

📚 Docs: https://docs.databricks.com/aws/en/delta/
🔗 More: https://www.nextgenlakehouse.com/answers/delta-lake

#DeltaLake #Databricks #DataEngineering #Lakehouse #DataPlatform #OpenSource #Bauplan

— Pinned comment —
Getting started with Bauplan Labs. The Delta Lake overview and our Delta Lake hub are linked below. Docs: https://docs.databricks.com/aws/en/delta/ Hub: https://www.nextgenlakehouse.com/answers/delta-lake

Delta Lake Tips and Tricks

This video shares practical Delta Lake tips for Databricks tables. Liquid clustering and the Delta Lake overview cover the main areas of table layout and performance. Delta Lake provides ACID transactions, time travel and scalable metadata for tables on Databricks. The liquid clustering page linked below explains the layout technique that replaces partitioning and ZORDER for many tables. Official docs: Overview of Delta Lake, the open storage layer under Databricks tables, with transactions and table management. Explains liquid clustering, a layout technique that replaces partitioning and ZORDER for many tables.

📚 Docs: https://docs.databricks.com/aws/en/delta/ https://docs.databricks.com/aws/en/delta/clustering
🔗 More: https://www.nextgenlakehouse.com/answers/delta-lake

#DeltaLake #Databricks #DataEngineering #Performance #LiquidClustering #ACID #Lakehouse

— Pinned comment —
Practical Delta Lake tips are covered here. The Delta Lake and liquid clustering docs are linked below. Docs: https://docs.databricks.com/aws/en/delta/ Hub: https://www.nextgenlakehouse.com/answers/delta-lake

Getting Started with the VARIANT Data Type in Databricks

The VARIANT data type stores semi-structured data such as JSON without a fixed schema and lets you query its fields directly. This video shows how to get started with it. Use the variant_get and try_variant_get functions to extract typed values from VARIANT columns. The query variant data doc has the full syntax. VARIANT suits event payloads and API responses whose shape changes, while stable columns should stay as typed columns for speed and clarity. 📚 Docs: https://docs.databricks.com/aws/en/semi-structured/variant. 🔗 More: https://www.nextgenlakehouse.com/learn (Delta Lake hub). #Databricks #Variant #SemiStructured #JSON #DeltaLake #Lakehouse #DataEngineering #DataAnalytics

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/semi-structured/variant More Delta Lake content lives at https://www.nextgenlakehouse.com/learn

Future of Delta Lake and Apache Iceberg with Tathagata Das

Tathagata Das discusses how Delta Lake and Apache Iceberg are converging and what that means for interoperability. Delta Lake tables can be read by Iceberg clients. Read Delta Lake tables with Iceberg clients is the doc for the interoperability setup discussed in this video. Interoperability lets engines like Iceberg read Delta tables without copying data, which reduces lock-in. Check the doc for which features and Delta versions the Iceberg read path supports. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://docs.databricks.com/aws/en/delta/iceberg-reads. 🔗 More: https://www.nextgenlakehouse.com/learn (Delta Lake hub). #Databricks #ApacheIceberg #OpenTableFormat #DeltaLake #Lakehouse #DataEngineering #DataAnalytics #AI

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/delta/iceberg-reads More Delta Lake content lives at https://www.nextgenlakehouse.com/learn

How to Make Your Delta Lake Tables Faster: OPTIMIZE, Liquid Clustering and Tuning

Delta Lake tables get faster when files are compacted with OPTIMIZE and data is laid out with liquid clustering. This video shows the tuning steps. OPTIMIZE compacts small files, and liquid clustering organises data without manual partitioning for many tables. The optimize doc and clustering doc give the exact commands. Start with the biggest tables, since compaction and clustering pay off most where scans read the most data. Measure query time before and after each change to confirm the gain. 📚 Docs: https://docs.databricks.com/aws/en/tables/operations/optimize and https://docs.databricks.com/aws/en/tables/clustering. 🔗 More: https://www.nextgenlakehouse.com/learn (Delta Lake hub). #Databricks #Performance #Optimize #LiquidClustering #DeltaLake #Lakehouse #DataEngineering #DataAnalytics

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/tables/operations/optimize More Delta Lake content lives at https://www.nextgenlakehouse.com/learn

Delta Lake Liquid Clustering Explained: How It Works and When to Use It

Liquid clustering organises Delta table data by chosen columns without fixed partitions, and the keys can be changed over time. This video explains how it works. The clustering doc shows how to enable liquid clustering with CLUSTER BY and when to choose it over partitioning. Use liquid clustering on tables with changing query patterns, where fixed partitions would be hard to maintain. Check the clustering doc for choosing columns and changing them, and measure query run time before and after. 📚 Docs: https://docs.databricks.com/aws/en/tables/clustering. 🔗 More: https://www.nextgenlakehouse.com/learn (Delta Lake hub). #Databricks #LiquidClustering #Performance #DataEngineering #DeltaLake #Lakehouse #DataAnalytics #AI

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/tables/clustering More Delta Lake content lives at https://www.nextgenlakehouse.com/learn

What's New in Apache XTable: Table Format Interoperability

What is new in Apache XTable? XTable is an open-source project that translates table metadata between open table formats, so one copy of the data can be read as Delta, Iceberg or Hudi. This video covers recent changes. For how Databricks exposes Delta tables to Iceberg readers, read the UniForm documentation linked below. If you mix engines, test the metadata sync on a non-critical table before you move production workloads. Interoperability projects change quickly, so check the release notes and the supported formats before you rely on any new XTable feature in production. 📚 Docs: https://docs.databricks.com/aws/en/delta/iceberg-reads 🔗 More: https://www.nextgenlakehouse.com/content#lakehouse #ApacheXTable #DeltaLake #ApacheIceberg #OpenTableFormats #Databricks #DataInterop

— Pinned comment —
Official reference for this topic: https://docs.databricks.com/aws/en/delta/iceberg-reads More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content#lakehouse

Databricks Supports Delta Lake and Apache Iceberg: UniForm Explained

Does Databricks support Apache Iceberg? Yes, through UniForm, which writes Iceberg metadata alongside Delta tables so Iceberg readers can query the same data. This video explains what that means for interoperability. The Delta UniForm documentation describes how to enable it, its requirements and the limits to check before you rely on it. Check the UniForm requirements and limits on the docs page before you enable it on production tables. Interoperability matters when different teams use different engines, so write down which readers need Iceberg access before you enable UniForm. 📚 Docs: https://docs.databricks.com/aws/en/delta/iceberg-reads https://docs.databricks.com/aws/en/delta/ 🔗 More: https://www.nextgenlakehouse.com/content#lakehouse #Databricks #DeltaLake #ApacheIceberg #UniForm #OpenTableFormats #Lakehouse

— Pinned comment —
Official reference for this topic: https://docs.databricks.com/aws/en/delta/iceberg-reads More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content#lakehouse

Open Table Formats Compared: Delta Lake, Apache Hudi and Apache Iceberg

Which open table format should you use: Delta Lake, Hudi or Iceberg? This video compares the three open table formats on how they store metadata, handle updates and support engines. Delta Lake is the default table format on Databricks, and UniForm exposes Delta tables to Iceberg readers. The Delta Lake documentation covers the format in detail. Choose the format based on the engines you read from, your update pattern and how much interoperability you need. Format choice affects compaction, schema evolution and which engines can read your data, so write those requirements down before you pick one. 📚 Docs: https://docs.databricks.com/aws/en/delta/ https://docs.databricks.com/aws/en/delta/iceberg-reads 🔗 More: https://www.nextgenlakehouse.com/content#lakehouse #OpenTableFormats #DeltaLake #ApacheHudi #ApacheIceberg #Databricks #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://docs.databricks.com/aws/en/delta/ More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content#lakehouse

Delta Lake and Apache Iceberg: How UniForm Lets You Use Both

How do Delta Lake and Apache Iceberg work together? This video explains how Databricks lets you use both formats on one copy of the data. With UniForm, a Delta table also publishes Iceberg metadata so Iceberg engines can read it. The UniForm documentation covers how to enable it and the requirements. Check the UniForm docs to confirm which Delta features and readers are supported before you rely on Iceberg reads. Using both formats on one copy of the data can reduce duplication, but test read performance with the engines your analysts actually use. 📚 Docs: https://docs.databricks.com/aws/en/delta/iceberg-reads https://docs.databricks.com/aws/en/delta/ 🔗 More: https://www.nextgenlakehouse.com/content#lakehouse #Databricks #DeltaLake #ApacheIceberg #UniForm #OpenTableFormats #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://docs.databricks.com/aws/en/delta/iceberg-reads More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content#lakehouse

Unity Catalog Managed Tables Explained: Delta and Iceberg | SuperSkills Ep 2

Unity Catalog managed tables are Delta Lake and Apache Iceberg tables whose storage and maintenance Unity Catalog manages for you. This SuperSkills episode explains how they work and when to use them. Managed tables keep data in a location Unity Catalog controls, so governance, optimization and lifecycle are handled without manual file management. The linked page covers creating and dropping managed Delta and Iceberg tables and how Unity Catalog manages their storage. Official docs: Covers creating and dropping Unity Catalog managed Delta and Iceberg tables and how their storage is managed.

📚 Docs: https://docs.databricks.com/aws/en/tables/managed
🔗 More: https://www.nextgenlakehouse.com/answers/unity-catalog

#Databricks #UnityCatalog #DeltaLake #ManagedTables #Iceberg #DataGovernance #SuperSkills

— Pinned comment —
This SuperSkills episode explains Unity Catalog managed tables. The docs page and our Unity Catalog hub are linked below. Docs: https://docs.databricks.com/aws/en/tables/managed Hub: https://www.nextgenlakehouse.com/answers/unity-catalog

How to Manage Secrets in Unity Catalog and Databricks (Step-by-Step)

Databricks secrets store credentials and API keys so notebooks and jobs can reference them without hardcoding values. This video shows how to create secrets, organize them in scopes and control access. The secrets management page covers storing sensitive values in secret scopes, managing access to those scopes, and referencing secrets from notebooks and jobs. Use it alongside Unity Catalog governance settings when you set permissions for production pipelines. Official docs: Explains storing credentials in secret scopes, managing access, and referencing secrets in notebooks and jobs.

📚 Docs: https://docs.databricks.com/aws/en/security/secrets/
🔗 More: https://www.nextgenlakehouse.com/answers/unity-catalog

#Databricks #UnityCatalog #Secrets #DataSecurity #DataEngineering #Security #Governance

— Pinned comment —
Secrets keep credentials out of notebook code. The secret management docs and our Unity Catalog hub are linked below. Docs: https://docs.databricks.com/aws/en/security/secrets/ Hub: https://www.nextgenlakehouse.com/answers/unity-catalog

Unity Catalog Managed Tables 2026: Product Manager Insights from DAIS

This video gives product-manager insights on Unity Catalog managed tables from the Data and AI Summit. Managed tables are the Unity Catalog-governed tables whose storage is managed for you. The managed tables page covers how Unity Catalog manages Delta Lake and Apache Iceberg table storage, optimization and external access, plus how to create and drop them. Pair it with the predictive optimization page for automatic maintenance. Official docs: Covers creating and dropping Unity Catalog managed Delta and Iceberg tables and how their storage is managed.

📚 Docs: https://docs.databricks.com/aws/en/tables/managed
🔗 More: https://www.nextgenlakehouse.com/answers/unity-catalog

#Databricks #UnityCatalog #ManagedTables #DeltaLake #Iceberg #DataAndAI #ProductManagement

— Pinned comment —
PM insights on managed tables from DAIS. The managed tables docs and our Unity Catalog hub are linked below. Docs: https://docs.databricks.com/aws/en/tables/managed Hub: https://www.nextgenlakehouse.com/answers/unity-catalog

Stop Manual Tuning: Predictive Optimization Explained

Predictive optimization runs maintenance such as OPTIMIZE, VACUUM and ANALYZE automatically on Unity Catalog managed tables. This video explains how it removes manual tuning. The predictive optimization page covers how Databricks decides which maintenance to run on managed tables, how to enable and verify it, and how to monitor it. It is the exact reference for the behavior described in the video. Official docs: Explains how predictive optimization runs OPTIMIZE, VACUUM and ANALYZE automatically on managed tables. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/optimizations/predictive-optimization
🔗 More: https://www.nextgenlakehouse.com/answers/unity-catalog

#Databricks #PredictiveOptimization #UnityCatalog #DeltaLake #Performance #DataEngineering #Optimization

— Pinned comment —
Predictive optimization removes manual table maintenance. The predictive optimization docs and our Unity Catalog hub are linked below. Docs: https://docs.databricks.com/aws/en/optimizations/predictive-optimization Hub: https://www.nextgenlakehouse.com/answers/unity-catalog

Data Governance with Unity Catalog on Databricks

Unity Catalog is the unified governance layer for data and AI on Databricks, covering access control, lineage and auditing. This video introduces data governance with Unity Catalog. The Unity Catalog overview explains how access control, lineage, auditing and sharing work across workspaces. The lineage page linked below shows how data flow is tracked at the table and column level. Official docs: Introduces Unity Catalog, the governance layer covering access control, lineage and auditing. Explains how Unity Catalog tracks lineage across tables, jobs, notebooks and dashboards, down to columns.

📚 Docs: https://docs.databricks.com/aws/en/data-governance/unity-catalog/ https://docs.databricks.com/aws/en/data-governance/unity-catalog/data-lineage
🔗 More: https://www.nextgenlakehouse.com/answers/unity-catalog

#Databricks #UnityCatalog #DataGovernance #DataLineage #AccessControl #DataSecurity #Lakehouse

— Pinned comment —
Data governance with Unity Catalog is the topic. The Unity Catalog docs and our Unity Catalog hub are linked below. Docs: https://docs.databricks.com/aws/en/data-governance/unity-catalog/ Hub: https://www.nextgenlakehouse.com/answers/unity-catalog

SET MANAGED: Convert External Tables to Unity Catalog Managed Tables

The SET MANAGED command converts external Delta Lake tables into Unity Catalog managed tables. This video shows the conversion step by step. The conversion page covers the ALTER TABLE ... SET MANAGED syntax and Catalog Explorer option, with prerequisites, downtime, rollback and troubleshooting. It is the exact procedure shown in the video. Official docs: Explains converting external or foreign Delta tables to Unity Catalog managed tables with SET MANAGED. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/tables/convert-external-managed
🔗 More: https://www.nextgenlakehouse.com/answers/unity-catalog

#Databricks #UnityCatalog #ManagedTables #DeltaLake #Migration #DataGovernance #SQL

— Pinned comment —
Converting external tables to UC managed tables is shown here. The conversion docs and our Unity Catalog hub are linked below. Docs: https://docs.databricks.com/aws/en/tables/convert-external-managed Hub: https://www.nextgenlakehouse.com/answers/unity-catalog

Getting Started with Governed Tags in Databricks

Governed tags are Unity Catalog tags with enforced rules, used to classify data and drive access policies. This video shows how to create and apply them. The tags page covers adding, updating and removing tags on Unity Catalog objects such as catalogs, tables and volumes, along with governed tag rules and tag-based search. Governed tags also feed attribute-based access control. Official docs: Explains adding, updating and removing tags on Unity Catalog objects, including governed tags and tag-based search. Explains attribute-based access control in Unity Catalog, driven by governed tags, with policies and grants.

📚 Docs: https://docs.databricks.com/aws/en/database-objects/tags https://docs.databricks.com/aws/en/data-governance/unity-catalog/abac/
🔗 More: https://www.nextgenlakehouse.com/answers/unity-catalog

#Databricks #UnityCatalog #GovernedTags #DataGovernance #ABAC #DataClassification #DataSecurity

— Pinned comment —
Governed tags classify and govern Unity Catalog data. The tags docs and our Unity Catalog hub are linked below. Docs: https://docs.databricks.com/aws/en/database-objects/tags Hub: https://www.nextgenlakehouse.com/answers/unity-catalog

Request Access in Unity Catalog: A Self-Service Workflow

Request access in Unity Catalog lets users ask owners for access to a data object from a self-service flow. This video shows the workflow end to end. The manage privileges page covers how owners and admins grant, revoke and check permissions on Unity Catalog objects, which the request flow feeds into. A dedicated request-access page was not found, so this is the closest verified reference. Official docs: Covers granting, revoking and checking Unity Catalog permissions and managing object ownership.

📚 Docs: https://docs.databricks.com/aws/en/data-governance/unity-catalog/manage-privileges/
🔗 More: https://www.nextgenlakehouse.com/answers/unity-catalog

#Databricks #UnityCatalog #Permissions #AccessControl #DataGovernance #SelfService #DataSecurity

— Pinned comment —
The Unity Catalog request-access workflow is shown here. The privileges docs and our Unity Catalog hub are linked below. Docs: https://docs.databricks.com/aws/en/data-governance/unity-catalog/manage-privileges/ Hub: https://www.nextgenlakehouse.com/answers/unity-catalog

Unity Catalog and Apache Iceberg Demo from the Data+AI Summit 2025

This demo shows Unity Catalog governing Apache Iceberg tables, with access from external engines. It was recorded at the Data and AI Summit 2025. The Iceberg page covers managed and foreign Iceberg tables in Databricks, their requirements, and how external engines access them. Pair it with the Unity Catalog overview for governance. Official docs: Explains Apache Iceberg support in Databricks, including managed and foreign tables and external engine access. Introduces Unity Catalog, the governance layer covering access control, lineage and auditing.

📚 Docs: https://docs.databricks.com/aws/en/iceberg/ https://docs.databricks.com/aws/en/data-governance/unity-catalog/
🔗 More: https://www.nextgenlakehouse.com/answers/unity-catalog

#Databricks #UnityCatalog #ApacheIceberg #DAIS2025 #OpenTable #DataGovernance #Lakehouse

— Pinned comment —
Unity Catalog with Apache Iceberg is the demo. The Iceberg docs and our Unity Catalog hub are linked below. Docs: https://docs.databricks.com/aws/en/iceberg/ Hub: https://www.nextgenlakehouse.com/answers/unity-catalog

Unity Catalog Discover Demo from the Data+AI Summit 2025

This demo shows how to discover data assets in Unity Catalog from the Data and AI Summit 2025. Discovery relies on the governance metadata that Unity Catalog stores. The Unity Catalog overview explains how catalogs, schemas, lineage and auditing fit together to make data findable and governed. The lineage page linked below covers table and column-level tracking. Official docs: Introduces Unity Catalog, the governance layer covering access control, lineage and auditing. Explains how Unity Catalog tracks lineage across tables, jobs, notebooks and dashboards, down to columns.

📚 Docs: https://docs.databricks.com/aws/en/data-governance/unity-catalog/ https://docs.databricks.com/aws/en/data-governance/unity-catalog/data-lineage
🔗 More: https://www.nextgenlakehouse.com/answers/unity-catalog

#Databricks #UnityCatalog #DataDiscovery #DataLineage #DAIS2025 #DataGovernance #Metadata

— Pinned comment —
Discovering data assets in Unity Catalog is the demo. The Unity Catalog docs and our Unity Catalog hub are linked below. Docs: https://docs.databricks.com/aws/en/data-governance/unity-catalog/ Hub: https://www.nextgenlakehouse.com/answers/unity-catalog

Getting Started with Attribute-Based Access Control (ABAC) in Databricks

Attribute-based access control (ABAC) in Unity Catalog applies row filters and column masks based on governed tags. This video shows how to set up ABAC policies. The ABAC tutorial walks through row filter and column mask policies in Unity Catalog, configured in Catalog Explorer. The ABAC overview linked below explains how governed tags drive the policies. Official docs: Walks through row filter and column mask ABAC policies in Unity Catalog using Catalog Explorer. Explains attribute-based access control in Unity Catalog, driven by governed tags, with policies and grants.

📚 Docs: https://docs.databricks.com/aws/en/data-governance/unity-catalog/abac/tutorial https://docs.databricks.com/aws/en/data-governance/unity-catalog/abac/
🔗 More: https://www.nextgenlakehouse.com/answers/unity-catalog

#Databricks #UnityCatalog #ABAC #DataGovernance #RowFilters #ColumnMasks #DataSecurity

— Pinned comment —
ABAC policies with row filters and column masks are covered here. The ABAC docs and our Unity Catalog hub are linked below. Docs: https://docs.databricks.com/aws/en/data-governance/unity-catalog/abac/tutorial Hub: https://www.nextgenlakehouse.com/answers/unity-catalog

Meet Databricks MVP Julia Førde: Data Migration and Unity Catalog Interview

This Databricks MVP interview with Julia Førde covers data migration and Unity Catalog. It shares practitioner experience rather than a product feature tutorial. For migration guidance, the Hive metastore upgrade doc describes how existing tables move to Unity Catalog. Migrations usually move in waves: inventory the Hive metastore, upgrade schemas and tables, then update jobs and permissions. This interview shares how a practitioner approached that work. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://docs.databricks.com/aws/en/data-governance/unity-catalog/migrate. 🔗 More: https://www.nextgenlakehouse.com/learn (Unity Catalog hub). #Databricks #DatabricksMVP #UnityCatalog #DataMigration #DataGovernance #Lakehouse #DataAnalytics #DataEngineering

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/data-governance/unity-catalog/migrate More Unity Catalog content lives at https://www.nextgenlakehouse.com/learn

Unity Catalog Managed Tables: The Ultimate Guide for Delta and Iceberg

Unity Catalog managed tables are tables whose data and metadata Databricks manages and governs in Unity Catalog. This guide explains how they work and when to use them. Managed tables use Delta Lake by default and receive automatic maintenance features. The Unity Catalog overview describes the namespace and access model. Managed tables are the simplest starting point, because Unity Catalog handles storage location and lifecycle. Choose external tables only when data must live in a location you control directly. 📚 Docs: https://docs.databricks.com/aws/en/tables/managed and https://docs.databricks.com/aws/en/data-governance/unity-catalog/. 🔗 More: https://www.nextgenlakehouse.com/learn (Unity Catalog hub). #Databricks #ManagedTables #Iceberg #DeltaLake #UnityCatalog #DataGovernance #Lakehouse #DataAnalytics

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/tables/managed More Unity Catalog content lives at https://www.nextgenlakehouse.com/learn

Why You Should Use an Open Source Semantic Layer for Consistent Metrics

An open source semantic layer defines business metrics once so every dashboard and AI tool reads the same numbers. This video explains why that matters and how to approach it. Unity Catalog metric views store governed metric definitions that BI tools and Genie can query. The metric views docs show how to define measures and dimensions. A semantic layer matters when many tools need the same definitions, since it stops each dashboard re-implementing revenue or churn differently. Start by defining the few metrics the business debates most. 📚 Docs: https://docs.databricks.com/aws/en/uc-semantics/metric-views. 🔗 More: https://www.nextgenlakehouse.com/learn (Unity Catalog hub). #Databricks #SemanticLayer #Metrics #OpenSource #UnityCatalog #DataGovernance #Lakehouse #DataAnalytics

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/uc-semantics/metric-views More Unity Catalog content lives at https://www.nextgenlakehouse.com/learn

Unity Catalog OSS: Open-Source Data Governance Explained

Unity Catalog OSS is the open source release of Unity Catalog, a governance layer for data and AI assets. This video explains what is open and how it fits the lakehouse. The unitycatalog GitHub repository hosts the open source project, and the Databricks blog post announces the open sourcing. Open sourcing the catalog means teams can read the code, self-host where appropriate and contribute fixes. Check the repository for the current feature set, which differs from the managed Databricks product. 📚 Docs: https://github.com/unitycatalog/unitycatalog and https://www.databricks.com/blog/open-sourcing-unity-catalog. 🔗 More: https://www.nextgenlakehouse.com/learn (Unity Catalog hub). #Databricks #UnityCatalog #OpenSource #DataGovernance #Lakehouse #DataAnalytics #DataEngineering #AI

— Pinned comment —
Official reference: https://github.com/unitycatalog/unitycatalog More Unity Catalog content lives at https://www.nextgenlakehouse.com/learn

Open-Source Governance and Data/AI Standards with Denny Lee

This conversation with Denny Lee covers open-source governance and how data and AI standards are consolidating. It connects open formats like Delta Lake and open catalogs like Unity Catalog. Delta Lake is the open table format used by the lakehouse, and the unitycatalog repo is the open source governance project. Standards discussions matter when choosing table formats and catalogs, because they determine how easily other engines can read your data later. Use this talk to weigh open versus managed governance options. 📚 Docs: https://delta.io/ and https://github.com/unitycatalog/unitycatalog. 🔗 More: https://www.nextgenlakehouse.com/learn (Unity Catalog hub). #Databricks #OpenSource #DeltaLake #DataGovernance #UnityCatalog #Lakehouse #DataAnalytics #DataEngineering

— Pinned comment —
Official reference: https://delta.io/ More Unity Catalog content lives at https://www.nextgenlakehouse.com/learn

Storage Wars: Cloud Storage for the Lakehouse with Kyle Weller

Cloud storage choices shape cost, governance and performance on the lakehouse, and this conversation with Kyle Weller compares them. It focuses on how Unity Catalog connects to cloud object storage. Unity Catalog uses storage credentials and external locations to govern access to cloud object storage. The cloud storage doc explains how to connect it. Storage choices affect cost and governance, so decide on locations, credentials and external locations before creating many tables. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://docs.databricks.com/aws/en/connect/unity-catalog/cloud-storage. 🔗 More: https://www.nextgenlakehouse.com/learn (Unity Catalog hub). #Databricks #CloudStorage #UnityCatalog #DataGovernance #Lakehouse #DataAnalytics #DataEngineering #AI

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/connect/unity-catalog/cloud-storage More Unity Catalog content lives at https://www.nextgenlakehouse.com/learn

Open Sourcing Unity Catalog: What It Means for Databricks Governance

Open sourcing Unity Catalog makes its governance code available to the community, which changes how teams can adopt it. This video explains the decision and what it means in practice. The Databricks blog post on open sourcing Unity Catalog and the unitycatalog GitHub repository are the primary references. Open sourcing makes the governance code inspectable, which helps security and platform teams that need to audit how permissions are enforced. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://www.databricks.com/blog/open-sourcing-unity-catalog and https://github.com/unitycatalog/unitycatalog. 🔗 More: https://www.nextgenlakehouse.com/learn (Unity Catalog hub). #Databricks #UnityCatalog #OpenSource #DataGovernance #Lakehouse #DataAnalytics #DataEngineering #AI

— Pinned comment —
Official reference: https://www.databricks.com/blog/open-sourcing-unity-catalog More Unity Catalog content lives at https://www.nextgenlakehouse.com/learn

Future of Data Governance and AI with Matei Zaharia, Databricks CTO

Matei Zaharia discusses how data governance and AI are converging, with Unity Catalog as the governance layer. This interview covers where the platform is heading. Unity Catalog governs tables, files, models and AI assets with a single permission model, and the AI Gateway doc covers governing model access. Governance and AI are converging, so the same permission model that protects tables also applies to models and other AI assets. Check the AI Gateway doc to see how model access is governed for LLM endpoints. 📚 Docs: https://docs.databricks.com/aws/en/data-governance/unity-catalog/ and https://docs.databricks.com/aws/en/ai-gateway/. 🔗 More: https://www.nextgenlakehouse.com/learn (Unity Catalog hub). #Databricks #DataGovernance #AI #UnityCatalog #Lakehouse #DataAnalytics #DataEngineering #SQL

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/data-governance/unity-catalog/ More Unity Catalog content lives at https://www.nextgenlakehouse.com/learn

Configuration d'Azure Databricks avec Unity Catalog (guide FR)

Configuring Azure Databricks with Unity Catalog means creating a metastore, attaching workspaces and setting up storage for governed access. This French-language video walks through the setup. The Azure Databricks Unity Catalog get-started guide covers the metastore, catalogs and storage prerequisites. Set up the metastore and storage credentials first, because workspaces and catalogs depend on them. Follow the get-started steps in order and check the Azure role assignments the guide lists. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/get-started. 🔗 More: https://www.nextgenlakehouse.com/learn (Unity Catalog hub). #Databricks #AzureDatabricks #UnityCatalog #Configuration #DataGovernance #Lakehouse #DataAnalytics #DataEngineering

— Pinned comment —
Official reference: https://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/get-started More Unity Catalog content lives at https://www.nextgenlakehouse.com/learn

Getting Started with Predictive Optimization for Unity Catalog Tables

Predictive optimization automatically runs OPTIMIZE, VACUUM and clustering maintenance on Unity Catalog managed tables. This video shows how to get started with it. The predictive optimization doc explains which tables are covered and how to enable it at the account or metastore level. Predictive optimization removes manual OPTIMIZE and VACUUM scheduling for covered tables. Check the doc for supported table types and regions, then review maintenance history after a week. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://docs.databricks.com/aws/en/optimizations/predictive-optimization. 🔗 More: https://www.nextgenlakehouse.com/learn (Unity Catalog hub). #Databricks #PredictiveOptimization #DeltaLake #Performance #UnityCatalog #DataGovernance #Lakehouse #DataAnalytics

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/optimizations/predictive-optimization More Unity Catalog content lives at https://www.nextgenlakehouse.com/learn

Getting Started with UCX: Unity Catalog Migration Toolkit

UCX is the Databricks Labs toolkit that helps assess and migrate workspaces to Unity Catalog. This video shows how to get started with it. The UCX project docs describe the assessment and migration workflows, and the GitHub repo hosts the source. Run the UCX assessment first to see what must change before migrating workspaces. Review the generated plan with data owners before running the migration steps, and keep the project docs open because workflows change between releases. 📚 Docs: https://databrickslabs.github.io/ucx/ and https://github.com/databrickslabs/ucx. 🔗 More: https://www.nextgenlakehouse.com/learn (Unity Catalog hub). #Databricks #UCX #Migration #DatabricksLabs #UnityCatalog #DataGovernance #Lakehouse #DataAnalytics

— Pinned comment —
Official reference: https://databrickslabs.github.io/ucx/ More Unity Catalog content lives at https://www.nextgenlakehouse.com/learn

What's New in Workspace-Catalog Binding in Unity Catalog

Workspace-catalog binding restricts a catalog so it is only accessible from specific workspaces. This video explains what is new in that feature. The workspace-catalog binding doc describes how to bind catalogs to workspaces and which privileges apply. Binding restricts a catalog to chosen workspaces, which supports separating production and development data. Check the binding doc for the privileges needed to change bindings, and test access from a non-bound workspace. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://docs.databricks.com/aws/en/data-governance/unity-catalog/access-control/workspace-catalog-binding. 🔗 More: https://www.nextgenlakehouse.com/learn (Unity Catalog hub). #Databricks #WorkspaceBinding #DataGovernance #Catalogs #UnityCatalog #Lakehouse #DataAnalytics #DataEngineering

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/data-governance/unity-catalog/access-control/workspace-catalog-binding More Unity Catalog content lives at https://www.nextgenlakehouse.com/learn

Row-Level and Column-Level Security in Unity Catalog: Getting Started

Row filters and column masks in Unity Catalog restrict which rows and columns each user sees, enforced in the query engine. This video shows how to set them up. The row filters and column masks doc describes creating filter functions and masking functions and attaching them to tables. Filters and masks attach to tables, so access rules apply in every query path, including SQL and dashboards. Test with a user who has restricted access to confirm the results. 📚 Docs: https://docs.databricks.com/aws/en/data-governance/unity-catalog/filters-and-masks. 🔗 More: https://www.nextgenlakehouse.com/learn (Unity Catalog hub). #Databricks #RowLevelSecurity #ColumnMasking #DataGovernance #UnityCatalog #Lakehouse #DataAnalytics #DataEngineering

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/data-governance/unity-catalog/filters-and-masks More Unity Catalog content lives at https://www.nextgenlakehouse.com/learn

Getting Started with Tags in Unity Catalog: Governance Tagging Guide

Tags in Unity Catalog attach key-value metadata to catalogs, schemas, tables and columns for discovery and governance. This video shows how to get started. The tags doc covers applying tags with SQL and the UI, and how tags support access policies. Tags help discover sensitive data and group assets by owner or domain. Apply them with a consistent naming convention so search and policies work across teams, and check the tags doc for supported objects. 📚 Docs: https://docs.databricks.com/aws/en/database-objects/tags. 🔗 More: https://www.nextgenlakehouse.com/learn (Unity Catalog hub). #Databricks #Tags #DataGovernance #Metadata #UnityCatalog #Lakehouse #DataAnalytics #DataEngineering

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/database-objects/tags More Unity Catalog content lives at https://www.nextgenlakehouse.com/learn

Getting Started with Lakehouse Federation: Query External Databases

Lakehouse Federation lets you query external databases from Unity Catalog without moving the data. This video shows how to get started. The federation doc covers creating connections and foreign catalogs, and the MySQL page shows one connector in detail. Foreign catalogs let you query external sources with the same SQL and governance you use elsewhere in Unity Catalog. Check the connection doc for source types and credentials, and start with a read-only database user. 📚 Docs: https://docs.databricks.com/aws/en/query-federation/ and https://docs.databricks.com/aws/en/query-federation/mysql. 🔗 More: https://www.nextgenlakehouse.com/learn (Unity Catalog hub). #Databricks #LakehouseFederation #Databases #DataIntegration #UnityCatalog #DataGovernance #Lakehouse #DataAnalytics

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/query-federation/ More Unity Catalog content lives at https://www.nextgenlakehouse.com/learn

Getting Started with Databricks Volumes for Files in Unity Catalog

Volumes in Unity Catalog govern non-tabular files such as CSVs, images and models under a catalog and schema. This video shows how to get started. The volumes doc covers creating volumes and accessing them from SQL, Python and the file browser. Volumes suit files that do not fit tables, such as raw CSVs staged for ingestion, images and documents. Set privileges on the volume like any other Unity Catalog object, and check the volumes doc for path formats. 📚 Docs: https://docs.databricks.com/aws/en/volumes/. 🔗 More: https://www.nextgenlakehouse.com/learn (Unity Catalog hub). #Databricks #Volumes #Files #DataGovernance #UnityCatalog #Lakehouse #DataAnalytics #DataEngineering

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/volumes/ More Unity Catalog content lives at https://www.nextgenlakehouse.com/learn

How to UNDROP Databricks Managed Tables and Recover Dropped Data

UNDROP restores a managed table that was dropped in Unity Catalog within its retention window. This video shows how to recover it. The UNDROP reference explains the syntax and retention behaviour for dropped managed tables. UNDROP only works within the retention window, so act quickly after an accidental drop. Check the reference for the syntax and conditions, and test the restore on a copy first if the table is important. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-syntax-ddl-undrop-table. 🔗 More: https://www.nextgenlakehouse.com/learn (Unity Catalog hub). #Databricks #UNDROP #DeltaLake #DataRecovery #UnityCatalog #DataGovernance #Lakehouse #DataAnalytics

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-syntax-ddl-undrop-table More Unity Catalog content lives at https://www.nextgenlakehouse.com/learn

Getting Started with Catalog Binding in Unity Catalog

Catalog binding limits a catalog to specific workspaces so data stays isolated by environment or team. This video shows how to get started. The workspace-catalog binding doc describes binding a catalog to one or more workspaces and checking the bindings. Binding is useful when the same catalog name should only be visible in specific workspaces. Check the binding doc for the steps to bind and unbind, and test access from each workspace after changes. 📚 Docs: https://docs.databricks.com/aws/en/data-governance/unity-catalog/access-control/workspace-catalog-binding. 🔗 More: https://www.nextgenlakehouse.com/learn (Unity Catalog hub). #Databricks #CatalogBinding #DataIsolation #DataGovernance #UnityCatalog #Lakehouse #DataAnalytics #DataEngineering

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/data-governance/unity-catalog/access-control/workspace-catalog-binding More Unity Catalog content lives at https://www.nextgenlakehouse.com/learn

Set Up Databricks Unity Catalog with Terraform: Step-by-Step

You can provision Unity Catalog metastores, catalogs and grants with the Databricks Terraform provider as infrastructure as code. This video shows the setup. The Terraform provider doc covers authentication and the resources for metastores, catalogs and grants. Terraform gives you reviewable, repeatable setup for metastores, catalogs and grants. Keep the state file and credentials in a secure backend, and apply changes in a dev workspace before production. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://docs.databricks.com/aws/en/dev-tools/terraform/. 🔗 More: https://www.nextgenlakehouse.com/learn (Unity Catalog hub). #Databricks #Terraform #IaC #DevOps #UnityCatalog #DataGovernance #Lakehouse #DataAnalytics

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/dev-tools/terraform/ More Unity Catalog content lives at https://www.nextgenlakehouse.com/learn

Upgrade Hive Metastore Tables to Unity Catalog with SYNC

The SYNC command upgrades Hive metastore tables to Unity Catalog without rewriting the data. This video shows how to run it. The Hive table upgrade doc covers the upgrade options, and the SYNC reference lists the syntax. SYNC upgrades Hive metastore tables to Unity Catalog, and the reference lists its options. Review the dry run output before running the upgrade on production tables, and check the permissions required. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://docs.databricks.com/aws/en/data-governance/unity-catalog/migrate and https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-syntax-aux-sync. 🔗 More: https://www.nextgenlakehouse.com/learn (Unity Catalog hub). #Databricks #HiveMetastore #Migration #SYNC #UnityCatalog #DataGovernance #Lakehouse #DataAnalytics

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/data-governance/unity-catalog/migrate More Unity Catalog content lives at https://www.nextgenlakehouse.com/learn

Azure Databricks: Set Up SCIM Provisioning in the Account Console

SCIM provisioning syncs users and groups from Microsoft Entra ID into Azure Databricks automatically. This video shows how to set it up in the account console. The Azure Databricks SCIM doc describes provisioning users and groups from the identity provider. SCIM keeps users and groups in step with your identity provider, so access changes flow automatically. Check the SCIM doc for provisioning settings and verify group membership after the first sync. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://learn.microsoft.com/en-us/azure/databricks/admin/users-groups/scim/. 🔗 More: https://www.nextgenlakehouse.com/learn (Unity Catalog hub). #Databricks #AzureDatabricks #SCIM #EntraID #UnityCatalog #DataGovernance #Lakehouse #DataAnalytics

— Pinned comment —
Official reference: https://learn.microsoft.com/en-us/azure/databricks/admin/users-groups/scim/ More Unity Catalog content lives at https://www.nextgenlakehouse.com/learn

Déploiement de Unity Catalog sur Azure Databricks (guide FR)

Deploying Unity Catalog on Azure Databricks means creating a metastore and assigning it to workspaces. This French-language video walks through the deployment. The Azure Unity Catalog overview covers the metastore, catalogs and access model. A metastore is the unit of governance that workspaces attach to. Follow the overview to understand the metastore, catalogs and access model, and plan the storage account and region before creating it. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/. 🔗 More: https://www.nextgenlakehouse.com/learn (Unity Catalog hub). #Databricks #AzureDatabricks #Deployment #DataGovernance #UnityCatalog #Lakehouse #DataAnalytics #DataEngineering

— Pinned comment —
Official reference: https://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/ More Unity Catalog content lives at https://www.nextgenlakehouse.com/learn

Azure Databricks Unity Catalog Setup Guide: Metastore, Catalogs and Storage

Setting up Unity Catalog on Azure Databricks involves a metastore, storage credentials and catalogs for governed access. This guide walks through the setup. The Azure Unity Catalog get-started guide lists the prerequisites and steps for the first metastore. Start by creating the metastore and storage credentials, then attach the workspace. Check the get-started guide for the Azure roles needed, and create one catalog per environment or domain. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/get-started. 🔗 More: https://www.nextgenlakehouse.com/learn (Unity Catalog hub). #Databricks #AzureDatabricks #Metastore #Setup #UnityCatalog #DataGovernance #Lakehouse #DataAnalytics

— Pinned comment —
Official reference: https://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/get-started More Unity Catalog content lives at https://www.nextgenlakehouse.com/learn

Introduction to Databricks Unity Catalog: Governance for Data and AI

Unity Catalog is the governance layer for data and AI assets on Databricks, covering access control, lineage and auditing. This introduction explains the core concepts. The Unity Catalog overview describes the metastore, catalogs, schemas and the permission model. Unity Catalog centralises permissions, lineage and audit for data and AI assets. Read the overview to learn the metastore, catalogs, schemas and privileges, and start with one catalog for a pilot team. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://docs.databricks.com/aws/en/data-governance/unity-catalog/ and https://docs.databricks.com/aws/en/data-governance/unity-catalog/get-started. 🔗 More: https://www.nextgenlakehouse.com/learn (Unity Catalog hub). #Databricks #Lakehouse #DataGovernance #UnityCatalog #DataAnalytics #DataEngineering #AI #SQL

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/data-governance/unity-catalog/ More Unity Catalog content lives at https://www.nextgenlakehouse.com/learn

Databricks Governance Hub: Govern AI Usage, Guardrails and Spend in One Place

How do you govern AI across a Databricks account? This video introduces Governance Hub, an account-level dashboard that shows token usage, guardrail coverage, AI budgets, spend and the active models and users. Governance Hub gives visibility and read-only recommendations, while Unity Catalog and Unity Gateway enforce access and traffic rules. An account admin enables it from the account console under Previews, as the feature is in Beta. Start by reviewing the guardrail coverage and AI budget views to see where usage and spend are not yet governed. 📚 Docs: https://docs.databricks.com/aws/en/admin/governance-hub https://docs.databricks.com/aws/en/admin/governance-hub/ai 🔗 More: https://www.nextgenlakehouse.com/content#unity-catalog #Databricks #AIGovernance #DataGovernance #UnityCatalog #UnityGateway #AI

— Pinned comment —
Official reference for this topic: https://docs.databricks.com/aws/en/admin/governance-hub More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content#unity-catalog

Unity Catalog Discover Demo: Finding Data in Catalog Explorer

How do you discover data in Unity Catalog? This Data+AI Summit 2025 demo shows searching and browsing data assets, including tables, views and their metadata, so users can find the right data. Catalog Explorer is the interface for this, and it works with the governance rules in Unity Catalog, so users only see what they are allowed to access. Use Catalog Explorer to check the permissions and lineage of a table before you share it with a new team. Discovery features matter most in large organisations where many teams publish tables, so set comments and tags on key assets to help people find them. 📚 Docs: https://docs.databricks.com/aws/en/catalog-explorer/ https://docs.databricks.com/aws/en/data-governance/unity-catalog/ 🔗 More: https://www.nextgenlakehouse.com/content#unity-catalog #Databricks #UnityCatalog #DataDiscovery #CatalogExplorer #DataGovernance #DataAISummit

— Pinned comment —
Official reference for this topic: https://docs.databricks.com/aws/en/catalog-explorer/ More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content#unity-catalog

Matei Zaharia on Unity Catalog at Data+AI Summit 2025

What did Matei Zaharia say about Unity Catalog? In this Data+AI Summit 2025 stage talk, the Databricks co-founder discusses Unity Catalog as the governance layer for data and AI assets across the lakehouse. The official documentation explains the catalog, schema and privilege model that these ideas map onto. Use the talk to understand the governance model, then apply the catalog and schema privileges it describes. Governance talks at the Summit often explain why the catalog model matters for AI as well as analytics, so the ideas apply to both. 📚 Docs: https://docs.databricks.com/aws/en/data-governance/unity-catalog/ 🔗 More: https://www.nextgenlakehouse.com/content#unity-catalog #Databricks #UnityCatalog #MateiZaharia #DataGovernance #DataAISummit #OpenSource

— Pinned comment —
Official reference for this topic: https://docs.databricks.com/aws/en/data-governance/unity-catalog/ More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content#unity-catalog

Why Unity Catalog Matters: Governance for Data and AI

Why is Unity Catalog important? This video explains why one governance layer matters for data and AI across a lakehouse. Unity Catalog provides a single place for access control, lineage and auditing of tables, volumes, models and other assets. The Unity Catalog documentation and product page describe these capabilities in detail. Start by setting up catalogs and permissions for one domain, then expand the governance model to other teams. Governance is easier to explain when you tie each policy to a business risk, such as access to customer data or the audit of AI outputs. 📚 Docs: https://docs.databricks.com/aws/en/data-governance/unity-catalog/ https://www.databricks.com/product/unity-catalog 🔗 More: https://www.nextgenlakehouse.com/content#unity-catalog #Databricks #UnityCatalog #DataGovernance #DataLineage #AIGovernance #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://docs.databricks.com/aws/en/data-governance/unity-catalog/ More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content#unity-catalog

Databricks Managed Tables Explained: Unity Catalog Storage

What are Databricks managed tables? This video explains managed tables, where Unity Catalog controls the storage location and lifecycle of the table data. Managed tables use Delta Lake by default and support features such as predictive optimisation. The managed tables documentation covers the differences from external tables. Choose managed tables for simpler governance and let Unity Catalog handle the storage, and use external tables when you must control the location. Managed tables suit most new workloads, while external tables fit cases where another system must own the files, so decide based on that ownership. 📚 Docs: https://docs.databricks.com/aws/en/tables/managed 🔗 More: https://www.nextgenlakehouse.com/content#unity-catalog #Databricks #ManagedTables #UnityCatalog #DeltaLake #Lakehouse #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://docs.databricks.com/aws/en/tables/managed More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content#unity-catalog

Denny Lee on Open Source Unity Catalog

What is open-source Unity Catalog? Denny Lee, from Databricks, explains the Unity Catalog open-source project and how teams can use it for governance across engines. The Databricks blog on open-sourcing Unity Catalog describes the release and its scope. Try the open-source Unity Catalog locally to see how its governance model works outside the managed service. Open-source governance is easier to test than to describe, so try one catalog and one policy locally before you plan a wider rollout. 📚 Docs: https://www.databricks.com/blog/open-sourcing-unity-catalog https://docs.databricks.com/aws/en/data-governance/unity-catalog/ 🔗 More: https://www.nextgenlakehouse.com/content#unity-catalog #Databricks #UnityCatalog #OpenSource #DataGovernance #DennyLee #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/blog/open-sourcing-unity-catalog More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content#unity-catalog

Unity Catalog Is Open Source: What It Means for Data Governance

Is Unity Catalog open source? Yes. This video explains the open-source release of Unity Catalog, a governance layer for data and AI assets. The release blog covers what was open-sourced and how the project fits with the Databricks product, and the Unity Catalog documentation covers the product features. Check the release blog for the exact scope of what was open-sourced before you plan a migration. Open-source governance lets teams review the code behind access policies, which matters for audits and for teams that run multiple engines. 📚 Docs: https://www.databricks.com/blog/open-sourcing-unity-catalog https://docs.databricks.com/aws/en/data-governance/unity-catalog/ 🔗 More: https://www.nextgenlakehouse.com/content#unity-catalog #Databricks #UnityCatalog #OpenSource #DataGovernance #DataAI #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/blog/open-sourcing-unity-catalog More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content#unity-catalog

Real-Time Analytics on the Lakehouse with Reyden

This video covers real-time analytics on the lakehouse with the partner Reyden. No Reyden-specific Databricks doc was verified, so the link points to the Databricks SQL overview. Real-time analytics on Databricks typically pairs streaming ingestion with SQL querying over governed tables. The Databricks SQL overview linked below describes the warehouse and query interfaces used for analytics on the lakehouse. Official docs: Introduces Databricks SQL, the lakehouse data warehouse for queries, monitoring and BI. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/sql/
🔗 More: https://www.nextgenlakehouse.com/answers/sql-analytics

#Databricks #RealTimeAnalytics #Lakehouse #DatabricksSQL #Analytics #Reyden #DataEngineering

— Pinned comment —
Real-time analytics on the lakehouse is the theme. The SQL overview and our SQL analytics hub are linked below. Docs: https://docs.databricks.com/aws/en/sql/ Hub: https://www.nextgenlakehouse.com/answers/sql-analytics

How to Use Temporary Tables in Databricks SQL

Databricks SQL documents temporary views, visible only to the session that creates them and dropped when the session ends. This video shows how to use them for intermediate results. The CREATE VIEW reference covers TEMPORARY and GLOBAL TEMPORARY views and their session scope. Databricks documents temporary views rather than a separate temporary table object, so the linked page is the exact syntax reference. Official docs: Documents CREATE VIEW, including TEMPORARY views that are visible only to the session that created them.

📚 Docs: https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-syntax-ddl-create-view
🔗 More: https://www.nextgenlakehouse.com/answers/sql-analytics

#Databricks #DatabricksSQL #SQL #TemporaryViews #DataEngineering #SQLTips #Analytics

— Pinned comment —
Temporary views are the documented Databricks SQL construct for session-scoped results. The CREATE VIEW docs and our SQL analytics hub are linked below. Docs: https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-syntax-ddl-create-view Hub: https://www.nextgenlakehouse.com/answers/sql-analytics

All You Need to Know About Databricks SQL (Part 2)

Databricks SQL is the lakehouse data warehouse for running analytics and BI on governed data. Part 2 of this series continues with the features and workflows. The Databricks SQL overview describes the query interfaces, monitoring tools and getting-started resources for SQL analytics. Use it as the reference for the topics covered in this part of the series. Official docs: Introduces Databricks SQL, the lakehouse data warehouse for queries, monitoring and BI. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/sql/
🔗 More: https://www.nextgenlakehouse.com/answers/sql-analytics

#Databricks #DatabricksSQL #DataWarehouse #SQL #Analytics #BI #Lakehouse

— Pinned comment —
Part 2 of the Databricks SQL series. The SQL overview and our SQL analytics hub are linked below. Docs: https://docs.databricks.com/aws/en/sql/ Hub: https://www.nextgenlakehouse.com/answers/sql-analytics

Getting the Most Out of AI/BI Dashboards with Databricks One and UC Metrics

This video shows how to get more from AI/BI dashboards using Databricks One and Unity Catalog metric views. Metric views define a business metric once for reuse across dashboards. Unity Catalog metric views let you define a metric once and group or filter it by any field, which is the semantic layer used by dashboards. The metric views page linked below is the exact reference for defining them. Official docs: Explains Unity Catalog metric views, which define a business metric once and group or filter it by any field. Introduces AI/BI dashboards, with guides for authoring, sharing, automating and monitoring them.

📚 Docs: https://docs.databricks.com/aws/en/metric-views/ https://docs.databricks.com/aws/en/dashboards/
🔗 More: https://www.nextgenlakehouse.com/answers/sql-analytics

#Databricks #AIBI #UCMetrics #MetricViews #SemanticLayer #Dashboards #UnityCatalog

— Pinned comment —
Dashboards built on UC metric views are covered here. The metric views docs and our Unity Catalog hub are linked below. Docs: https://docs.databricks.com/aws/en/metric-views/ Hub: https://www.nextgenlakehouse.com/answers/sql-analytics

AI/BI Dashboard Caching: 100K Rows, Filters vs. Parameters

This video compares filters and parameters in AI/BI dashboards and shows how caching behaves with large datasets of around 100K rows. The author dashboards page is the closest reference for building dashboard filters. The author dashboards page covers creating and organizing dashboard pages and filter widgets. Caching behavior is not covered on the linked page, so check the dashboards overview for performance notes. Official docs: Covers creating, organizing and editing AI/BI dashboard pages, plus bookmarks and filter widgets. Introduces AI/BI dashboards, with guides for authoring, sharing, automating and monitoring them.

📚 Docs: https://docs.databricks.com/aws/en/dashboards/manage/ https://docs.databricks.com/aws/en/dashboards/
🔗 More: https://www.nextgenlakehouse.com/answers/sql-analytics

#Databricks #AIBI #Dashboards #Performance #Caching #DatabricksSQL #BI

— Pinned comment —
Filters, parameters and caching in AI/BI dashboards are compared here. The dashboard docs and our SQL analytics hub are linked below. Docs: https://docs.databricks.com/aws/en/dashboards/manage/ Hub: https://www.nextgenlakehouse.com/answers/sql-analytics

Collations: Case-Insensitive SQL and Legacy Data Warehouse Migration

Collations in Databricks SQL control how strings compare and sort, including case-insensitive matching. This video shows how they help when migrating from a legacy data warehouse. The collation reference explains collation names, case, accent and trailing-space sensitivity, plus precedence and derivation rules. Use it to choose the right collation when porting SQL from a legacy warehouse. Official docs: Explains collations for string comparison, case and accent sensitivity, precedence and derivation. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-collation
🔗 More: https://www.nextgenlakehouse.com/answers/sql-analytics

#Databricks #DatabricksSQL #Collations #SQL #DataMigration #DataWarehouse #SQLTips

— Pinned comment —
Collations for case-insensitive SQL and migrations are covered here. The collation docs and our SQL analytics hub are linked below. Docs: https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-collation Hub: https://www.nextgenlakehouse.com/answers/sql-analytics

Spatial SQL in Databricks: ST_Point, ST_Contains and ST_Distance

Spatial SQL in Databricks uses geometry functions such as ST_Point, ST_Contains and ST_Distance on location data. This video shows how to run them in SQL. The st_point reference documents the function that builds a point geometry from X and Y coordinates with an optional SRID. Use the spatial functions reference alongside it for ST_Contains and ST_Distance. Official docs: Documents st_point, which builds a point geometry from X and Y coordinates with an optional SRID. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/sql/language-manual/functions/st_point
🔗 More: https://www.nextgenlakehouse.com/answers/sql-analytics

#Databricks #SpatialSQL #GeoSpatial #SQL #ST_Point #DatabricksSQL #Analytics

— Pinned comment —
Spatial SQL with ST_Point, ST_Contains and ST_Distance is covered here. The st_point docs and our SQL analytics hub are linked below. Docs: https://docs.databricks.com/aws/en/sql/language-manual/functions/st_point Hub: https://www.nextgenlakehouse.com/answers/sql-analytics

All You Need to Know About Databricks SQL (Part 1)

Databricks SQL is the lakehouse data warehouse for analytics and BI on governed data. Part 1 of this series introduces the core features and how to get started. The Databricks SQL overview describes the query interfaces, monitoring tools and getting-started resources for SQL analytics. Use it as the reference for the basics covered in this part of the series. Official docs: Introduces Databricks SQL, the lakehouse data warehouse for queries, monitoring and BI. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/sql/
🔗 More: https://www.nextgenlakehouse.com/answers/sql-analytics

#Databricks #DatabricksSQL #DataWarehouse #SQL #Analytics #BI #Lakehouse

— Pinned comment —
Part 1 of the Databricks SQL series. The SQL overview and our SQL analytics hub are linked below. Docs: https://docs.databricks.com/aws/en/sql/ Hub: https://www.nextgenlakehouse.com/answers/sql-analytics

Getting Started with Recursive CTEs in Databricks SQL

Recursive common table expressions (CTEs) in Databricks SQL let a query reference itself to walk hierarchies such as org charts or graphs. This video shows the syntax with examples. The CTE reference documents the WITH clause, including recursive queries, its parameters, limitations and error conditions. Use it for the exact recursive syntax used in the video. Official docs: Documents the WITH clause and recursive common table expressions, with limitations and examples. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-syntax-qry-select-cte
🔗 More: https://www.nextgenlakehouse.com/answers/sql-analytics

#Databricks #DatabricksSQL #RecursiveCTE #SQL #CTE #SQLTips #DataEngineering

— Pinned comment —
Recursive CTEs in Databricks SQL are covered here. The CTE docs and our SQL analytics hub are linked below. Docs: https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-syntax-qry-select-cte Hub: https://www.nextgenlakehouse.com/answers/sql-analytics

Getting Started with Stored Procedures in Databricks SQL

Stored procedures in Databricks SQL are Unity Catalog routines you define once and call from SQL. This video shows how to create and call one. The CREATE PROCEDURE reference covers defining a Unity Catalog stored procedure with SQL, its parameters, security modes and examples. Use it as the exact syntax reference for the video. Official docs: Documents CREATE PROCEDURE for defining Unity Catalog stored procedures in SQL. Watch the full walkthrough on the NextGenLakehouse channel. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-syntax-ddl-create-procedure
🔗 More: https://www.nextgenlakehouse.com/answers/sql-analytics

#Databricks #DatabricksSQL #StoredProcedures #SQL #UnityCatalog #SQLTips #DataEngineering

— Pinned comment —
Stored procedures in Databricks SQL are covered here. The CREATE PROCEDURE docs and our SQL analytics hub are linked below. Docs: https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-syntax-ddl-create-procedure Hub: https://www.nextgenlakehouse.com/answers/sql-analytics

AI/BI and Databricks One Demo from the Data+AI Summit 2025

This demo shows AI/BI dashboards and Databricks One from the Data and AI Summit 2025. The dashboards overview is the reference for building and sharing them. AI/BI dashboards let you author, share and automate visualizations on governed data. Databricks One brings them to business users, and the linked dashboards overview covers the authoring workflow shown in the demo. Official docs: Introduces AI/BI dashboards, with guides for authoring, sharing, automating and monitoring them. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/dashboards/
🔗 More: https://www.nextgenlakehouse.com/answers/sql-analytics

#Databricks #AIBI #DatabricksOne #Dashboards #DAIS2025 #BusinessIntelligence #DataAI

— Pinned comment —
AI/BI and Databricks One from DAIS 2025. The dashboards docs and our SQL analytics hub are linked below. Docs: https://docs.databricks.com/aws/en/dashboards/ Hub: https://www.nextgenlakehouse.com/answers/sql-analytics

What's New in AI/BI: Data and AI Summit 2025 Edition

This video covers what is new in AI/BI at the Data and AI Summit 2025. AI/BI dashboards and Genie are the core experiences it describes. AI/BI dashboards let you author, share and automate visualizations, and Genie answers natural-language questions over governed data. The dashboards overview linked below covers the authoring features. Official docs: Introduces AI/BI dashboards, with guides for authoring, sharing, automating and monitoring them. Describes Genie, the AI experience for asking governed data questions in natural language.

📚 Docs: https://docs.databricks.com/aws/en/dashboards/ https://docs.databricks.com/aws/en/genie/
🔗 More: https://www.nextgenlakehouse.com/answers/sql-analytics

#Databricks #AIBI #Genie #Dashboards #DAIS2025 #BusinessIntelligence #DataAI

— Pinned comment —
What is new in AI/BI from DAIS 2025. The dashboards docs and our SQL analytics hub are linked below. Docs: https://docs.databricks.com/aws/en/dashboards/ Hub: https://www.nextgenlakehouse.com/answers/sql-analytics

Getting Started with UC Metrics: Databricks Semantic Layer

Unity Catalog metric views are the Databricks semantic layer, where you define a business metric once and reuse it. This video shows how to create and query them. The metric views page explains how to set up a business metric once and group or filter it by any field, with guides to creating, querying and managing metric views. It is the exact reference for the semantic layer covered here. Official docs: Explains Unity Catalog metric views, which define a business metric once and group or filter it by any field.

📚 Docs: https://docs.databricks.com/aws/en/metric-views/
🔗 More: https://www.nextgenlakehouse.com/answers/sql-analytics

#Databricks #UCMetrics #MetricViews #SemanticLayer #UnityCatalog #BusinessIntelligence #DatabricksSQL

— Pinned comment —
The Databricks semantic layer with UC metric views is the topic. The metric views docs and our SQL analytics hub are linked below. Docs: https://docs.databricks.com/aws/en/metric-views/ Hub: https://www.nextgenlakehouse.com/answers/sql-analytics

Getting Started with Databricks SQL Scripting

SQL scripting on Databricks adds procedural logic to SQL, with compound blocks, variables, cursors, condition handling and control flow. This video introduces the syntax. The SQL scripting page explains compound blocks, variables, cursors, error handling and control flow, plus stored procedures. Use it as the exact syntax reference for the scripts in the video. Official docs: Covers SQL scripting: compound blocks, variables, cursors, condition handling and control flow. Watch the full walkthrough on the NextGenLakehouse channel. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-scripting
🔗 More: https://www.nextgenlakehouse.com/answers/sql-analytics

#Databricks #DatabricksSQL #SQLScripting #SQL #Procedural #SQLTips #DataEngineering

— Pinned comment —
SQL scripting in Databricks is covered here. The SQL scripting docs and our SQL analytics hub are linked below. Docs: https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-scripting Hub: https://www.nextgenlakehouse.com/answers/sql-analytics

Databricks SQL Cost Control: The Complete Step-by-Step Guide

SQL warehouses drive most Databricks SQL cost, so controlling warehouse size and behavior is the main cost lever. This guide walks through the steps. The SQL warehouse page explains serverless and pro warehouses, how to start and create them, and how to connect tools. The billable usage page documents the system table you use to track spend. Official docs: Explains how to create, start and connect SQL warehouses and third-party BI tools. Documents system.billing.usage, the account-wide billable usage table with record types and billing origin products.

📚 Docs: https://docs.databricks.com/aws/en/compute/sql-warehouse/ https://docs.databricks.com/aws/en/admin/system-tables/billing
🔗 More: https://www.nextgenlakehouse.com/answers/sql-analytics

#Databricks #DatabricksSQL #CostControl #FinOps #SQLWarehouse #Serverless #CloudCost

— Pinned comment —
Controlling Databricks SQL cost step by step is covered here. The SQL warehouse and billing docs and our SQL analytics hub are linked below. Docs: https://docs.databricks.com/aws/en/compute/sql-warehouse/ Hub: https://www.nextgenlakehouse.com/answers/sql-analytics

Mastering the DBSQL Warehouse Advisor Dashboard: A Comprehensive Guide

The DBSQL Warehouse Advisor dashboard helps you size and tune SQL warehouses from usage data. This guide walks through the dashboard and what each view tells you. The SQL warehouse page covers serverless and pro warehouses, how to start and create them, and how to connect tools. The system tables overview lists the usage data the advisor dashboard reads. A dedicated advisor page was not found. Official docs: Explains how to create, start and connect SQL warehouses and third-party BI tools. Overview of the system catalog that stores operational data for cost, security and compute monitoring.

📚 Docs: https://docs.databricks.com/aws/en/compute/sql-warehouse/ https://docs.databricks.com/aws/en/admin/system-tables/
🔗 More: https://www.nextgenlakehouse.com/answers/sql-analytics

#Databricks #DatabricksSQL #SQLWarehouse #WarehouseAdvisor #Performance #Dashboards #FinOps

— Pinned comment —
The DBSQL Warehouse Advisor dashboard is covered here. The SQL warehouse docs and our SQL analytics hub are linked below. Docs: https://docs.databricks.com/aws/en/compute/sql-warehouse/ Hub: https://www.nextgenlakehouse.com/answers/sql-analytics

AI/BI Dashboards Ultimate Guide: Build, Share and Govern Dashboards on Databricks

AI/BI Dashboards let you build interactive dashboards on Unity Catalog tables with no separate BI server. This guide covers building, sharing and governing them. Dashboard pages run SQL datasets on a SQL warehouse. The dashboard tutorials walk through creating your first dashboard step by step. Plan datasets, page layout and filters before building, which keeps large dashboards maintainable as they grow. Access to the underlying tables still follows Unity Catalog permissions. 📚 Docs: https://docs.databricks.com/aws/en/dashboards and https://docs.databricks.com/aws/en/dashboards/tutorials/. 🔗 More: https://www.nextgenlakehouse.com/learn (SQL & Analytics hub). #Databricks #AIBI #Dashboards #BI #DatabricksSQL #SQL #Analytics #DataAnalytics

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/dashboards More SQL & Analytics content lives at https://www.nextgenlakehouse.com/learn

Databricks SQL: From $0 to $400M in 2.5 Years with Reynold Xin

Databricks SQL went from $0 to $400M in about two and a half years, per the video title, and this conversation with Reynold Xin covers how. It covers how the SQL warehouse product evolved. SQL warehouses are the compute that runs Databricks SQL queries, dashboards and alerts. The Databricks SQL get-started guide covers creating a warehouse and running your first query. The growth story is useful context, but for sizing a SQL warehouse for your own workload, start with the warehouse types and serverless option in the docs. 📚 Docs: https://docs.databricks.com/aws/en/sql/get-started and https://docs.databricks.com/aws/en/compute/sql-warehouse/. 🔗 More: https://www.nextgenlakehouse.com/learn (SQL & Analytics hub). #Databricks #DatabricksSQL #SQLWarehouse #DataWarehouse #SQL #Analytics #DataAnalytics #DataEngineering

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/sql/get-started More SQL & Analytics content lives at https://www.nextgenlakehouse.com/learn

The Self-Service Analytics Tool That Saves You Hours on Databricks

This video presents a self-service tool for working with data on Databricks, with the goal of saving analysts hours. The tool is not named in the title, so this summary stays general. Self-service analytics on Databricks is covered by Genie and AI/BI Dashboards, and the Genie overview is the reference for natural-language data exploration. The title does not name the tool, so check the video chapters for the specific product demonstrated and use the matching doc linked in the description. 📚 Docs: https://docs.databricks.com/aws/en/genie/. 🔗 More: https://www.nextgenlakehouse.com/learn (SQL & Analytics hub). #Databricks #SelfService #Analytics #AIBI #DatabricksSQL #SQL #DataAnalytics #DataEngineering

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/genie/ More SQL & Analytics content lives at https://www.nextgenlakehouse.com/learn

Power BI and Databricks SQL: Best Practices for Fast Dashboards

Connecting Power BI to Databricks SQL works best with a right-sized SQL warehouse and deliberate DirectQuery or Import choices. This video covers the main best practices. The Power BI integration doc describes the connector and authentication options for Databricks SQL warehouses. Import mode is faster for small models, while DirectQuery keeps data live against the warehouse, so choose per dashboard based on freshness and volume. Size the warehouse for concurrent report users, not only the largest refresh. 📚 Docs: https://docs.databricks.com/aws/en/partners/bi/power-bi and https://docs.databricks.com/aws/en/compute/sql-warehouse/. 🔗 More: https://www.nextgenlakehouse.com/learn (SQL & Analytics hub). #Databricks #PowerBI #BI #Performance #DatabricksSQL #SQL #Analytics #DataAnalytics

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/partners/bi/power-bi More SQL & Analytics content lives at https://www.nextgenlakehouse.com/learn

What's New in Databricks AI/BI at Data+AI Summit 2024 with Chao Cai and Miranda Luna

This Data+AI Summit 2024 session covers what is new in Databricks AI/BI, including dashboards and natural-language analytics. Chao Cai and Miranda Luna present the updates. AI/BI Dashboards let you build dashboards on Unity Catalog data, and the dashboards doc describes the authoring features. Teams evaluating the update should note which features the current docs mark as preview or generally available. Use the dashboard tutorials to rebuild one of the demo dashboards in your own workspace. 📚 Docs: https://docs.databricks.com/aws/en/dashboards. 🔗 More: https://www.nextgenlakehouse.com/learn (SQL & Analytics hub). #Databricks #AIBI #Dashboards #DataAISummit #DatabricksSQL #SQL #Analytics #DataAnalytics

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/dashboards More SQL & Analytics content lives at https://www.nextgenlakehouse.com/learn

Introduction to Databricks SQL: Warehouses, Queries and Dashboards

Databricks SQL lets analysts run SQL on SQL warehouses against Unity Catalog tables and build dashboards on the results. This introduction covers the core objects. Start by creating a SQL warehouse, then run a query in the SQL editor and save it as a dashboard source. The get-started doc walks through each step. Once the warehouse and first query work, the next step is saving results into a dashboard. Start with a small warehouse and scale it only if queries are slow under real load. 📚 Docs: https://docs.databricks.com/aws/en/sql/get-started and https://docs.databricks.com/aws/en/compute/sql-warehouse/. 🔗 More: https://www.nextgenlakehouse.com/learn (SQL & Analytics hub). #Databricks #DatabricksSQL #SQL #Analytics #DataAnalytics #DataEngineering #Lakehouse #AI

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/sql/get-started More SQL & Analytics content lives at https://www.nextgenlakehouse.com/learn

Databricks and AtScale Beginner's Guide: Semantic Layer for BI

AtScale is a semantic layer that gives BI tools consistent metrics on top of Databricks. This beginner's guide explains how to connect them. The Databricks partner directory lists AtScale, and metric views in Unity Catalog cover a similar governed-metrics use case. Semantic layers help BI tools agree on metric definitions, which matters when several dashboards report the same KPI. Compare the partner setup with Unity Catalog metric views to decide where definitions should live. 📚 Docs: https://www.databricks.com/partners and https://docs.databricks.com/aws/en/uc-semantics/metric-views. 🔗 More: https://www.nextgenlakehouse.com/learn (SQL & Analytics hub). #Databricks #AtScale #SemanticLayer #BI #DatabricksSQL #SQL #Analytics #DataAnalytics

— Pinned comment —
Official reference: https://www.databricks.com/partners More SQL & Analytics content lives at https://www.nextgenlakehouse.com/learn

The Future of Data Warehousing with Databricks SQL and the Lakehouse

Data warehousing on Databricks combines SQL warehouses with open Delta tables in the lakehouse. This video covers where that model is heading. The Databricks SQL docs describe the warehouse types, query engine and governance model used for analytics. The lakehouse model suits teams that want one copy of data for BI, SQL and AI workloads. Check the warehouse docs for sizing and serverless options, and note that governance applies through Unity Catalog across all these uses. 📚 Docs: https://docs.databricks.com/aws/en/compute/sql-warehouse/ and https://docs.databricks.com/aws/en/sql/get-started. 🔗 More: https://www.nextgenlakehouse.com/learn (SQL & Analytics hub). #Databricks #DataWarehouse #DatabricksSQL #SQL #Analytics #DataAnalytics #DataEngineering #Lakehouse

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/compute/sql-warehouse/ More SQL & Analytics content lives at https://www.nextgenlakehouse.com/learn

AI/BI Dashboards Rebranding Explained: What Changed from Lakeview

AI/BI Dashboards is the current name for the dashboard product formerly called Lakeview, and this video explains what changed. The rename brings AI-assisted authoring and governance into one product. The dashboards doc describes AI-assisted authoring, the visual editor and sharing controls. Check the dashboards doc for current features, since the product has changed since the rename. Use the tutorials to rebuild one existing dashboard as a test. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://docs.databricks.com/aws/en/dashboards. 🔗 More: https://www.nextgenlakehouse.com/learn (SQL & Analytics hub). #Databricks #AIBI #Dashboards #Lakeview #DatabricksSQL #SQL #Analytics #DataAnalytics

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/dashboards More SQL & Analytics content lives at https://www.nextgenlakehouse.com/learn

Databricks SQL Execution API Explained: Run SQL Over REST

The Databricks SQL Statement Execution API runs SQL statements on a SQL warehouse over REST and returns results. This video explains how to use it. The Statement Execution API tutorial shows how to submit a statement and poll for results, and the API reference lists the parameters. Use the API for applications and scripts that need SQL results without a driver, and the SQL editor for ad hoc work. Check the polling and result size limits in the reference before building on it. 📚 Docs: https://docs.databricks.com/aws/en/dev-tools/sql-execution-tutorial and https://docs.databricks.com/api/statement-execution/v1/execute-statement. 🔗 More: https://www.nextgenlakehouse.com/learn (SQL & Analytics hub). #Databricks #SQLAPI #RESTAPI #DatabricksSQL #SQL #Analytics #DataAnalytics #DataEngineering

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/dev-tools/sql-execution-tutorial More SQL & Analytics content lives at https://www.nextgenlakehouse.com/learn

ThoughtSpot and Databricks: The Future of Analytics on the Lakehouse

ThoughtSpot connects search-driven analytics to Databricks data so business users can ask questions directly. This video explores the future of analytics with the two platforms. The ThoughtSpot connection doc covers connecting ThoughtSpot to a Databricks SQL warehouse. ThoughtSpot works best when the Databricks tables are modelled clearly for business questions. Connect it to a SQL warehouse, govern access through Unity Catalog, and check the connection doc for authentication options. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://docs.databricks.com/aws/en/partners/bi/thoughtspot. 🔗 More: https://www.nextgenlakehouse.com/learn (SQL & Analytics hub). #Databricks #ThoughtSpot #BI #Analytics #DatabricksSQL #SQL #DataAnalytics #DataEngineering

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/partners/bi/thoughtspot More SQL & Analytics content lives at https://www.nextgenlakehouse.com/learn

Getting Started with AI/BI Dashboards (Formerly Lakeview) on Databricks

AI/BI Dashboards, formerly Lakeview, let you build dashboards on Unity Catalog data with a visual editor. This video shows how to get started. The dashboard tutorials walk through creating datasets, adding widgets and publishing a dashboard. Dashboards built here use Unity Catalog data and SQL warehouses, so the same permissions apply. Use the tutorial to create datasets, widgets and a published dashboard step by step, then check the sharing options. 📚 Docs: https://docs.databricks.com/aws/en/dashboards and https://docs.databricks.com/aws/en/dashboards/tutorials/. 🔗 More: https://www.nextgenlakehouse.com/learn (SQL & Analytics hub). #Databricks #AIBI #Lakeview #Dashboards #DatabricksSQL #SQL #Analytics #DataAnalytics

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/dashboards More SQL & Analytics content lives at https://www.nextgenlakehouse.com/learn

Power BI and Databricks: Connect, Model and Visualize the Lakehouse

Power BI connects to Databricks SQL warehouses so analysts can model and visualise lakehouse data. This video shows how the connection and modelling work together. The Power BI integration doc describes the connector and the authentication options for Databricks. The Power BI connector lets analysts build models on Databricks SQL data using familiar tools. Check the integration doc for connection and authentication options, and size the SQL warehouse for report refresh. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://docs.databricks.com/aws/en/partners/bi/power-bi. 🔗 More: https://www.nextgenlakehouse.com/learn (SQL & Analytics hub). #Databricks #PowerBI #Microsoft #BI #DatabricksSQL #SQL #Analytics #DataAnalytics

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/partners/bi/power-bi More SQL & Analytics content lives at https://www.nextgenlakehouse.com/learn

Databricks Photon Engine Explained: Faster SQL and Spark Queries

What is Databricks Photon? Photon is a vectorized query engine written in C++ that Databricks uses to speed up SQL and Spark DataFrame workloads on supported compute. This video explains what it does and why it speeds up queries. The Photon documentation covers supported compute, how to enable it, and which workloads benefit. Confirm the runtime and cluster settings in your workspace before you benchmark. Run your own benchmark with and without Photon on a representative query before you change production settings. 📚 Docs: https://docs.databricks.com/aws/en/compute/photon https://www.databricks.com/product/photon 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #Photon #SQL #Spark #QueryPerformance #Lakehouse

— Pinned comment —
Official reference for this topic: https://docs.databricks.com/aws/en/compute/photon More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Databricks SQL vs the Market: Warehouse Comparison for Analytics

How does Databricks SQL compare with other data warehouses? This short video positions Databricks SQL, the serverless SQL warehouse layer of the lakehouse, against alternatives on the market. Warehouse comparisons depend on price, concurrency and workload shape, so benchmark with your own queries. The Databricks SQL documentation describes warehouses, query editing and dashboards. Benchmark your own dashboards and queries on both platforms, because warehouse results depend on data layout and concurrency. Look at query latency, concurrency and cost per query on your own dashboards before you commit to a warehouse choice for a team. 📚 Docs: https://docs.databricks.com/aws/en/sql/ 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #DatabricksSQL #DataWarehouse #SQL #Analytics #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://docs.databricks.com/aws/en/sql/ More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Understanding Datasets in AI/BI Dashboards on Databricks

What is a dataset in AI/BI? This video explains the datasets that power Databricks AI/BI dashboards. A dataset is a SQL query over Unity Catalog tables or views, and every dashboard widget reads from one. Understanding how datasets are defined, parameterised and shared helps you keep dashboards fast and consistent with the Genie space that uses the same data. Reuse datasets across widgets instead of duplicating SQL, which keeps the dashboard and the Genie space consistent. When you clean up a dashboard, check that each dataset has a clear name and a documented purpose so colleagues can reuse it safely. 📚 Docs: https://docs.databricks.com/aws/en/dashboards/manage/data-modeling/datasets https://docs.databricks.com/aws/en/dashboards/ 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #AIBI #Dashboards #SQL #UnityCatalog #DataModeling

— Pinned comment —
Official reference for this topic: https://docs.databricks.com/aws/en/dashboards/manage/data-modeling/datasets More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Semantic Layer Migration Podcast: Moving Metrics to Databricks

How do you migrate a semantic layer? This podcast discusses moving business metrics from an existing semantic layer into a new one. Databricks metric views define measures and dimensions in Unity Catalog so BI tools and Genie can query the same definitions. The metric views tutorial walks through building one step by step. Define each metric once as a metric view so BI tools and Genie return the same numbers. Metric migrations go smoother when you list every dashboard that uses a metric and map each one to its new definition before you cut over. 📚 Docs: https://docs.databricks.com/aws/en/uc-semantics/metric-views/tpch-example 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #SemanticLayer #MetricViews #UnityCatalog #BI #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://docs.databricks.com/aws/en/uc-semantics/metric-views/tpch-example More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Is Data Warehousing a Solved Problem? A Lakehouse Perspective

Is data warehousing a solved problem? This video argues that analytics and warehousing still face open challenges around cost, openness and AI workloads. The lakehouse approach brings warehousing and data engineering onto one platform. The Databricks SQL documentation covers the warehouse layer that serves analytics on lakehouse data. Test the lakehouse against your current warehouse on cost, concurrency and AI workloads before you decide. Warehousing questions often come down to concurrency, governance and openness, so test those three on your own data before you decide. 📚 Docs: https://docs.databricks.com/aws/en/sql/ https://www.databricks.com/product/databricks-lakehouse 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #DataWarehouse #Lakehouse #DatabricksSQL #Analytics #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://docs.databricks.com/aws/en/sql/ More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Feature Engineering on Unity Catalog (Feature Store): Getting Started

Feature Engineering in Unity Catalog stores feature tables as governed assets that models can reuse. This video shows how to get started. The Unity Catalog feature tables doc explains creating feature tables and using them for training and serving. Feature tables in Unity Catalog share the same governance as other tables, so teams can reuse features safely. Check the feature table doc for table requirements before creating one. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://docs.databricks.com/aws/en/machine-learning/feature-store/uc/feature-tables-uc and https://docs.databricks.com/aws/en/machine-learning/feature-store/. 🔗 More: https://www.nextgenlakehouse.com/learn (AI & ML hub). #Databricks #FeatureStore #MachineLearning #MLOps #AI #DataAnalytics #DataEngineering #Lakehouse

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/machine-learning/feature-store/uc/feature-tables-uc More AI & ML content lives at https://www.nextgenlakehouse.com/learn

Databricks Lakewatch: Agentic AI for Your SOC (Security Operations)

Lakewatch applies agentic AI to security operations on Databricks, for teams running a SOC. The video introduces the product and the agent approach behind it. It covers how AI agents can support security operations work on lakehouse data, with the emphasis on analyst workflows rather than dashboards. The linked Agent Bricks page is the closest verified official reference for building and governing agents on Databricks. Official docs: Introduces Agent Bricks for building, deploying, governing and monitoring agents on Databricks.

📚 Docs: https://docs.databricks.com/aws/en/generative-ai/agent-bricks/
🔗 More: https://www.nextgenlakehouse.com/answers/genai

#Databricks #Lakewatch #AgenticAI #SOC #CyberSecurity #AgentBricks #GenAI

— Pinned comment —
Lakewatch is Databricks' agentic-AI use case for security operations. Docs and the answers hub are linked below. Docs: https://docs.databricks.com/aws/en/generative-ai/agent-bricks/ Hub: https://www.nextgenlakehouse.com/answers/genai

Omnigent: Multi-Agent Orchestration and Real-Time Collaboration on Databricks

Omnigent covers multi-agent orchestration and real-time collaboration for AI agents. The video explains the approach; the Agent Bricks docs are the closest official reference for agent building on Databricks. Multi-agent systems split a task across specialized agents that coordinate with each other. The Agent Bricks page linked below documents how Databricks builds, deploys and governs agents, which is the platform context for orchestration. Official docs: Introduces Agent Bricks for building, deploying, governing and monitoring agents on Databricks. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/generative-ai/agent-bricks/
🔗 More: https://www.nextgenlakehouse.com/answers/genai

#Databricks #MultiAgent #AIAgents #AgentBricks #GenAI #Orchestration #Collaboration

— Pinned comment —
This video covers multi-agent orchestration. The Agent Bricks docs and our GenAI hub are linked below. Docs: https://docs.databricks.com/aws/en/generative-ai/agent-bricks/ Hub: https://www.nextgenlakehouse.com/answers/genai

Extract and Structure Enterprise Documents with AI Functions on Databricks

AI Functions let you extract structured data from documents and text directly in SQL on Databricks. This video shows how to pull enterprise document content into structured tables. The ai_extract function pulls structured data from text and documents according to a schema you define, including nested objects and arrays. The AI Functions overview linked below covers the broader set of task-specific and general-purpose functions. Official docs: Documents ai_extract, which pulls structured data from text and documents according to a user-defined schema. Covers AI Functions that apply LLMs to data in SQL, with task-specific and general-purpose options.

📚 Docs: https://docs.databricks.com/aws/en/sql/language-manual/functions/ai_extract https://docs.databricks.com/aws/en/large-language-models/ai-functions
🔗 More: https://www.nextgenlakehouse.com/answers/genai

#Databricks #AIFunctions #ai_extract #DocumentAI #DatabricksSQL #GenAI #DataExtraction

— Pinned comment —
AI Functions extract structure from enterprise documents in SQL. The ai_extract docs and our GenAI hub are linked below. Docs: https://docs.databricks.com/aws/en/sql/language-manual/functions/ai_extract Hub: https://www.nextgenlakehouse.com/answers/genai

Build No-Code AI Agents with Kasal on Databricks

Kasal is a no-code way to build AI agents on Databricks, shown here as a Databricks App. This video shows how to create an agent without writing code. Agent Bricks is the Databricks platform for building and deploying agents, and the page linked below documents the agent development flow. Kasal is a community-style builder, so use the linked page for the official agent platform reference. Official docs: Introduces Agent Bricks for building, deploying, governing and monitoring agents on Databricks.

📚 Docs: https://docs.databricks.com/aws/en/generative-ai/agent-bricks/
🔗 More: https://www.nextgenlakehouse.com/answers/genai

#Databricks #Kasal #NoCode #AIAgents #DatabricksApps #GenAI #AgentBricks

— Pinned comment —
No-code agent building on Databricks is the focus. The Agent Bricks docs and our GenAI hub are linked below. Docs: https://docs.databricks.com/aws/en/generative-ai/agent-bricks/ Hub: https://www.nextgenlakehouse.com/answers/genai

Getting Started with Kasal: Genie and Perplexity Integration

This video shows how to connect Kasal, a no-code agent builder, with Genie and Perplexity on Databricks. It is a getting-started walkthrough for the integration. The Genie setup guide covers creating and managing a Genie Agent that business users can query in plain language, which is the Genie side of the integration. The Agent Bricks page covers the official agent platform. Official docs: Explains how to build, share and administer a Genie Agent for business users. Introduces Agent Bricks for building, deploying, governing and monitoring agents on Databricks.

📚 Docs: https://docs.databricks.com/aws/en/genie/set-up https://docs.databricks.com/aws/en/generative-ai/agent-bricks/
🔗 More: https://www.nextgenlakehouse.com/answers/genai

#Databricks #Kasal #Genie #Perplexity #AIAgents #GenAI #NoCode

— Pinned comment —
Kasal integrated with Genie and Perplexity is the walkthrough. The Genie setup docs and our GenAI hub are linked below. Docs: https://docs.databricks.com/aws/en/genie/set-up Hub: https://www.nextgenlakehouse.com/answers/genai

Getting Started with Kasal: Low-Code Way to Build an Agent in Databricks

Kasal is a low-code way to build an AI agent on Databricks. This getting-started video shows the first agent from start to finish. The Agent Bricks page documents the Databricks platform for building and deploying agents, including the development flow and governance. Use it as the official reference for the agent concepts used in the walkthrough. Official docs: Introduces Agent Bricks for building, deploying, governing and monitoring agents on Databricks. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/generative-ai/agent-bricks/
🔗 More: https://www.nextgenlakehouse.com/answers/genai

#Databricks #Kasal #LowCode #AIAgents #AgentBricks #GenAI #DatabricksApps

— Pinned comment —
Low-code agent building with Kasal is the walkthrough. The Agent Bricks docs and our GenAI hub are linked below. Docs: https://docs.databricks.com/aws/en/generative-ai/agent-bricks/ Hub: https://www.nextgenlakehouse.com/answers/genai

Building the Future of AI: Classic ML to GenAI with Patrick Wendell

Patrick Wendell, Databricks co-founder, discusses the path from classic machine learning to generative AI. This is a conversation rather than a how-to, so the link points to the Agent Bricks docs as the closest product reference. The discussion covers how AI workloads on the lakehouse evolved from classic ML to generative AI and agents. The Agent Bricks page linked below is the official reference for building generative AI and agent applications on Databricks. Official docs: Introduces Agent Bricks for building, deploying, governing and monitoring agents on Databricks.

📚 Docs: https://docs.databricks.com/aws/en/generative-ai/agent-bricks/
🔗 More: https://www.nextgenlakehouse.com/answers/genai

#Databricks #GenAI #MachineLearning #AI #DataAndAI #AgentBricks #Leadership

— Pinned comment —
Classic ML to GenAI, with Databricks co-founder Patrick Wendell. The Agent Bricks docs and our GenAI hub are linked below. Docs: https://docs.databricks.com/aws/en/generative-ai/agent-bricks/ Hub: https://www.nextgenlakehouse.com/answers/genai

Getting Started with Chuck Data from Amperity

Chuck Data from Amperity is a data agent that runs on Databricks. This video shows how to start with it; the Agent Bricks page is the closest official reference for agents on the platform. Agent Bricks documents how Databricks builds, deploys and governs agents, which is the platform context for a data agent like Chuck Data. Check the partner's own documentation for product-specific setup. Official docs: Introduces Agent Bricks for building, deploying, governing and monitoring agents on Databricks. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/generative-ai/agent-bricks/
🔗 More: https://www.nextgenlakehouse.com/answers/genai

#Databricks #ChuckData #Amperity #AIAgents #GenAI #DataAgents #AgentBricks

— Pinned comment —
Getting started with Chuck Data from Amperity. The Agent Bricks docs and our GenAI hub are linked below. Docs: https://docs.databricks.com/aws/en/generative-ai/agent-bricks/ Hub: https://www.nextgenlakehouse.com/answers/genai

The Truth About Product Management and AI's Future with David Meyer

David Meyer, Databricks SVP of product, discusses product management and the future of AI. It is a leadership conversation, so the link points to the Agent Bricks docs as the closest product reference. The conversation covers how product teams think about AI and agents. The Agent Bricks page linked below is the official reference for the agent platform that many of these product choices concern. Official docs: Introduces Agent Bricks for building, deploying, governing and monitoring agents on Databricks.

📚 Docs: https://docs.databricks.com/aws/en/generative-ai/agent-bricks/
🔗 More: https://www.nextgenlakehouse.com/answers/genai

#Databricks #ProductManagement #AI #GenAI #DataAndAI #AgentBricks #Leadership

— Pinned comment —
Product management and the future of AI with David Meyer. The Agent Bricks docs and our GenAI hub are linked below. Docs: https://docs.databricks.com/aws/en/generative-ai/agent-bricks/ Hub: https://www.nextgenlakehouse.com/answers/genai

Agent Bricks Demo from the Data+AI Summit 2025

This demo shows Agent Bricks, the Databricks platform for building and deploying AI agents, from the Data and AI Summit 2025. The Agent Bricks docs are the reference for the workflow. Agent Bricks covers developing, deploying and monitoring agents on Databricks, with governance and observability. The Agent Bricks page linked below is the official documentation for the workflow shown in the demo. Official docs: Introduces Agent Bricks for building, deploying, governing and monitoring agents on Databricks. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/generative-ai/agent-bricks/
🔗 More: https://www.nextgenlakehouse.com/answers/genai

#Databricks #AgentBricks #AIAgents #GenAI #DAIS2025 #AIApps #DataAI

— Pinned comment —
Agent Bricks demo from DAIS 2025. The Agent Bricks docs and our GenAI hub are linked below. Docs: https://docs.databricks.com/aws/en/generative-ai/agent-bricks/ Hub: https://www.nextgenlakehouse.com/answers/genai

Unlocking AI for Everyone: Insights from Youssef Mrini at Data+AI Summit

Youssef Mrini, Databricks Senior Developer Advocate, shares how AI is becoming accessible to everyone on the platform. It is a talk rather than a feature how-to, so the link points to the Agent Bricks docs. The talk covers making AI on Databricks accessible to non-specialists. The Agent Bricks page linked below is the official reference for building AI agents and applications on the platform. Official docs: Introduces Agent Bricks for building, deploying, governing and monitoring agents on Databricks. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/generative-ai/agent-bricks/
🔗 More: https://www.nextgenlakehouse.com/answers/genai

#Databricks #AI #GenAI #DataAndAISummit #AIForEveryone #AgentBricks #DataAI

— Pinned comment —
AI for everyone, from a DAIS 2025 talk. The Agent Bricks docs and our GenAI hub are linked below. Docs: https://docs.databricks.com/aws/en/generative-ai/agent-bricks/ Hub: https://www.nextgenlakehouse.com/answers/genai

, Demo from Data+AI Summit 2024: Building GenAI Apps on Databricks

This Data+AI Summit 2024 demo shows, building and serving generative AI applications on Databricks. It walks through an end-to-end example. The Databricks machine learning docs cover model training, serving and the AI capabilities demonstrated in the session. Demos like this show a full path from data to model to serving, which is easier to follow than isolated API examples. Check the 2024 details against current docs, since product names and APIs have changed. 📚 Docs: https://docs.databricks.com/aws/en/machine-learning/. 🔗 More: https://www.nextgenlakehouse.com/learn (GenAI hub). #Databricks #MosaicAI #MachineLearning #DataAISummit #GenAI #LLM #AI #DataAnalytics

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/machine-learning/ More GenAI content lives at https://www.nextgenlakehouse.com/learn

The Secret Sauce Behind DBRX: Jonathan Frankle on Training an Open LLM

DBRX is Databricks' open general-purpose large language model, and this conversation with Jonathan Frankle explains how it was trained. The DBRX launch blog post is the primary reference. The DBRX announcement covers the model architecture, training approach and evaluation results released with it. DBRX is an open model, so check the release notes and model card for licensing and evaluation details before using it. The interview explains the training decisions behind it. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://www.databricks.com/blog/introducing-dbrx-new-state-art-open-llm. 🔗 More: https://www.nextgenlakehouse.com/learn (GenAI hub). #Databricks #DBRX #LLM #OpenSourceAI #GenAI #AI #DataAnalytics #DataEngineering

— Pinned comment —
Official reference: https://www.databricks.com/blog/introducing-dbrx-new-state-art-open-llm More GenAI content lives at https://www.nextgenlakehouse.com/learn

SQL AI Functions: Sentiment, Entity Extraction and Classification in Databricks

SQL AI functions such as ai_analyze_sentiment, ai_extract and ai_classify let you run LLM tasks directly in SQL queries. This video shows sentiment, extraction and classification examples. The ai_analyze_sentiment function returns a sentiment label for text, and ai_extract and ai_classify handle structured extraction and labelling. The AI Functions overview lists all of them. Use these functions on small samples first to check output quality and cost before running them across a full table. 📚 Docs: https://docs.databricks.com/aws/en/sql/language-manual/functions/ai_analyze_sentiment and https://docs.databricks.com/aws/en/sql/language-manual/functions/ai_extract. 🔗 More: https://www.nextgenlakehouse.com/learn (GenAI hub). #Databricks #AIFunctions #SQL #Sentiment #GenAI #LLM #AI #DataAnalytics

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/sql/language-manual/functions/ai_analyze_sentiment More GenAI content lives at https://www.nextgenlakehouse.com/learn

Getting Started with LLMs in SQL: ai_query and AI Functions in Databricks SQL

You can call LLMs from Databricks SQL with ai_query and the AI Functions, without writing Python. This video shows how to get started with LLM queries in SQL. ai_query sends a prompt to a served model from a SQL statement, and the AI Functions overview lists the purpose-built functions. Start with a small table and a simple prompt, then check output quality before scaling to millions of rows. Watch model call costs as row counts grow. 📚 Docs: https://docs.databricks.com/aws/en/sql/language-manual/functions/ai_query and https://docs.databricks.com/aws/en/large-language-models/ai-functions. 🔗 More: https://www.nextgenlakehouse.com/learn (GenAI hub). #Databricks #LLM #SQL #AIFunctions #GenAI #AI #DataAnalytics #DataEngineering

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/sql/language-manual/functions/ai_query More GenAI content lives at https://www.nextgenlakehouse.com/learn

Getting Started with Databricks Foundation Model APIs and External Models

Foundation Model APIs give you pay-per-token access to hosted foundation models, and External Models route requests to third-party providers. This video shows how to get started with both. The Foundation Model APIs doc lists supported models and endpoints, and the external models doc covers configuring third-party providers. Pay-per-token endpoints suit prototypes and variable workloads, while provisioned throughput fits steady high volume. Check the external models doc for the provider keys you need. 📚 Docs: https://docs.databricks.com/aws/en/machine-learning/foundation-model-apis/ and https://docs.databricks.com/aws/en/machine-learning/foundation-models/external-models. 🔗 More: https://www.nextgenlakehouse.com/learn (GenAI hub). #Databricks #FoundationModels #ModelServing #LLM #GenAI #AI #DataAnalytics #DataEngineering

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/machine-learning/foundation-model-apis/ More GenAI content lives at https://www.nextgenlakehouse.com/learn

Getting Started with Databricks Vector Search (now AI Search)

Databricks Vector Search is now called AI Search, and it builds vector indexes over Unity Catalog tables for retrieval. This video shows how to get started. The AI Search doc explains how managed indexes stay in sync with source tables and reuse embeddings for unchanged rows. Managed indexes keep embeddings in sync with the source table, so you avoid rebuilding the index after each change. Evaluate retrieval quality on real questions before relying on it in a RAG app. 📚 Docs: https://docs.databricks.com/aws/en/ai-search/ai-search. 🔗 More: https://www.nextgenlakehouse.com/learn (GenAI hub). #Databricks #VectorSearch #AISearch #RAG #GenAI #LLM #AI #DataAnalytics

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/ai-search/ai-search More GenAI content lives at https://www.nextgenlakehouse.com/learn

Custom AI Coding Agents: Governance and Design on Databricks

How should you design and govern custom AI coding agents? This video covers the governance questions teams hit when they build their own coding agents: which models the agent can call, which tools and data it can reach, and how usage is audited. In Databricks, the AI governance guide describes how Unity Catalog and Unity Gateway control AI assets and traffic. Start with access policies on the agent's resources before you widen its scope. Decide the agent's tool and data scope first, then add the gateway and catalog policies that enforce those limits in production. 📚 Docs: https://docs.databricks.com/aws/en/ai-gateway/ai-governance https://docs.databricks.com/aws/en/unity-gateway 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #AIAgents #CodingAgents #AgenticAI #AIGovernance #UnityGateway

— Pinned comment —
Official reference for this topic: https://docs.databricks.com/aws/en/ai-gateway/ai-governance More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Business Leaders Share AI Horror Stories: Lessons from Data+AI Summit

What goes wrong when businesses roll out AI? This Data+AI Summit interview features business leaders who describe AI projects that went badly, from data quality problems to governance gaps and unrealistic expectations. The stories illustrate why data foundations, evaluation and access controls matter before you scale AI. Use the Summit site to find the sessions and announcements that address the same problems. The lessons apply to any AI rollout, so use them to pressure-test your own data quality, evaluation and access controls before launch. 📚 Docs: https://www.databricks.com/dataaisummit 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #DataAISummit #AI #GenAI #DataStrategy #Interview

— Pinned comment —
Official reference for this topic: https://www.databricks.com/dataaisummit More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Getting Started with Unity Gateway (formerly Unity Gateway) on Databricks

How do you get started with Unity Gateway? This video introduces the gateway that Databricks formerly called Unity Gateway, which controls model access, cost limits, security and agent integrations. The official overview covers getting started, and the concepts page explains how models, tools and access fit together. Start by configuring one endpoint and checking the request logs before you expand access to other teams. Configure one model endpoint first, check the request logs, then widen access to other teams and agents. 📚 Docs: https://docs.databricks.com/aws/en/unity-gateway https://docs.databricks.com/aws/en/unity-gateway/concepts 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #UnityGateway #AIGateway #UnityCatalog #GenAI #AgenticAI

— Pinned comment —
Official reference for this topic: https://docs.databricks.com/aws/en/unity-gateway More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Databricks AI Upgrade: 3x Faster and 90% Accuracy Claims Explained

How much faster and more accurate is the latest Databricks AI upgrade? The video presents a headline of 3x faster responses and 90% accuracy. Treat these figures as the video's claims and check the benchmark setup, dataset and configuration before quoting them. For the official behaviour and setup of Databricks AI agents, read the Agent Bricks documentation linked below. Check whether the benchmark used your kind of data and workload, because speed and accuracy gains vary widely between use cases. 📚 Docs: https://docs.databricks.com/aws/en/agents 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #AgentBricks #GenAI #AI #MosaicAI #Performance

— Pinned comment —
Official reference for this topic: https://docs.databricks.com/aws/en/agents More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

ai_parse_document Explained: Parse PDFs and Office Files with SQL

What does the ai_parse_document function do? This video explains the new SQL function that reads documents from a Unity Catalog volume and returns a VARIANT with pages and elements, including text, HTML tables, figures and layout information. It supports PDF, JPG, PNG, TIFF, DOC/DOCX and PPT/PPTX files. The SQL reference lists the syntax and options, and the Document Parsing guide shows a UI and REST workflow. Run it on a small volume of files first and inspect the VARIANT output before you build a pipeline on top of it. 📚 Docs: https://docs.databricks.com/aws/en/sql/language-manual/functions/ai_parse_document https://docs.databricks.com/aws/en/agents/agent-bricks/document-parsing 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #AIParseDocument #SQL #DocumentAI #GenAI #UnityCatalog

— Pinned comment —
Official reference for this topic: https://docs.databricks.com/aws/en/sql/language-manual/functions/ai_parse_document More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Agent Bricks Demo from Data+AI Summit: Building Enterprise AI Agents

How does Agent Bricks work in practice? This Data+AI Summit demo shows Agent Bricks, Databricks' product for building agents on enterprise data with managed evaluation and deployment. The Agent Bricks documentation describes the agent types, how they connect to Unity Catalog data, and how to evaluate them before rollout. Watch the demo steps, then read the agent documentation to plan how you would evaluate an agent before rollout. Agent evaluation is the step most teams skip, so decide your quality metrics and test cases before you show a demo to stakeholders. 📚 Docs: https://docs.databricks.com/aws/en/agents 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #AgentBricks #AIAgents #GenAI #MosaicAI #DataAISummit

— Pinned comment —
Official reference for this topic: https://docs.databricks.com/aws/en/agents More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Anthropic CEO and Ali Ghodsi at Data+AI Summit

What did the Anthropic CEO discuss with Ali Ghodsi? This Data+AI Summit conversation features the Anthropic CEO with Databricks CEO Ali Ghodsi, covering the state of enterprise AI. For how Databricks serves Claude and other models through its platform, check the AI Gateway and serving documentation. The conversation is high level, so use the Summit sessions for the details on how Databricks and model providers work together. Conversations between industry leaders often point to where enterprise AI is heading, so note the themes and compare them with your own plans. 📚 Docs: https://www.databricks.com/dataaisummit 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #Anthropic #Claude #AliGhodsi #EnterpriseAI #DataAISummit

— Pinned comment —
Official reference for this topic: https://www.databricks.com/dataaisummit More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

How Software Development Is Changing: Data+AI Summit Discussion

How is development changing with AI? This Data+AI Summit discussion covers how AI-assisted tools change the work of building software and data products. Genie Code is the Databricks coding agent for building notebooks, pipelines and dashboards, and its documentation is linked below. Pick one routine task in your workflow and test how an AI coding assistant changes the time it takes. Changes in how software gets built affect review, testing and ownership, so agree on those rules with your team before you adopt new AI tools widely. 📚 Docs: https://docs.databricks.com/aws/en/genie-code 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #AI #SoftwareDevelopment #GenieCode #AIAgents #DataAISummit

— Pinned comment —
Official reference for this topic: https://docs.databricks.com/aws/en/genie-code More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

English Is the New Coding Language: Building with Genie Code

Is English the new coding language? This video discusses how natural-language prompts are changing how people build data pipelines, notebooks and dashboards. Genie Code is the Databricks coding agent that generates and debugs code from plain-language requests inside the workspace. The documentation explains the requirements and how to use it. Describe the outcome you want in plain language, review the generated code, and run it on sample data before production. Natural-language requests work best when you name the tables, the expected output and the checks, so review the result against those. 📚 Docs: https://docs.databricks.com/aws/en/genie-code 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #GenieCode #AI #NaturalLanguage #Coding #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://docs.databricks.com/aws/en/genie-code More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

DBRX: The Cycle of Creating an Open Model

How was DBRX created? This video covers the creation cycle behind DBRX, the open general-purpose language model from Databricks. The DBRX launch post describes the model's architecture, training and benchmark results, which is the official source for its specifications. Read the launch post alongside the video to see the specifications and benchmarks behind the model. Model launches involve data, evaluation and infrastructure work, so the story shows the full cycle rather than only the final model. The official launch post is the reference for the model's specifications and benchmark numbers. 📚 Docs: https://www.databricks.com/blog/introducing-dbrx-new-state-art-open-llm 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #DBRX #OpenModel #LLM #GenAI #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/blog/introducing-dbrx-new-state-art-open-llm More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Research in Industry: Lessons from Building DBRX

What is research like in industry? This video discusses applied research at Databricks, drawing on the work behind the DBRX open model. It is a perspective on how research turns into products. The DBRX launch post is the official reference for the model. Look at how research results are evaluated and shipped before you judge an idea's readiness for production. Applied research teams balance publication goals with product deadlines, so the discussion is useful if you manage similar trade-offs. 📚 Docs: https://www.databricks.com/blog/introducing-dbrx-new-state-art-open-llm 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #DBRX #AIResearch #LLM #GenAI #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/blog/introducing-dbrx-new-state-art-open-llm More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Why Train Your Own Model? The DBRX Perspective

Why train your own model? This video discusses when a company should train its own model rather than use an existing one, with the DBRX experience as the example. The answer depends on data, cost and control needs. The DBRX launch post explains what Databricks built and why. Compare the cost, data and control trade-offs before deciding whether to train, fine-tune or use an existing model. Start with a clear use case and an evaluation set, because those decide whether custom training is worth the cost for your team. 📚 Docs: https://www.databricks.com/blog/introducing-dbrx-new-state-art-open-llm 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #DBRX #LLM #ModelTraining #GenAI #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/blog/introducing-dbrx-new-state-art-open-llm More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

DBRX Mascot: The Story Behind the Brand

What is the DBRX mascot about? This short video shares the story behind the DBRX mascot, a branding piece around the open model. It is not technical content. The DBRX launch post is the official source for the model itself. Enjoy the brand story, and check the launch post if you want the model details. Brand characters help people remember a launch, so the video is a light break between the technical content in the channel. It is a brand story rather than a technical walkthrough, so use the launch post for model details. 📚 Docs: https://www.databricks.com/blog/introducing-dbrx-new-state-art-open-llm 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #DBRX #Branding #OpenModel #GenAI #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/blog/introducing-dbrx-new-state-art-open-llm More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Why Generative AI? Matei Zaharia on the Case for GenAI

Why should you care about generative AI? Matei Zaharia, Databricks co-founder, explains the case for generative AI in enterprise data work. Agent Bricks is the Databricks product for building agents on your own data, and its documentation shows how these ideas become working systems. Start with one business use case and measure the result before you scale generative AI across the organisation. Generative AI projects succeed when they target a specific process with clear success measures, so define those before you choose a model. 📚 Docs: https://docs.databricks.com/aws/en/agents 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #GenAI #MateiZaharia #AgentBricks #AI #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://docs.databricks.com/aws/en/agents More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Genie ZeroOps: Find and Fix Data Pipeline Issues in Minutes

Genie ZeroOps uses Genie to find and fix data pipeline issues quickly. The video shows how natural-language questions help troubleshoot pipelines on Databricks. Genie is the Databricks AI experience for asking governed data questions in natural language and working with data assets. The Genie overview linked below describes the feature set, including Genie Agents for operational tasks. Official docs: Describes Genie, the AI experience for asking governed data questions in natural language. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/genie/
🔗 More: https://www.nextgenlakehouse.com/answers/genie

#Databricks #Genie #ZeroOps #DataPipelines #DataEngineering #Observability #AI

— Pinned comment —
Genie helps troubleshoot pipeline issues in minutes. The Genie docs and our Genie hub are linked below. Docs: https://docs.databricks.com/aws/en/genie/ Hub: https://www.nextgenlakehouse.com/answers/genie

SuperSkills Ep 1: Mastering Genie Agent on Databricks

Genie Agent is a chat tool that lets business users query governed data in plain language. This SuperSkills episode walks through using and building it. The Genie Agent guide covers how to build, share and administer a Genie Agent, which business users access to query their data. Use it to set up the space and the data it can see. Official docs: Explains how to build, share and administer a Genie Agent for business users. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/genie/set-up
🔗 More: https://www.nextgenlakehouse.com/answers/genie

#Databricks #Genie #GenieAgent #SuperSkills #AIBI #BusinessIntelligence #GenAI

— Pinned comment —
This SuperSkills episode covers the Genie Agent. The Genie setup docs and our Genie hub are linked below. Docs: https://docs.databricks.com/aws/en/genie/set-up Hub: https://www.nextgenlakehouse.com/answers/genie

Databricks Cost Management: System Tables, Genie and FinOps

System tables give you account-wide billing and usage data for cost management, and Genie lets you query it in plain language. This video shows a FinOps workflow built on both. The billable usage system table reference documents system.billing.usage, including record types and billing origin products, the data behind cost dashboards. The system tables overview linked below lists the other operational tables. Official docs: Documents system.billing.usage, the account-wide billable usage table with record types and billing origin products. Overview of the system catalog that stores operational data for cost, security and compute monitoring.

📚 Docs: https://docs.databricks.com/aws/en/admin/system-tables/billing https://docs.databricks.com/aws/en/admin/system-tables/
🔗 More: https://www.nextgenlakehouse.com/answers/genie

#Databricks #FinOps #CostManagement #SystemTables #Genie #Billing #DataGovernance

— Pinned comment —
Cost management with system tables and Genie is the focus. The billing docs and our Genie hub are linked below. Docs: https://docs.databricks.com/aws/en/admin/system-tables/billing Hub: https://www.nextgenlakehouse.com/answers/genie

LLM-Powered AI Dashboards from Genie One

Genie One builds AI dashboards from natural-language questions on Databricks, with LLM-powered analysis on top of governed data. This video shows how to create one. AI/BI dashboards on Databricks let you author, share and automate visualizations, and Genie answers questions over the same data. The dashboards overview linked below covers authoring and sharing; the Genie page covers the natural-language layer. Official docs: Introduces AI/BI dashboards, with guides for authoring, sharing, automating and monitoring them. Describes Genie, the AI experience for asking governed data questions in natural language.

📚 Docs: https://docs.databricks.com/aws/en/dashboards/ https://docs.databricks.com/aws/en/genie/
🔗 More: https://www.nextgenlakehouse.com/answers/genie

#Databricks #Genie #AIBI #Dashboards #GenAI #BusinessIntelligence #LLM

— Pinned comment —
Genie One and LLM-powered dashboards are shown here. The dashboards docs and our Genie hub are linked below. Docs: https://docs.databricks.com/aws/en/dashboards/ Hub: https://www.nextgenlakehouse.com/answers/genie

Getting Started with Data Science Agent in Databricks Assistant

The Data Science Agent in Databricks Assistant helps you explore data and write analysis code in natural language. This video shows how to start using it. The Genie Code page describes the coding, data analysis and documentation-search features that Databricks Assistant offers across notebooks, SQL, jobs and dashboards. Use it as the reference for the agent features shown in the video. Official docs: Describes Genie Code features for coding, data analysis and documentation search across notebooks, SQL, jobs and dashboards.

📚 Docs: https://docs.databricks.com/aws/en/notebooks/use-databricks-assistant
🔗 More: https://www.nextgenlakehouse.com/answers/genie

#Databricks #DatabricksAssistant #DataScience #GenAI #Notebooks #Genie #AI

— Pinned comment —
Data Science Agent in Databricks Assistant is the topic. The Genie Code docs and our Genie hub are linked below. Docs: https://docs.databricks.com/aws/en/notebooks/use-databricks-assistant Hub: https://www.nextgenlakehouse.com/answers/genie

All You Need to Know About Genie One

Genie One is the Databricks AI experience for asking governed data questions in natural language. This video covers what it does and how to use it. The Genie overview describes the AI experience for governed data questions, dashboards and apps, and introduces Genie One, Genie Code and Genie Agents. Use it as the entry point for the features covered in the video. Official docs: Describes Genie, the AI experience for asking governed data questions in natural language. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/genie/
🔗 More: https://www.nextgenlakehouse.com/answers/genie

#Databricks #GenieOne #Genie #GenAI #AIBI #BusinessIntelligence #DataAI

— Pinned comment —
Everything about Genie One in one video. The Genie docs and our Genie hub are linked below. Docs: https://docs.databricks.com/aws/en/genie/ Hub: https://www.nextgenlakehouse.com/answers/genie

Chat Your Way to Cost Visibility with Genie Agents

Genie Agents let you ask natural-language questions about Databricks cost and usage. This video shows how to get cost visibility by chatting with the data. The billable usage system table documents system.billing.usage, the source for cost questions, including record types and billing origin products. The Genie Agent guide linked below covers how to build and share a Genie Agent over that data. Official docs: Documents system.billing.usage, the account-wide billable usage table with record types and billing origin products. Explains how to build, share and administer a Genie Agent for business users.

📚 Docs: https://docs.databricks.com/aws/en/admin/system-tables/billing https://docs.databricks.com/aws/en/genie/set-up
🔗 More: https://www.nextgenlakehouse.com/answers/genie

#Databricks #Genie #GenieAgents #CostVisibility #FinOps #SystemTables #Billing

— Pinned comment —
Cost visibility through Genie Agents is the topic. The billing docs and our Genie hub are linked below. Docs: https://docs.databricks.com/aws/en/admin/system-tables/billing Hub: https://www.nextgenlakehouse.com/answers/genie

From Beginner to Pro: The Ultimate Genie Agents (AI/BI Genie) Best Practices Guide

Genie Agents, the AI/BI Genie feature, answer business questions in natural language over governed data. This guide covers best practices for building good spaces and agents. The Genie Agent guide covers how to build, share and administer a Genie Agent for business users. Use it as the reference for setup choices covered in the best-practices walkthrough. Official docs: Explains how to build, share and administer a Genie Agent for business users. Describes Genie, the AI experience for asking governed data questions in natural language.

📚 Docs: https://docs.databricks.com/aws/en/genie/set-up https://docs.databricks.com/aws/en/genie/
🔗 More: https://www.nextgenlakehouse.com/answers/genie

#Databricks #Genie #AIBI #GenieAgents #BestPractices #BusinessIntelligence #GenAI

— Pinned comment —
Best practices for Genie Agents, from beginner to pro. The Genie Agent docs and our Genie hub are linked below. Docs: https://docs.databricks.com/aws/en/genie/set-up Hub: https://www.nextgenlakehouse.com/answers/genie

Upload a CSV to Genie and Explore Your Data in Natural Language

You can upload a CSV file to Genie and ask questions about it in plain English, with answers generated as SQL over the data. This video walks through the upload and first questions. Genie Agents (formerly Genie Spaces) is where you add tables and instructions to improve answer accuracy. Start with the setup doc, then use the Genie overview to see how questions are answered. Uploaded files are a point-in-time snapshot, so re-upload when the source changes. Clear table and column descriptions help Genie pick the right data for each question. 📚 Docs: https://docs.databricks.com/aws/en/genie-agents/set-up and https://docs.databricks.com/aws/en/genie/. 🔗 More: https://www.nextgenlakehouse.com/learn (Genie hub). #Databricks #NaturalLanguage #SQL #Genie #GenAI #AIBI #DataAnalytics #DataEngineering

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/genie-agents/set-up More Genie content lives at https://www.nextgenlakehouse.com/learn

Databricks AI Assistant Guide: Genie Code (Formerly Assistant) Setup and Tips

Databricks AI Assistant, now documented as Genie Code, helps you write, explain and fix code in notebooks, the SQL editor and pipelines. This guide shows how to use it effectively. Genie Code generates and runs code, builds pipelines and dashboards, and debugs errors while working with Unity Catalog. Read the Genie Code doc for supported editors and how chats persist. Review generated code before running it on production data, especially when the assistant edits pipelines or touches permissions. 📚 Docs: https://docs.databricks.com/aws/en/genie-code. 🔗 More: https://www.nextgenlakehouse.com/learn (Genie hub). #Databricks #AIAssistant #GenieCode #Notebooks #Genie #GenAI #AIBI #DataAnalytics

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/genie-code More Genie content lives at https://www.nextgenlakehouse.com/learn

Text-to-SQL with Genie Agents: Setup and Accuracy Tips with Chao Cai

Genie Agents (formerly Genie Spaces) turn natural-language questions into SQL over your tables. This conversation with Chao Cai covers how to set them up for accurate answers. Add the right tables and instructions to a Genie Agent, then test sample questions. The Genie Agents setup doc lists the configuration steps. Genie accuracy improves with clear table descriptions, sample questions and instructions about business terms, so budget time for that setup. Test with real questions from business users. 📚 Docs: https://docs.databricks.com/aws/en/genie-agents/set-up. 🔗 More: https://www.nextgenlakehouse.com/learn (Genie hub). #Databricks #TextToSQL #GenAI #AIBI #Genie #DataAnalytics #DataEngineering #Lakehouse

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/genie-agents/set-up More Genie content lives at https://www.nextgenlakehouse.com/learn

Getting Started with Databricks Assistant (Genie Code) in Notebooks

The Databricks Assistant, now documented as Genie Code, helps you write, explain and fix code in Databricks. This video shows how to get started with it. Genie Code runs in notebooks, the SQL editor and the pipeline editor, and its doc describes what it can generate and debug. Use it to explain unfamiliar code, generate cells and debug errors inside the notebook you are working in. Review generated code before running it against production data. 📚 Docs: https://docs.databricks.com/aws/en/genie-code. 🔗 More: https://www.nextgenlakehouse.com/learn (Genie hub). #Databricks #DatabricksAssistant #GenieCode #Notebooks #Genie #GenAI #AIBI #DataAnalytics

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/genie-code More Genie content lives at https://www.nextgenlakehouse.com/learn

Chat with Genie ZeroOps: Ask Your Data Operations Agent Questions

What is Genie ZeroOps and how do you chat with it? Genie ZeroOps is a Databricks operations agent that monitors logs, metrics and run history for jobs, pipelines, tables and machine-learning workloads. It investigates issues using Unity Catalog lineage, proposes a fix, tests it in an isolated sandbox using shallow clones of production data, and waits for approval. The announcement says it is in private preview and that Apps and Lakebase are roadmap items. Ask about specific jobs or tables, then check the investigation trail and proposed fix before approving any change. 📚 Docs: https://www.databricks.com/blog/introducing-genie-zeroops https://docs.databricks.com/aws/en/genie/ 🔗 More: https://www.nextgenlakehouse.com/content#genie #Databricks #GenieZeroOps #Genie #DataOps #AIAgents #Observability

— Pinned comment —
Official reference for this topic: https://www.databricks.com/blog/introducing-genie-zeroops More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content#genie

Genie ZeroOps Fixes Data Fires: Detect, Investigate, Remediate, Verify

How does Genie ZeroOps fix data fires? The agent detects failures and silent data-quality issues, uses Unity Catalog lineage to trace the root cause to a schema change, bad data or a code defect, drafts a remediation, and verifies it in a sandbox against shallow clones of production data. A human then reviews the proposed fix. Databricks announced it on June 16, 2026, and public material describes it as private preview. The verify step runs against shallow clones, so the fix is tested without touching production data before anyone approves it. 📚 Docs: https://www.databricks.com/blog/introducing-genie-zeroops https://docs.databricks.com/aws/en/genie/ 🔗 More: https://www.nextgenlakehouse.com/content#genie #Databricks #GenieZeroOps #Genie #DataOps #DataQuality #UnityCatalog

— Pinned comment —
Official reference for this topic: https://www.databricks.com/blog/introducing-genie-zeroops More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content#genie

Making AI/BI Genie Reliable: Practical Tips for Accurate Answers

How do you make AI/BI Genie reliable? This video covers practical steps for getting consistent answers from a Genie space. Scope the space to a small set of well-documented tables, add clear instructions and sample questions, and review the SQL Genie generates. The Genie best practices guide lists these steps in more detail. Review the answers Genie returns against the source tables, and tighten the instructions where the SQL misses your business rules. Reliable Genie answers depend on the same clean data you would use for dashboards, so check the table comments and column names before you add more instructions. 📚 Docs: https://docs.databricks.com/aws/en/genie-agents/best-practices https://docs.databricks.com/aws/en/genie-agents/set-up 🔗 More: https://www.nextgenlakehouse.com/content#genie #Databricks #Genie #AIBI #GenAI #DataAnalytics #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://docs.databricks.com/aws/en/genie-agents/best-practices More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content#genie

Improving AI/BI Genie: Space Setup and Best Practices

How do you improve AI/BI Genie? This video walks through improvements to a Genie space, including the data it can access, how it is configured and how to review its answers. The Genie setup guide covers creating a space, and the best practices page lists tuning steps you can apply. Apply one improvement at a time and rerun your sample questions, so you can see which change actually helped. Use sample questions from real business users to test a space, because those questions expose gaps that synthetic tests can miss. 📚 Docs: https://docs.databricks.com/aws/en/genie-agents/set-up https://docs.databricks.com/aws/en/genie-agents/best-practices 🔗 More: https://www.nextgenlakehouse.com/content#genie #Databricks #Genie #AIBI #GenAI #DataAnalytics #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://docs.databricks.com/aws/en/genie-agents/set-up More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content#genie

What Is LTAP? Lakebase and Genie Explained by Databricks CTO

This video explains LTAP, the pairing of Lakebase operational Postgres with Genie for natural-language access to data. It is a conceptual explainer from the Databricks CTO. Lakebase is Databricks' managed Postgres for low-latency application workloads, and Genie lets business users ask governed questions of data in plain language. The Lakebase overview and the Genie overview linked below are the official references for the two components. Official docs: Introduces Lakebase, the fully managed Postgres database integrated into the Databricks platform. Describes Genie, the AI experience for asking governed data questions in natural language.

📚 Docs: https://docs.databricks.com/aws/en/oltp/ https://docs.databricks.com/aws/en/genie/
🔗 More: https://www.nextgenlakehouse.com/answers/lakebase

#Databricks #Lakebase #Genie #Postgres #OLTP #AI #LTAP

— Pinned comment —
Lakebase and Genie are the two halves of this explainer. Official Lakebase and Genie docs are linked below. Docs: https://docs.databricks.com/aws/en/oltp/ Hub: https://www.nextgenlakehouse.com/answers/lakebase

Lakebase Demo from the Data+AI Summit 2025

This demo shows Lakebase, the managed Postgres for low-latency application workloads on Databricks, from the Data and AI Summit 2025. The Lakebase overview is the entry point. Lakebase is a fully managed Postgres database integrated into Databricks, with branching and autoscaling. The branches page linked below explains copy-on-write branches, which the demo can be used to exercise. Official docs: Introduces Lakebase, the fully managed Postgres database integrated into the Databricks platform. Explains copy-on-write branching in Lakebase for development, testing, schema changes and point-in-time recovery.

📚 Docs: https://docs.databricks.com/aws/en/oltp/ https://docs.databricks.com/aws/en/oltp/projects/branches
🔗 More: https://www.nextgenlakehouse.com/answers/lakebase

#Databricks #Lakebase #Postgres #OLTP #DAIS2025 #AppDevelopment #DataEngineering

— Pinned comment —
Lakebase demo from DAIS 2025. The Lakebase and branching docs and our Lakebase hub are linked below. Docs: https://docs.databricks.com/aws/en/oltp/ Hub: https://www.nextgenlakehouse.com/answers/lakebase

Neon Founder on Stage at Data+AI Summit: Serverless Postgres and Lakebase

What is Neon and how does it relate to Lakebase? The founder of Neon, the serverless Postgres company now part of Databricks, speaks on stage at Data+AI Summit. Lakebase is Databricks' Postgres database for operational apps and AI agents. The Lakebase projects documentation explains how to create and manage these Postgres databases. Use the founder's story as context, then try a Lakebase project to see the Postgres experience for yourself. Founder talks often explain design choices, such as how branching and scale-to-zero work, which help when you evaluate a Postgres service. 📚 Docs: https://docs.databricks.com/aws/en/oltp/projects https://www.databricks.com/product/lakebase 🔗 More: https://www.nextgenlakehouse.com/content#lakebase #Databricks #Lakebase #Neon #Postgres #ServerlessDatabase #DataAISummit

— Pinned comment —
Official reference for this topic: https://docs.databricks.com/aws/en/oltp/projects More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content#lakebase

Lakebase Demo: Postgres on the Lakehouse in Action

What does a Lakebase demo look like? This video demonstrates Lakebase, the Postgres database in Databricks built for operational applications and AI agents. The projects documentation explains how to create a database, connect with standard Postgres clients and sync data from the lakehouse. Follow the demo steps in the video and check the docs for connection details before adapting it to your app. Demos are the fastest way to see a database respond, so time your own queries against a Lakebase instance to get real numbers for your team. 📚 Docs: https://docs.databricks.com/aws/en/oltp/projects https://www.databricks.com/product/lakebase 🔗 More: https://www.nextgenlakehouse.com/content#lakebase #Databricks #Lakebase #Postgres #OLTP #Demo #Lakehouse

— Pinned comment —
Official reference for this topic: https://docs.databricks.com/aws/en/oltp/projects More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content#lakebase

Lakebase at Data+AI Summit 2025: Postgres for the Lakehouse

What is Lakebase, announced at Data+AI Summit 2025? This session video introduces Lakebase, a managed Postgres database in Databricks for transactional apps and AI agents. Lakebase lets applications run on operational data next to lakehouse analytics. The projects documentation covers setup and connection options. Compare the Lakebase setup steps with your current operational database to plan any migration. Session recordings from the Summit explain how Lakebase fits with lakehouse analytics, which helps when you design a data product that serves apps. 📚 Docs: https://docs.databricks.com/aws/en/oltp/projects https://www.databricks.com/product/lakebase 🔗 More: https://www.nextgenlakehouse.com/content#lakebase #Databricks #Lakebase #Postgres #DataAISummit #OLTP #Lakehouse

— Pinned comment —
Official reference for this topic: https://docs.databricks.com/aws/en/oltp/projects More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content#lakebase

What's New in Databricks Apps in 2026

This video covers what is new in Databricks Apps in 2026, the platform for building and deploying data and AI applications. The Databricks Apps docs linked below cover setup, development and deployment. Databricks Apps lets developers build and deploy full-stack data and AI applications on the platform, with resources such as SQL warehouses and model endpoints. The apps resources page explains how to connect an app to platform services without hardcoding credentials. Official docs: Overview of building, deploying and managing data and AI applications on Databricks Apps. Shows how to attach SQL warehouses, jobs, model endpoints and secrets to an app as managed resources.

📚 Docs: https://docs.databricks.com/aws/en/dev-tools/databricks-apps/ https://docs.databricks.com/aws/en/dev-tools/databricks-apps/resources
🔗 More: https://www.nextgenlakehouse.com/answers/apps

#Databricks #DatabricksApps #WebApps #AppDevelopment #DataApps #Python #FastAPI

— Pinned comment —
New Databricks Apps capabilities for 2026. The Apps docs and our Apps hub are linked below. Docs: https://docs.databricks.com/aws/en/dev-tools/databricks-apps/ Hub: https://www.nextgenlakehouse.com/answers/apps

Dive Into Databricks Apps Made Easy

This video walks through building apps on Databricks Apps, the platform for data and AI applications. The apps overview and resources page cover setup and connecting services. Databricks Apps lets you build, deploy and manage full-stack data and AI applications. The resources page explains how to attach SQL warehouses, jobs, model endpoints and secrets as managed resources instead of hardcoded IDs. Official docs: Overview of building, deploying and managing data and AI applications on Databricks Apps. Shows how to attach SQL warehouses, jobs, model endpoints and secrets to an app as managed resources.

📚 Docs: https://docs.databricks.com/aws/en/dev-tools/databricks-apps/ https://docs.databricks.com/aws/en/dev-tools/databricks-apps/resources
🔗 More: https://www.nextgenlakehouse.com/answers/apps

#Databricks #DatabricksApps #WebApps #AppDevelopment #DataApps #Python #FastAPI

— Pinned comment —
Building apps on Databricks Apps is the walkthrough. The Apps docs and our Apps hub are linked below. Docs: https://docs.databricks.com/aws/en/dev-tools/databricks-apps/ Hub: https://www.nextgenlakehouse.com/answers/apps

Configure Delta Sharing for Data Lakes Behind a Firewall

This video shows how to configure Delta Sharing for data lakes behind a firewall. Delta Sharing lets you share data without copying it, and recipients connect through a secure channel. The Delta Sharing page covers sharing live data across platforms, and the recipient page explains how to create recipients and manage their access. Use both together when the data lake sits behind a firewall. Official docs: Overview of Delta Sharing and OpenSharing for securely sharing data and AI assets. Explains creating Databricks-to-Databricks recipients over a managed secure connection.

📚 Docs: https://docs.databricks.com/aws/en/delta-sharing/ https://docs.databricks.com/aws/en/delta-sharing/create-recipient
🔗 More: https://www.nextgenlakehouse.com/answers/data-sharing

#Databricks #DeltaSharing #DataSharing #Firewall #DataLake #OpenSharing #DataGovernance

— Pinned comment —
Delta Sharing for firewalled data lakes is the topic. The Delta Sharing docs and our data sharing hub are linked below. Docs: https://docs.databricks.com/aws/en/delta-sharing/ Hub: https://www.nextgenlakehouse.com/answers/data-sharing

Meet Databricks MVP Jaco van Gelder: Data Lineage and Delta Sharing Interview

This Databricks MVP interview with Jaco van Gelder covers data lineage and Delta Sharing. It is a community perspective on sharing governed data. Delta Sharing lets you share live tables with recipients outside Databricks through an open protocol. The data sharing docs describe how providers and recipients are configured. Lineage shows how data moves between tables, which helps when auditing a shared dataset or tracing a change to its source. Read the sharing docs alongside the interview for the provider and recipient setup. 📚 Docs: https://docs.databricks.com/aws/en/data-sharing/. 🔗 More: https://www.nextgenlakehouse.com/learn (Data Sharing hub). #Databricks #DatabricksMVP #DeltaSharing #DataLineage #DataSharing #DataProducts #DataAnalytics #DataEngineering

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/data-sharing/ More Data Sharing content lives at https://www.nextgenlakehouse.com/learn

Simplify Your Data Stack with Hightouch and Databricks (Reverse ETL)

Hightouch syncs data from Databricks to operational tools, which simplifies the data stack for activation. This video shows how the two fit together. The Databricks partner directory lists Hightouch alongside other activation and ingestion integrations. Reverse ETL moves curated tables to the tools where teams act on them, so keep the source tables well modelled. Check the partner directory for the Hightouch setup and supported sync modes, and start with one sync. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://www.databricks.com/partners. 🔗 More: https://www.nextgenlakehouse.com/learn (Data Sharing hub). #Databricks #Hightouch #ReverseETL #DataActivation #DataSharing #DeltaSharing #DataProducts #DataAnalytics

— Pinned comment —
Official reference: https://www.databricks.com/partners More Data Sharing content lives at https://www.nextgenlakehouse.com/learn

Getting Started with Delta Sharing in Databricks: Share Data Securely

Delta Sharing lets you share live data with other organisations through an open protocol, without copying it. This video shows how to get started. The data sharing doc covers creating shares and recipients, and the open sharing page covers the protocol. Shares and recipients let partners read the data they are granted, without copies to manage. Check the sharing doc for the provider and recipient steps, and review what each recipient can read. 📚 Docs: https://docs.databricks.com/aws/en/data-sharing/ and https://docs.databricks.com/aws/en/opensharing. 🔗 More: https://www.nextgenlakehouse.com/learn (Data Sharing hub). #Databricks #DeltaSharing #OpenSource #UnityCatalog #DataSharing #DataProducts #DataAnalytics #DataEngineering

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/data-sharing/ More Data Sharing content lives at https://www.nextgenlakehouse.com/learn

Getting Started with Databricks Marketplace: Discover and Install Data

Databricks Marketplace lets consumers find and install shared datasets and AI assets from providers. This video shows how to get started as a consumer. The marketplace consumer guide covers finding listings and installing them into Unity Catalog. Marketplace listings let consumers find data products and install them into Unity Catalog. Check the listing for terms and update schedule, and use the consumer guide to set access for your team after install. 📚 Docs: https://docs.databricks.com/aws/en/marketplace/ and https://docs.databricks.com/aws/en/marketplace/get-started-consumer. 🔗 More: https://www.nextgenlakehouse.com/learn (Data Sharing hub). #Databricks #DatabricksMarketplace #DataProducts #UnityCatalog #DataSharing #DeltaSharing #DataAnalytics #DataEngineering

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/marketplace/ More Data Sharing content lives at https://www.nextgenlakehouse.com/learn

Top 3 Databricks Announcements You Cannot Ignore

This video recaps three recent Databricks announcements. No single product doc matches the whole recap, so the link points to the closest core reference, the Databricks SQL overview. Because the video covers several announcements, it is a roundup rather than a how-to. Check each announcement's own docs page, and use the SQL overview linked below as the general entry point for analytics features. Official docs: Introduces Databricks SQL, the lakehouse data warehouse for queries, monitoring and BI. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/sql/
🔗 More: https://www.nextgenlakehouse.com/answers/platform

#Databricks #DataAndAI #Lakehouse #Announcements #DatabricksSQL #DataEngineering #AI

— Pinned comment —
This roundup covers three recent Databricks announcements. The SQL overview and our platform hub are linked below. Docs: https://docs.databricks.com/aws/en/sql/ Hub: https://www.nextgenlakehouse.com/answers/platform

One Login, All Workspaces: Custom URLs for Databricks

This video shows how to use custom URLs so one login reaches multiple Databricks workspaces. The official custom workspace URL page was not found, so the link points to the networking page, the closest verified reference. Custom workspace URLs make it easier to route users to the right workspace from a single entry point. The networking page covers how Databricks secures access paths to workspaces, which is the closest verified context for this topic. Official docs: Explains how Databricks secures user access, control plane and classic compute connectivity, and serverless data access.

📚 Docs: https://docs.databricks.com/aws/en/security/network/
🔗 More: https://www.nextgenlakehouse.com/answers/platform

#Databricks #Workspace #SSO #Authentication #Admin #Security #Platform

— Pinned comment —
Single-login access across workspaces is covered here. The networking docs and our platform hub are linked below. Docs: https://docs.databricks.com/aws/en/security/network/ Hub: https://www.nextgenlakehouse.com/answers/platform

Databricks Data+AI Summit 2026 Invitation

This is an invitation to the Databricks Data and AI Summit 2026. It is an event announcement, not a how-to, so the link points to the Databricks SQL overview as the closest product reference. The Summit covers the latest Databricks product directions across data engineering, analytics and AI. Use the SQL overview linked below as a starting point for the analytics product area. Official docs: Introduces Databricks SQL, the lakehouse data warehouse for queries, monitoring and BI. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/sql/
🔗 More: https://www.nextgenlakehouse.com/answers/platform

#Databricks #DataAndAISummit #DAIS2026 #Event #DataAndAI #Lakehouse #Conference

— Pinned comment —
This video is a Data and AI Summit 2026 invitation. The SQL overview and our platform hub are linked below. Docs: https://docs.databricks.com/aws/en/sql/ Hub: https://www.nextgenlakehouse.com/answers/platform

How to Audit and Harden Your Databricks Workspace Security with SAT

The Security Analysis Tool (SAT) audits a Databricks workspace against security best practices. The security-analysis page was not found, so the link points to the Databricks networking docs, the closest verified security reference. Workspace hardening covers network access, cluster configuration and identity controls. The networking page linked below explains how Databricks secures access paths and connectivity, which is the baseline a security audit checks against. Official docs: Explains how Databricks secures user access, control plane and classic compute connectivity, and serverless data access.

📚 Docs: https://docs.databricks.com/aws/en/security/network/
🔗 More: https://www.nextgenlakehouse.com/answers/platform

#Databricks #Security #SAT #CloudSecurity #WorkspaceSecurity #Governance #Compliance

— Pinned comment —
Workspace security auditing with SAT is covered here. The networking docs and our platform hub are linked below. Docs: https://docs.databricks.com/aws/en/security/network/ Hub: https://www.nextgenlakehouse.com/answers/platform

Build Dashboards and Pipelines for Free: Databricks Free Edition Tutorial

Databricks Free Edition lets you build dashboards and pipelines at no cost for learning and experimentation. This tutorial walks through the first projects. The Free Edition page explains how to sign up for the no-cost account and what it allows for personal learning and experimentation. Follow it to set up the workspace used in the tutorial. Official docs: Explains how to sign up for the no-cost Databricks Free Edition for learning and experimentation. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/getting-started/free-edition
🔗 More: https://www.nextgenlakehouse.com/answers/platform

#Databricks #FreeEdition #AIBI #Dashboards #DataEngineering #Learning #Lakeflow

— Pinned comment —
Build dashboards and pipelines on Free Edition. The Free Edition docs and our platform hub are linked below. Docs: https://docs.databricks.com/aws/en/getting-started/free-edition Hub: https://www.nextgenlakehouse.com/answers/platform

NextGenLakehouse Wishes for 2026

This channel video shares NextGenLakehouse wishes for Databricks in 2026. It is a community message, not a product how-to, so the link points to the Databricks SQL overview as the general reference. The video is a non-technical roundup of hopes for the year ahead on the Databricks platform. The SQL overview linked below is the general reference for the analytics area that most of the wishes touch. Official docs: Introduces Databricks SQL, the lakehouse data warehouse for queries, monitoring and BI.

📚 Docs: https://docs.databricks.com/aws/en/sql/
🔗 More: https://www.nextgenlakehouse.com/answers/platform

#Databricks #Lakehouse #DataEngineering #2026 #Community #DataAndAI #NextGenLakehouse

— Pinned comment —
A community wish list for Databricks in 2026. The SQL overview and our platform hub are linked below. Docs: https://docs.databricks.com/aws/en/sql/ Hub: https://www.nextgenlakehouse.com/answers/platform

Invitation au Data+AI World Tour Paris 2025

This is a French-language invitation to the Databricks Data and AI World Tour in Paris 2025. It is an event announcement, so the link points to the Databricks SQL overview as the general product reference. The World Tour covers Databricks data and AI product updates for the French market. The SQL overview linked below is the general reference for the analytics area covered at the event. Official docs: Introduces Databricks SQL, the lakehouse data warehouse for queries, monitoring and BI.

📚 Docs: https://docs.databricks.com/aws/en/sql/
🔗 More: https://www.nextgenlakehouse.com/answers/platform

#Databricks #DataAndAI #WorldTour #Paris #Event #DataEngineering #Lakehouse

— Pinned comment —
Invitation au Data+AI World Tour Paris 2025. The SQL overview and our platform hub are linked below. Docs: https://docs.databricks.com/aws/en/sql/ Hub: https://www.nextgenlakehouse.com/answers/platform

Booster vos projets Data et IA sur Databricks avec le Serverless

Cette vidéo explique comment accélérer vos projets data et IA sur Databricks grâce au calcul serverless. Serverless compute runs notebooks, jobs and pipelines without managing clusters. The serverless compute page introduces Databricks-managed serverless compute for notebooks, workflows and Lakeflow pipelines, with setup guides and FAQs. The limitations page linked below lists what serverless does not yet support. Official docs: Introduces Databricks-managed serverless compute for notebooks, workflows and Lakeflow pipelines. Lists current restrictions on serverless compute for notebooks and jobs, including UDFs and streaming triggers.

📚 Docs: https://docs.databricks.com/aws/en/compute/serverless/ https://docs.databricks.com/aws/en/compute/serverless/limitations
🔗 More: https://www.nextgenlakehouse.com/answers/platform

#Databricks #Serverless #DataEngineering #IA #DataScience #Lakeflow #French

— Pinned comment —
Serverless Databricks for vos projets data et IA. La doc serverless et notre hub plateforme sont liés ci-dessous. Docs: https://docs.databricks.com/aws/en/compute/serverless/ Hub: https://www.nextgenlakehouse.com/answers/platform

Inside the Game: How Databricks Is Shaping the Future of Gaming

This conversation with Carly Taylor and Joe Reis covers how Databricks is used in gaming. It is an industry discussion, so the link points to the Databricks SQL overview as the closest product reference. The discussion covers how game studios use the lakehouse for analytics and data products, not a specific feature. The SQL overview linked below is the general reference for the analytics layer used in those workloads. Official docs: Introduces Databricks SQL, the lakehouse data warehouse for queries, monitoring and BI.

📚 Docs: https://docs.databricks.com/aws/en/sql/
🔗 More: https://www.nextgenlakehouse.com/answers/platform

#Databricks #Gaming #DataEngineering #Lakehouse #JoeReis #Industry #DataAndAI

— Pinned comment —
How gaming companies use Databricks, from the industry view. The SQL overview and our platform hub are linked below. Docs: https://docs.databricks.com/aws/en/sql/ Hub: https://www.nextgenlakehouse.com/answers/platform

Invitation au Data+AI World Tour Paris

This is a French-language invitation to a Databricks Data and AI World Tour event in Paris. It is an announcement, so the link points to the Databricks SQL overview as the general product reference. The event covers Databricks data and AI product updates for the French market. The SQL overview linked below is the general reference for the analytics area covered there. Official docs: Introduces Databricks SQL, the lakehouse data warehouse for queries, monitoring and BI. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/sql/
🔗 More: https://www.nextgenlakehouse.com/answers/platform

#Databricks #DataAndAI #WorldTour #Paris #Event #DataEngineering #Lakehouse

— Pinned comment —
Invitation au Data+AI World Tour Paris. The SQL overview and our platform hub are linked below. Docs: https://docs.databricks.com/aws/en/sql/ Hub: https://www.nextgenlakehouse.com/answers/platform

Getting Started with Databricks Serverless Workspaces

Serverless workspaces run Databricks compute without managing clusters, so notebooks and jobs start quickly. This video covers how to get started with serverless in a new workspace. The serverless compute page introduces Databricks-managed serverless compute for notebooks, workflows and Lakeflow pipelines, and links to setup guides. The serverless limitations page covers current restrictions. Official docs: Introduces Databricks-managed serverless compute for notebooks, workflows and Lakeflow pipelines. Lists current restrictions on serverless compute for notebooks and jobs, including UDFs and streaming triggers.

📚 Docs: https://docs.databricks.com/aws/en/compute/serverless/ https://docs.databricks.com/aws/en/compute/serverless/limitations
🔗 More: https://www.nextgenlakehouse.com/answers/platform

#Databricks #Serverless #Workspaces #DataEngineering #Lakeflow #Notebooks #Compute

— Pinned comment —
Getting started with serverless workspaces. The serverless compute docs and our platform hub are linked below. Docs: https://docs.databricks.com/aws/en/compute/serverless/ Hub: https://www.nextgenlakehouse.com/answers/platform

Inside the Data and AI Summit: Insights from Databricks MVP Josue Bogran

Databricks MVP Josue Bogran shares his takeaways from the Data and AI Summit. It is a community perspective, so the link points to the Databricks SQL overview as the general product reference. The conversation covers Summit themes and community perspectives on the Databricks platform. The SQL overview linked below is the general reference for the analytics product area discussed. Official docs: Introduces Databricks SQL, the lakehouse data warehouse for queries, monitoring and BI. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/sql/
🔗 More: https://www.nextgenlakehouse.com/answers/platform

#Databricks #DataAndAISummit #MVP #Community #DataEngineering #Lakehouse #DataAndAI

— Pinned comment —
A community view of the Data and AI Summit from an MVP. The SQL overview and our platform hub are linked below. Docs: https://docs.databricks.com/aws/en/sql/ Hub: https://www.nextgenlakehouse.com/answers/platform

Why the Databricks Community Matters

This video explains why the Databricks community matters to data practitioners. It is a community perspective, so the link points to the Databricks SQL overview as the general product reference. The discussion covers learning, sharing and contributing in the Databricks community. The SQL overview linked below is the general reference for the product area most community questions cover. Official docs: Introduces Databricks SQL, the lakehouse data warehouse for queries, monitoring and BI. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/sql/
🔗 More: https://www.nextgenlakehouse.com/answers/platform

#Databricks #Community #DataEngineering #Lakehouse #Learning #DataAndAI #Career

— Pinned comment —
Why the Databricks community matters. The SQL overview and our platform hub are linked below. Docs: https://docs.databricks.com/aws/en/sql/ Hub: https://www.nextgenlakehouse.com/answers/platform

Data+AI Summit 2025 Impressions

This video shares impressions from the Data and AI Summit 2025. It is an event recap, so the link points to the Databricks SQL overview as the general product reference. The recap covers Summit sessions and product themes from the perspective of a Databricks advocate. The SQL overview linked below is the general reference for the analytics area discussed. Official docs: Introduces Databricks SQL, the lakehouse data warehouse for queries, monitoring and BI. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/sql/
🔗 More: https://www.nextgenlakehouse.com/answers/platform

#Databricks #DataAndAISummit #DAIS2025 #Event #DataEngineering #Lakehouse #DataAndAI

— Pinned comment —
Impressions from the Data and AI Summit 2025. The SQL overview and our platform hub are linked below. Docs: https://docs.databricks.com/aws/en/sql/ Hub: https://www.nextgenlakehouse.com/answers/platform

Data and AI Summit 2025 Highlights

This video highlights the main announcements from the Data and AI Summit 2025. It is a recap, so the link points to the Databricks SQL overview as the general product reference. The highlights span data engineering, analytics and AI across the Summit keynotes and sessions. The SQL overview linked below is the general reference for the analytics area covered. Official docs: Introduces Databricks SQL, the lakehouse data warehouse for queries, monitoring and BI. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/sql/
🔗 More: https://www.nextgenlakehouse.com/answers/platform

#Databricks #DataAndAISummit #DAIS2025 #Highlights #DataEngineering #Lakehouse #DataAndAI

— Pinned comment —
Highlights from the Data and AI Summit 2025. The SQL overview and our platform hub are linked below. Docs: https://docs.databricks.com/aws/en/sql/ Hub: https://www.nextgenlakehouse.com/answers/platform

Databricks MVPs at Data+AI Summit 2025

Databricks MVPs share their Data and AI Summit 2025 experience in this video. It is a community feature, so the link points to the Databricks SQL overview as the general product reference. The video covers community perspectives on Summit sessions and product themes. The SQL overview linked below is the general reference for the analytics area discussed. Official docs: Introduces Databricks SQL, the lakehouse data warehouse for queries, monitoring and BI. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/sql/
🔗 More: https://www.nextgenlakehouse.com/answers/platform

#Databricks #MVP #DataAndAISummit #DAIS2025 #Community #DataEngineering #Lakehouse

— Pinned comment —
Databricks MVPs on the Data and AI Summit 2025. The SQL overview and our platform hub are linked below. Docs: https://docs.databricks.com/aws/en/sql/ Hub: https://www.nextgenlakehouse.com/answers/platform

Data and AI Summit 2025 Day 4 Highlights

This video recaps day 4 of the Data and AI Summit 2025. It is an event recap, so the link points to the Databricks SQL overview as the general product reference. The day 4 recap covers Summit sessions and product themes. The SQL overview linked below is the general reference for the analytics area discussed. Official docs: Introduces Databricks SQL, the lakehouse data warehouse for queries, monitoring and BI. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/sql/
🔗 More: https://www.nextgenlakehouse.com/answers/platform

#Databricks #DataAndAISummit #DAIS2025 #Highlights #DataEngineering #Lakehouse #DataAndAI

— Pinned comment —
Day 4 of the Data and AI Summit 2025. The SQL overview and our platform hub are linked below. Docs: https://docs.databricks.com/aws/en/sql/ Hub: https://www.nextgenlakehouse.com/answers/platform

Back From Summit: France 2025

This video shares what the French team brought back from the Data and AI Summit 2025. It is a recap, so the link points to the Databricks SQL overview as the general product reference. The recap covers Summit takeaways for the French data community. The SQL overview linked below is the general reference for the analytics area discussed. Official docs: Introduces Databricks SQL, the lakehouse data warehouse for queries, monitoring and BI. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/sql/
🔗 More: https://www.nextgenlakehouse.com/answers/platform

#Databricks #DataAndAISummit #DAIS2025 #France #DataEngineering #Lakehouse #DataAndAI

— Pinned comment —
Takeaways from the Summit for France in 2025. The SQL overview and our platform hub are linked below. Docs: https://docs.databricks.com/aws/en/sql/ Hub: https://www.nextgenlakehouse.com/answers/platform

Data and AI Summit 2025 Day 3 Highlights

This video recaps day 3 of the Data and AI Summit 2025. It is an event recap, so the link points to the Databricks SQL overview as the general product reference. The day 3 recap covers Summit sessions and product themes. The SQL overview linked below is the general reference for the analytics area discussed. Official docs: Introduces Databricks SQL, the lakehouse data warehouse for queries, monitoring and BI. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/sql/
🔗 More: https://www.nextgenlakehouse.com/answers/platform

#Databricks #DataAndAISummit #DAIS2025 #Highlights #DataEngineering #Lakehouse #DataAndAI

— Pinned comment —
Day 3 of the Data and AI Summit 2025. The SQL overview and our platform hub are linked below. Docs: https://docs.databricks.com/aws/en/sql/ Hub: https://www.nextgenlakehouse.com/answers/platform

Databricks and Microsoft Extend Strategic Partnership for Azure Databricks

Databricks and Microsoft extend their strategic partnership for Azure Databricks. This is a partnership announcement, so the link points to the networking docs as the closest verified platform reference. The partnership announcement covers the Azure Databricks offering and its integrations. Azure-specific setup is documented separately, and the networking page linked below is the closest verified platform reference for enterprise deployments. Official docs: Explains how Databricks secures user access, control plane and classic compute connectivity, and serverless data access. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/security/network/
🔗 More: https://www.nextgenlakehouse.com/answers/platform

#Databricks #Azure #Microsoft #Partnership #AzureDatabricks #CloudData #DataAndAI

— Pinned comment —
The Databricks and Microsoft partnership for Azure Databricks. The networking docs and our platform hub are linked below. Docs: https://docs.databricks.com/aws/en/security/network/ Hub: https://www.nextgenlakehouse.com/answers/platform

Databricks Setup Made Easy: Launch Your Environment in 120 Seconds

This video shows how to launch a Databricks environment in about 120 seconds. The Free Edition page covers the no-cost account option for learning and experimentation. The Free Edition page explains how to sign up for the no-cost account and what it allows. Use it as the setup reference for the quick-start shown in the video. Official docs: Explains how to sign up for the no-cost Databricks Free Edition for learning and experimentation. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/getting-started/free-edition
🔗 More: https://www.nextgenlakehouse.com/answers/platform

#Databricks #Setup #FreeEdition #Lakehouse #DataEngineering #Learning #Quickstart

— Pinned comment —
A fast Databricks environment setup is shown here. The Free Edition docs and our platform hub are linked below. Docs: https://docs.databricks.com/aws/en/getting-started/free-edition Hub: https://www.nextgenlakehouse.com/answers/platform

Data and AI Summit 2025 Day 2 Highlights

This video recaps day 2 of the Data and AI Summit 2025. It is an event recap, so the link points to the Databricks SQL overview as the general product reference. The day 2 recap covers Summit sessions and product themes. The SQL overview linked below is the general reference for the analytics area discussed. Official docs: Introduces Databricks SQL, the lakehouse data warehouse for queries, monitoring and BI. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/sql/
🔗 More: https://www.nextgenlakehouse.com/answers/platform

#Databricks #DataAndAISummit #DAIS2025 #Highlights #DataEngineering #Lakehouse #DataAndAI

— Pinned comment —
Day 2 of the Data and AI Summit 2025. The SQL overview and our platform hub are linked below. Docs: https://docs.databricks.com/aws/en/sql/ Hub: https://www.nextgenlakehouse.com/answers/platform

Databricks Free Edition

Databricks Free Edition is a no-cost way to learn and experiment with the platform. This video introduces the edition and what you can do with it. The Free Edition page explains how to sign up for the no-cost account and what it allows for personal learning and experimentation. Use it as the reference for the features covered in the video. Official docs: Explains how to sign up for the no-cost Databricks Free Edition for learning and experimentation. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/getting-started/free-edition
🔗 More: https://www.nextgenlakehouse.com/answers/platform

#Databricks #FreeEdition #Learning #DataEngineering #Lakehouse #AI #Developers

— Pinned comment —
An introduction to Databricks Free Edition. The Free Edition docs and our platform hub are linked below. Docs: https://docs.databricks.com/aws/en/getting-started/free-edition Hub: https://www.nextgenlakehouse.com/answers/platform

Data and AI Summit 2025 Day 1 Highlights

This video recaps day 1 of the Data and AI Summit 2025. It is an event recap, so the link points to the Databricks SQL overview as the general product reference. The day 1 recap covers Summit sessions and product themes. The SQL overview linked below is the general reference for the analytics area discussed. Official docs: Introduces Databricks SQL, the lakehouse data warehouse for queries, monitoring and BI. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/sql/
🔗 More: https://www.nextgenlakehouse.com/answers/platform

#Databricks #DataAndAISummit #DAIS2025 #Highlights #DataEngineering #Lakehouse #DataAndAI

— Pinned comment —
Day 1 of the Data and AI Summit 2025. The SQL overview and our platform hub are linked below. Docs: https://docs.databricks.com/aws/en/sql/ Hub: https://www.nextgenlakehouse.com/answers/platform

Data and AI Summit 2025 Backstage Footage

This video shows behind-the-scenes footage from the Data and AI Summit 2025. It is not a product how-to, so the link points to the Databricks SQL overview as the general reference. The footage covers the event atmosphere and sessions rather than a specific feature. The SQL overview linked below is the general reference for the product area discussed at the Summit. Official docs: Introduces Databricks SQL, the lakehouse data warehouse for queries, monitoring and BI. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/sql/
🔗 More: https://www.nextgenlakehouse.com/answers/platform

#Databricks #DataAndAISummit #DAIS2025 #Backstage #DataEngineering #Lakehouse #DataAndAI

— Pinned comment —
Backstage at the Data and AI Summit 2025. The SQL overview and our platform hub are linked below. Docs: https://docs.databricks.com/aws/en/sql/ Hub: https://www.nextgenlakehouse.com/answers/platform

Databricks Data+AI Platform Overview (Arabic)

This Arabic-language video gives an overview of the Databricks Data and AI platform. The Unity Catalog overview linked below is the governance reference for the platform's core layer. The platform overview covers data engineering, analytics and AI on one lakehouse, with governance across them. The Unity Catalog page covers the access control and lineage layer that ties these workloads together. Official docs: Introduces Unity Catalog, the governance layer covering access control, lineage and auditing. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/data-governance/unity-catalog/
🔗 More: https://www.nextgenlakehouse.com/answers/platform

#Databricks #Lakehouse #DataAndAI #Arabic #DataEngineering #Platform #UnityCatalog

— Pinned comment —
Overview de la plateforme Databricks, en arabe. Unity Catalog docs and our platform hub are linked below. Docs: https://docs.databricks.com/aws/en/data-governance/unity-catalog/ Hub: https://www.nextgenlakehouse.com/answers/platform

Databricks Data+AI Summit 2025 Invitation

This is an invitation to the Databricks Data and AI Summit 2025. It is an event announcement, so the link points to the Databricks SQL overview as the general product reference. The Summit covers Databricks product directions across data engineering, analytics and AI. The SQL overview linked below is the general reference for the analytics area. Official docs: Introduces Databricks SQL, the lakehouse data warehouse for queries, monitoring and BI. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/sql/
🔗 More: https://www.nextgenlakehouse.com/answers/platform

#Databricks #DataAndAISummit #DAIS2025 #Event #DataEngineering #Lakehouse #DataAndAI

— Pinned comment —
An invitation to the Data and AI Summit 2025. The SQL overview and our platform hub are linked below. Docs: https://docs.databricks.com/aws/en/sql/ Hub: https://www.nextgenlakehouse.com/answers/platform

Meet a Databricks MVP: Scott Haines

This video introduces Databricks MVP Scott Haines and his work with the community. It is a profile, so the link points to the Databricks SQL overview as the general product reference. The profile covers the MVP's background and community contributions around Databricks. The SQL overview linked below is the general reference for the analytics product area they often discuss. Official docs: Introduces Databricks SQL, the lakehouse data warehouse for queries, monitoring and BI. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/sql/
🔗 More: https://www.nextgenlakehouse.com/answers/platform

#Databricks #MVP #Community #DataEngineering #Lakehouse #Profile #DataAndAI

— Pinned comment —
Meet Databricks MVP Scott Haines. The SQL overview and our platform hub are linked below. Docs: https://docs.databricks.com/aws/en/sql/ Hub: https://www.nextgenlakehouse.com/answers/platform

Meet a Databricks MVP: Andrew Sitz

This video introduces Databricks MVP Andrew Sitz and his work with the community. It is a profile, so the link points to the Databricks SQL overview as the general product reference. The profile covers the MVP's background and community contributions around Databricks. The SQL overview linked below is the general reference for the analytics product area they often discuss. Official docs: Introduces Databricks SQL, the lakehouse data warehouse for queries, monitoring and BI. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/sql/
🔗 More: https://www.nextgenlakehouse.com/answers/platform

#Databricks #MVP #Community #DataEngineering #Lakehouse #Profile #DataAndAI

— Pinned comment —
Meet Databricks MVP Andrew Sitz. The SQL overview and our platform hub are linked below. Docs: https://docs.databricks.com/aws/en/sql/ Hub: https://www.nextgenlakehouse.com/answers/platform

Deploy a Databricks Workspace Behind a Firewall

This video shows how to deploy a Databricks workspace behind a firewall, with private networking and controlled connectivity. The networking page explains the connectivity model. The networking page covers how Databricks secures user access to workspaces, control plane and classic compute connectivity, and serverless access to data sources. It is the reference for the firewall and private connectivity setup in the video. Official docs: Explains how Databricks secures user access, control plane and classic compute connectivity, and serverless data access.

📚 Docs: https://docs.databricks.com/aws/en/security/network/
🔗 More: https://www.nextgenlakehouse.com/answers/platform

#Databricks #Networking #Firewall #PrivateLink #Security #CloudInfrastructure #Admin

— Pinned comment —
Deploying a Databricks workspace behind a firewall is the topic. The networking docs and our platform hub are linked below. Docs: https://docs.databricks.com/aws/en/security/network/ Hub: https://www.nextgenlakehouse.com/answers/platform

Meet a Databricks MVP: Rajaniesh Kaushikk

This video introduces Databricks MVP Rajaniesh Kaushikk and his community work. It is a profile, so the link points to the Databricks SQL overview as the general product reference. The profile covers the MVP's background and contributions around Databricks. The SQL overview linked below is the general reference for the analytics product area they often discuss. Official docs: Introduces Databricks SQL, the lakehouse data warehouse for queries, monitoring and BI. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/sql/
🔗 More: https://www.nextgenlakehouse.com/answers/platform

#Databricks #MVP #Community #DataEngineering #Lakehouse #Profile #DataAndAI

— Pinned comment —
Meet Databricks MVP Rajaniesh Kaushikk. The SQL overview and our platform hub are linked below. Docs: https://docs.databricks.com/aws/en/sql/ Hub: https://www.nextgenlakehouse.com/answers/platform

Mastering Unity Catalog Compute Architecture

Unity Catalog compute architecture covers how clusters, SQL warehouses and serverless compute access governed data. This video explains the architecture and its access model. The compute overview describes the serverless, classic and SQL warehouse options and how they fit data engineering, data science and analytics. Pair it with the Unity Catalog overview for the access model. Official docs: Overview of compute options for data engineering, data science and analytics, including serverless and SQL warehouses. Introduces Unity Catalog, the governance layer covering access control, lineage and auditing.

📚 Docs: https://docs.databricks.com/aws/en/compute/ https://docs.databricks.com/aws/en/data-governance/unity-catalog/
🔗 More: https://www.nextgenlakehouse.com/answers/platform

#Databricks #UnityCatalog #Compute #Clusters #SQLWarehouse #Architecture #DataGovernance

— Pinned comment —
Compute architecture with Unity Catalog is the topic. The compute docs and our platform hub are linked below. Docs: https://docs.databricks.com/aws/en/compute/ Hub: https://www.nextgenlakehouse.com/answers/platform

Meet a Databricks MVP: Daniel Sahal

This video introduces Databricks MVP Daniel Sahal and his community work. It is a profile, so the link points to the Databricks SQL overview as the general product reference. The profile covers the MVP's background and contributions around Databricks. The SQL overview linked below is the general reference for the analytics product area they often discuss. Official docs: Introduces Databricks SQL, the lakehouse data warehouse for queries, monitoring and BI. Watch the full walkthrough on the NextGenLakehouse channel.

📚 Docs: https://docs.databricks.com/aws/en/sql/
🔗 More: https://www.nextgenlakehouse.com/answers/platform

#Databricks #MVP #Community #DataEngineering #Lakehouse #Profile #DataAndAI

— Pinned comment —
Meet Databricks MVP Daniel Sahal. The SQL overview and our platform hub are linked below. Docs: https://docs.databricks.com/aws/en/sql/ Hub: https://www.nextgenlakehouse.com/answers/platform

Databricks Observability and Cost Tracking: Future Roadmap with Greg Kroleski

Databricks observability and cost tracking rely on system tables, which record billing, usage and compute data in Unity Catalog. This conversation covers where that tooling is heading. The billing system table reports usage by SKU and workspace, which is the starting point for chargeback and FinOps dashboards. Browse the system tables reference for the other observability tables. Observability questions such as which jobs or warehouses drive spend are answered from the system tables rather than the UI alone. Build a dashboard on top to share the results with finance and platform teams. 📚 Docs: https://docs.databricks.com/aws/en/admin/system-tables/billing and https://docs.databricks.com/aws/en/admin/system-tables/. 🔗 More: https://www.nextgenlakehouse.com/learn (Platform hub). #Databricks #Observability #FinOps #SystemTables #Lakehouse #DataAI #DataIntelligence #DataAnalytics

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/admin/system-tables/billing More Platform content lives at https://www.nextgenlakehouse.com/learn

Cut Your Databricks Costs: Ultimate Guide to Cost Control with System Tables and Budgets

Cut Databricks costs by tracking usage with system tables and attributing serverless spend with usage policies. This guide covers the main cost-control levers. The billing system table gives per-SKU usage you can query, and serverless budget policies tag usage so you can attribute it to teams or projects. Start with the billing table, then set up policies for serverless workloads. Start with the largest cost drivers in the billing data, then set budget policies and right-size compute to match the workload. Review the results monthly so savings stick. 📚 Docs: https://docs.databricks.com/aws/en/admin/system-tables/billing and https://docs.databricks.com/aws/en/admin/usage/budget-policies. 🔗 More: https://www.nextgenlakehouse.com/learn (Platform hub). #Databricks #FinOps #CostControl #SystemTables #Lakehouse #DataAI #DataIntelligence #DataAnalytics

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/admin/system-tables/billing More Platform content lives at https://www.nextgenlakehouse.com/learn

Meet Databricks MVP Ajay Pandey: Community Expert Interview

This video introduces Databricks MVP Ajay Pandey and his work with the Databricks community. It is a community interview rather than a feature tutorial. For official Databricks product documentation, start from the Databricks partner and learning pages linked below. Community interviews show how practitioners use Databricks day to day, which complements the product docs. Look for the MVP's own posts and talks for hands-on detail on their work. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://www.databricks.com/partners. 🔗 More: https://www.nextgenlakehouse.com/learn (Platform hub). #Databricks #DatabricksMVP #DataCommunity #Lakehouse #DataAI #DataIntelligence #DataAnalytics #DataEngineering

— Pinned comment —
Official reference: https://www.databricks.com/partners More Platform content lives at https://www.nextgenlakehouse.com/learn

The Future of Data and AI Platforms with Adam Conway

Adam Conway discusses where data intelligence platforms are heading, with Databricks' platform as the reference point. This interview covers data, AI and governance in one system. The Databricks product overview describes the Data and AI Platform, which unifies data engineering, analytics and AI on Unity Catalog governance. The conversation connects product strategy with what customers build, which is useful context when comparing platform approaches. Pair it with the platform overview for current capabilities. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://www.databricks.com/product/platform. 🔗 More: https://www.nextgenlakehouse.com/learn (Platform hub). #Databricks #DataIntelligence #AI #Lakehouse #DataAI #DataAnalytics #DataEngineering #DataGovernance

— Pinned comment —
Official reference: https://www.databricks.com/product/platform More Platform content lives at https://www.nextgenlakehouse.com/learn

Meet Databricks MVP Eddie Edgeworth: Community Expert Interview

This video is a Databricks MVP interview with Eddie Edgeworth about his work and experience with the platform. It is a community profile, not a product walkthrough. For official product documentation, use the Databricks partner and learning pages linked below. Community profiles like this give a practitioner view of how the MVP uses Databricks in real work. Check the video description for links to the MVP's own write-ups and talks. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://www.databricks.com/partners. 🔗 More: https://www.nextgenlakehouse.com/learn (Platform hub). #Databricks #DatabricksMVP #DataCommunity #Lakehouse #DataAI #DataIntelligence #DataAnalytics #DataEngineering

— Pinned comment —
Official reference: https://www.databricks.com/partners More Platform content lives at https://www.nextgenlakehouse.com/learn

Meet Databricks MVP Rishabh Pandey: Community Expert Interview

This video is a Databricks MVP interview with Rishabh Pandey about his experience with the platform. It is a community profile rather than a feature tutorial. For official product documentation, use the Databricks partner and learning pages linked below. Community profiles like this give a practitioner view of how the MVP uses Databricks in real work. Check the video description for links to the MVP's own write-ups and talks. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://www.databricks.com/partners. 🔗 More: https://www.nextgenlakehouse.com/learn (Platform hub). #Databricks #DatabricksMVP #DataCommunity #Lakehouse #DataAI #DataIntelligence #DataAnalytics #DataEngineering

— Pinned comment —
Official reference: https://www.databricks.com/partners More Platform content lives at https://www.nextgenlakehouse.com/learn

From Berkeley to a $60B Data+AI Company: Arsalan Tavakoli on Databricks' Story

This interview traces Databricks' path from its Berkeley roots to becoming a large Data+AI company. It is a company and strategy conversation rather than a product tutorial. The Databricks product page describes the Data and AI Platform that the company sells today. Company history talks give context for how the platform's roadmap evolved, which helps when reading product announcements. Use the product page for current capabilities, since the interview reflects when it was recorded. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://www.databricks.com/product/platform. 🔗 More: https://www.nextgenlakehouse.com/learn (Platform hub). #Databricks #DataAI #Startups #DataIntelligence #Lakehouse #DataAnalytics #DataEngineering #AI

— Pinned comment —
Official reference: https://www.databricks.com/product/platform More Platform content lives at https://www.nextgenlakehouse.com/learn

Databricks Serverless Explained: Performance, Cost Control and the Data Ecosystem

Databricks Serverless compute removes cluster management and bills for the compute you use, which affects both performance and cost. This video covers the trade-offs. Serverless budget policies let you tag serverless usage for cost attribution. The serverless compute docs explain which workloads run on it. Serverless removes many cluster sizing decisions, so the main levers become workload choice and budget policies. Compare serverless and classic compute cost for your own jobs before standardising. 📚 Docs: https://docs.databricks.com/aws/en/compute/serverless/ and https://docs.databricks.com/aws/en/admin/usage/budget-policies. 🔗 More: https://www.nextgenlakehouse.com/learn (Platform hub). #Databricks #Serverless #Performance #CostControl #Lakehouse #DataAI #DataIntelligence #DataAnalytics

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/compute/serverless/ More Platform content lives at https://www.nextgenlakehouse.com/learn

Databricks and Salesforce: The Future of Customer Data on the Lakehouse

Databricks and Salesforce data can be combined so customer data is analysed alongside the lakehouse. This video explores the future of customer data with this integration. Lakehouse Federation lets you query external systems from Unity Catalog without copying data first. The federation overview is the starting point for connecting external sources. Customer data work usually starts with identity resolution and a clear picture of which systems hold each attribute. Map those sources first, then decide which to federate and which to ingest. 📚 Docs: https://docs.databricks.com/aws/en/query-federation/. 🔗 More: https://www.nextgenlakehouse.com/learn (Platform hub). #Databricks #Salesforce #CustomerData #CDP #Lakehouse #DataAI #DataIntelligence #DataAnalytics

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/query-federation/ More Platform content lives at https://www.nextgenlakehouse.com/learn

Franco Patano on Data+AI Summit 2024: Key Takeaways

Franco Patano shares his takeaways from Data+AI Summit 2024 in this conversation. It is an event recap rather than a product tutorial. The Data+AI Summit agenda page lists the sessions and announcements from the event. Event interviews reflect the speakers' views at the time, so verify product details against current docs. The conversation is most useful for context on priorities and where the platform was heading in 2024. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://www.databricks.com/dataaisummit/agenda. 🔗 More: https://www.nextgenlakehouse.com/learn (Platform hub). #Databricks #DataAISummit #DataAI #Event #Lakehouse #DataIntelligence #DataAnalytics #DataEngineering

— Pinned comment —
Official reference: https://www.databricks.com/dataaisummit/agenda More Platform content lives at https://www.nextgenlakehouse.com/learn

My Interview at Data+AI Summit 2024: Databricks Community Perspective

This video is a personal interview from Data+AI Summit 2024 about the event and the Databricks community. It is not a product tutorial. The Data+AI Summit agenda page lists the sessions from that year. Personal interviews from the event give a community view, and the agenda is the best place to find the technical sessions they reference. This makes the video a companion to the session list rather than a standalone tutorial. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://www.databricks.com/dataaisummit/agenda. 🔗 More: https://www.nextgenlakehouse.com/learn (Platform hub). #Databricks #DataAISummit #DataCommunity #Event #Lakehouse #DataAI #DataIntelligence #DataAnalytics

— Pinned comment —
Official reference: https://www.databricks.com/dataaisummit/agenda More Platform content lives at https://www.nextgenlakehouse.com/learn

FinOps in Databricks: Cost Tracking Strategies from Data+AI Summit 2024

FinOps on Databricks means tracking cost by workload and team using system tables and serverless budget policies. This Data+AI Summit 2024 edition covers those practices. The billing system table is the source for per-SKU usage, and serverless usage policies tag spend for attribution. The billing table lets you allocate spend to teams, jobs and workspaces, the foundation for any FinOps review. Use it with the budget policy doc to tag serverless spend by project. 📚 Docs: https://docs.databricks.com/aws/en/admin/system-tables/billing and https://docs.databricks.com/aws/en/admin/usage/budget-policies. 🔗 More: https://www.nextgenlakehouse.com/learn (Platform hub). #Databricks #FinOps #CostTracking #SystemTables #Lakehouse #DataAI #DataIntelligence #DataAnalytics

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/admin/system-tables/billing More Platform content lives at https://www.nextgenlakehouse.com/learn

Expert Analysis of Data+AI Summit 2024: Databricks Announcements Explained

This expert analysis breaks down the main announcements from Data+AI Summit 2024. It is an event commentary rather than a product tutorial. The Data+AI Summit agenda page lists the sessions that the analysis covers. Expert analysis is most useful alongside the official announcements, which carry the exact product details. Read the session pages for the technical detail behind each point the analysis raises. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://www.databricks.com/dataaisummit/agenda. 🔗 More: https://www.nextgenlakehouse.com/learn (Platform hub). #Databricks #DataAISummit #DataAI #Analysis #Lakehouse #DataIntelligence #DataAnalytics #DataEngineering

— Pinned comment —
Official reference: https://www.databricks.com/dataaisummit/agenda More Platform content lives at https://www.nextgenlakehouse.com/learn

Data+AI Summit 2024 Day 4 Highlights: Databricks Announcements Recap

This video recaps the Day 4 highlights of Data+AI Summit 2024, covering the announcements and sessions from that day. It is an event recap. The Data+AI Summit agenda page lists the full session schedule for the event. Day recaps show which announcements mattered most, and the agenda leads to the full sessions. Use the timestamps to jump to the announcements that matter for your team, and check current docs before adopting any feature. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://www.databricks.com/dataaisummit/agenda. 🔗 More: https://www.nextgenlakehouse.com/learn (Platform hub). #Databricks #DataAISummit #Recap #DataAI #Lakehouse #DataIntelligence #DataAnalytics #DataEngineering

— Pinned comment —
Official reference: https://www.databricks.com/dataaisummit/agenda More Platform content lives at https://www.nextgenlakehouse.com/learn

Data+AI Summit 2024 Day 3 Highlights: Databricks Announcements Recap

This video recaps the Day 3 highlights of Data+AI Summit 2024, covering the announcements and sessions from that day. It is an event recap. The Data+AI Summit agenda page lists the full session schedule for the event. Day recaps show which announcements mattered most, and the agenda leads to the full sessions. Use the timestamps to jump to the sessions that match your roadmap. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://www.databricks.com/dataaisummit/agenda. 🔗 More: https://www.nextgenlakehouse.com/learn (Platform hub). #Databricks #DataAISummit #Recap #DataAI #Lakehouse #DataIntelligence #DataAnalytics #DataEngineering

— Pinned comment —
Official reference: https://www.databricks.com/dataaisummit/agenda More Platform content lives at https://www.nextgenlakehouse.com/learn

Data+AI Summit 2024 Day 1 Highlights: Databricks Announcements Recap

This video recaps the Day 1 highlights of Data+AI Summit 2024, covering the keynote announcements and early sessions. It is an event recap. The Data+AI Summit agenda page lists the full session schedule for the event. Keynote recaps summarise the biggest announcements, and the agenda and session pages hold the detail. Use the timestamps to find the announcements for your team, then check current docs before adopting any feature. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://www.databricks.com/dataaisummit/agenda. 🔗 More: https://www.nextgenlakehouse.com/learn (Platform hub). #Databricks #DataAISummit #Recap #DataAI #Lakehouse #DataIntelligence #DataAnalytics #DataEngineering

— Pinned comment —
Official reference: https://www.databricks.com/dataaisummit/agenda More Platform content lives at https://www.nextgenlakehouse.com/learn

Josue A. Bogran Interview at Data+AI Summit 2024

This Data+AI Summit 2024 interview with Josue A. Bogran covers his perspective on the event and Databricks. It is a community interview rather than a product tutorial. The Data+AI Summit agenda page lists the sessions from that year. Community interviews reflect individual experience, so treat the stories as examples rather than product guidance. Look up the sessions the interviewee mentions and cross-check details against current docs. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://www.databricks.com/dataaisummit/agenda. 🔗 More: https://www.nextgenlakehouse.com/learn (Platform hub). #Databricks #DataAISummit #Interview #DataCommunity #Lakehouse #DataAI #DataIntelligence #DataAnalytics

— Pinned comment —
Official reference: https://www.databricks.com/dataaisummit/agenda More Platform content lives at https://www.nextgenlakehouse.com/learn

Data+AI Summit 2024 Day 2 Highlights: Databricks Announcements Recap

This video recaps the Day 2 highlights of Data+AI Summit 2024, covering the announcements and sessions from that day. It is an event recap. The Data+AI Summit agenda page lists the full session schedule for the event. Day recaps show which announcements mattered most, and the agenda leads to the full sessions. Use the timestamps to jump to the sessions that match your roadmap. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://www.databricks.com/dataaisummit/agenda. 🔗 More: https://www.nextgenlakehouse.com/learn (Platform hub). #Databricks #DataAISummit #Recap #DataAI #Lakehouse #DataIntelligence #DataAnalytics #DataEngineering

— Pinned comment —
Official reference: https://www.databricks.com/dataaisummit/agenda More Platform content lives at https://www.nextgenlakehouse.com/learn

Databricks Data+AI Summit 2024 Invitation

This video is an invitation to Databricks Data+AI Summit 2024. It is an event promotion rather than a product tutorial. The Data+AI Summit agenda page lists the sessions and registration details for the event. Invitations like this one are event promotions, so check the official event page for dates and registration. The agenda page lists the sessions once they are published, which is the best way to plan attendance. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://www.databricks.com/dataaisummit/agenda. 🔗 More: https://www.nextgenlakehouse.com/learn (Platform hub). #Databricks #DataAISummit #Event #DataAI #Lakehouse #DataIntelligence #DataAnalytics #DataEngineering

— Pinned comment —
Official reference: https://www.databricks.com/dataaisummit/agenda More Platform content lives at https://www.nextgenlakehouse.com/learn

Digital Natives Podcast: David Sykes of Octopus Energy on Data and Databricks

This podcast episode features David Sykes of Octopus Energy discussing data work and Databricks. It is a customer conversation rather than a product tutorial. The Databricks platform page describes the Data and AI Platform referenced in the discussion. Customer stories show how a utility team uses Databricks in practice, which can help when making a similar case internally. Use the platform overview to check current capabilities rather than relying on the conversation. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://www.databricks.com/product/platform. 🔗 More: https://www.nextgenlakehouse.com/learn (Platform hub). #Databricks #OctopusEnergy #Podcast #DataEngineering #Lakehouse #DataAI #DataIntelligence #DataAnalytics

— Pinned comment —
Official reference: https://www.databricks.com/product/platform More Platform content lives at https://www.nextgenlakehouse.com/learn

Déploiement d'un workspace Azure Databricks (guide FR)

Deploying an Azure Databricks workspace means creating the resource and configuring networking and access. This French-language video walks through the deployment. The Azure Databricks create-workspace doc lists the steps and options for a new workspace. Choosing networking and the resource group before starting avoids rework later. Follow the create-workspace steps in order and note the settings you choose for later audits. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://learn.microsoft.com/en-us/azure/databricks/admin/workspace/create-workspace. 🔗 More: https://www.nextgenlakehouse.com/learn (Platform hub). #Databricks #AzureDatabricks #Deployment #Workspace #Lakehouse #DataAI #DataIntelligence #DataAnalytics

— Pinned comment —
Official reference: https://learn.microsoft.com/en-us/azure/databricks/admin/workspace/create-workspace More Platform content lives at https://www.nextgenlakehouse.com/learn

Getting Started with the New Databricks Notebooks Experience

The new Databricks notebooks experience changes the editor, cell UI and debugging workflow. This video helps you get started with it. The notebooks docs describe the editor features, how to run cells and how notebooks connect to compute. Learn the new cell and debugging controls in a scratch notebook before using them on shared code. The notebooks doc lists the editor features and how notebooks attach to compute. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://docs.databricks.com/aws/en/notebooks/ and https://docs.databricks.com/aws/en/notebooks/notebooks-code. 🔗 More: https://www.nextgenlakehouse.com/learn (Platform hub). #Databricks #Notebooks #Python #SQL #Lakehouse #DataAI #DataIntelligence #DataAnalytics

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/notebooks/ More Platform content lives at https://www.nextgenlakehouse.com/learn

Databricks CEO Presents the Data and AI Platform at Microsoft Ignite 2023

At Microsoft Ignite 2023, the Databricks CEO presented the Data and AI Platform on Azure. This video covers that keynote presentation. The Databricks product page describes the Data and AI Platform presented in the keynote. Keynote presentations give the vision at a point in time, so check current product docs for what is available now. The product page describes the platform as it stands today. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://www.databricks.com/product/platform. 🔗 More: https://www.nextgenlakehouse.com/learn (Platform hub). #Databricks #Azure #DataIntelligence #MicrosoftIgnite #Lakehouse #DataAI #DataAnalytics #DataEngineering

— Pinned comment —
Official reference: https://www.databricks.com/product/platform More Platform content lives at https://www.nextgenlakehouse.com/learn

Getting Started with Databricks System Tables for Billing, Audit and Usage

Databricks system tables are Unity Catalog tables that record billing, audit and compute usage across your account. This video shows how to get started querying them. The system tables reference lists the schemas, and the billing table reports usage by SKU and workspace. Start with the billing and audit tables, which answer the most common questions about cost and access. Query them with SQL or build a dashboard on top, and check the reference for retention and the access needed. 📚 Docs: https://docs.databricks.com/aws/en/admin/system-tables/ and https://docs.databricks.com/aws/en/admin/system-tables/billing. 🔗 More: https://www.nextgenlakehouse.com/learn (Platform hub). #Databricks #SystemTables #Observability #FinOps #Lakehouse #DataAI #DataIntelligence #DataAnalytics

— Pinned comment —
Official reference: https://docs.databricks.com/aws/en/admin/system-tables/ More Platform content lives at https://www.nextgenlakehouse.com/learn

Best of Data and AI Summit 2023: Databricks Highlights

This video collects the best sessions and announcements from Data+AI Summit 2023. It is an event recap rather than a product tutorial. The Data+AI Summit agenda page lists the sessions and content from the event. Best-of compilations are event recaps, so use the agenda to find the full sessions behind each clip. Check current docs before adopting any feature, since products have changed since 2023. Watch the full NextGenLakehouse video for the hands-on walkthrough, and use the linked doc as the reference for exact syntax and settings. 📚 Docs: https://www.databricks.com/dataaisummit/agenda. 🔗 More: https://www.nextgenlakehouse.com/learn (Platform hub). #Databricks #DataAISummit #Recap #DataAI #Lakehouse #DataIntelligence #DataAnalytics #DataEngineering

— Pinned comment —
Official reference: https://www.databricks.com/dataaisummit/agenda More Platform content lives at https://www.nextgenlakehouse.com/learn

Comment se former à Databricks: Databricks Academy Training Path (French)

Comment se former à Databricks? This French-language video explains how to start learning the Databricks Data and AI Platform, from lakehouse basics to hands-on data engineering and analytics. The official starting point is the Databricks training catalog, which lists self-paced courses, instructor-led classes and certification paths by role. Work through one role-based path first, then build a small project in a workspace to reinforce each module. Pick the role that matches your job, such as data engineer, analyst or ML engineer, and finish each module with a short hands-on exercise. 📚 Docs: https://www.databricks.com/learn/training/home 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #DatabricksAcademy #Formation #DataEngineering #Lakehouse #DataAI

— Pinned comment —
Official reference for this topic: https://www.databricks.com/learn/training/home More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

My Favorite Databricks Feature: A Quick Pick from Data+AI Summit

What is a favourite Databricks feature for a developer advocate? This short clip from Data+AI Summit shares a personal pick and why it stands out in day-to-day work on the platform. The title does not name the specific feature, so the official Summit page is the most reliable place to find announcements from the event. Watch to see which feature is highlighted and how it fits into a lakehouse workflow. Use the chapter markers in the video to jump to the feature demo and see how it fits your own lakehouse workflow. 📚 Docs: https://www.databricks.com/dataaisummit 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #DataAISummit #DataEngineering #Lakehouse #NextGenLakehouse #DataAI

— Pinned comment —
Official reference for this topic: https://www.databricks.com/dataaisummit More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Matei Zaharia on the Most Exciting Databricks Products

Which Databricks products are the most exciting right now? Matei Zaharia, a Databricks co-founder and creator of Apache Spark, shares the products he is most excited about at Data+AI Summit. The clip gives a founder's view of where the platform is heading across data engineering, governance and AI. Check the official product pages for the current feature set and availability of each product. Compare the products he highlights against your own roadmap, then check the release notes and docs before adopting any of them. 📚 Docs: https://www.databricks.com/dataaisummit 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #DataAISummit #ApacheSpark #LakehouseAI #DataEngineering #MateiZaharia

— Pinned comment —
Official reference for this topic: https://www.databricks.com/dataaisummit More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Data+AI Summit After Party with The Chainsmokers: Event Recap

What was the Data+AI Summit after party like? This event video captures the evening social with The Chainsmokers, part of the Data+AI Summit week. It is community and networking content rather than a product walkthrough. For the technical sessions and announcements from the same event, use the official Summit page. The clip is social content, so it is a good companion to the Summit sessions rather than a source for product facts. If you attended the Summit, this clip is a good way to remember the community side of the week, and the rest of the conference is covered in the session recordings. 📚 Docs: https://www.databricks.com/dataaisummit 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #DataAISummit #Community #Networking #NextGenLakehouse #DataAI

— Pinned comment —
Official reference for this topic: https://www.databricks.com/dataaisummit More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Data+AI Summit Day 2: Conference Vibes and Highlights

What was day two of Data+AI Summit like? This video is a quick vibe check from the conference floor, capturing the atmosphere, sessions and conversations between attendees. Use it as a way to see the event rather than as a technical reference. The official Summit page lists the sessions and product announcements from the event. Look for the sessions and expo booths you missed on the Summit page, since the video captures the mood rather than the content. Day two of the conference is when many teams pick their sessions for the rest of the event, so use the clip to get a feel for the pace of the programme. 📚 Docs: https://www.databricks.com/dataaisummit 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #DataAISummit #Conference #DataAI #NextGenLakehouse #DataAI

— Pinned comment —
Official reference for this topic: https://www.databricks.com/dataaisummit More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

World Cup Games at Data+AI Summit: Event Highlights

What were the World Cup games at Data+AI Summit? This event video shows the social games set up for attendees during the conference. It is community content and does not cover a Databricks product. For the technical content from the same event, see the official Summit page. It captures the social side of the conference, so treat it as event context rather than technical content. Social activities like this one help attendees meet each other between sessions, which many people value as much as the technical content. 📚 Docs: https://www.databricks.com/dataaisummit 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #DataAISummit #Community #WorldCup #NextGenLakehouse #DataAI

— Pinned comment —
Official reference for this topic: https://www.databricks.com/dataaisummit More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

DevConnect Meetup at Data+AI Summit: Developer Community Recap

What is DevConnect at Data+AI Summit? DevConnect is the developer community meetup held alongside the conference, where engineers connect with Databricks staff and each other. This video is a recap of the meetup. It is community content rather than a product walkthrough, so use the Summit page for the sessions and product news. Developer meetups like this one are a good way to meet Databricks engineers and advocates in person. Developer meetups give you a chance to ask engineers direct questions about features you are building, which is harder to do in large keynotes. 📚 Docs: https://www.databricks.com/dataaisummit 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #DevConnect #DataAISummit #DeveloperCommunity #NextGenLakehouse #Lakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/dataaisummit More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

DevConnect Meetup Recap: Developer Community at Data+AI Summit

What happened at the DevConnect meetup? This video covers another DevConnect session held during Data+AI Summit, focused on building community among developers who use Databricks. It is a community recap rather than a technical tutorial. Check the Summit page for the sessions and product announcements from the conference. Watch for the community discussion that follows the talks, which often shows the questions developers are asking right now. If you are a developer who builds on Databricks, meetups like this are a practical way to find peers, hear from the team and get feedback on your projects. 📚 Docs: https://www.databricks.com/dataaisummit 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #DevConnect #DataAISummit #DeveloperCommunity #NextGenLakehouse #Lakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/dataaisummit More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

DevConnect Meetup at Data+AI Summit: Developer Networking Recap

What does a DevConnect meetup involve? This Data+AI Summit recap shows a developer meetup that brings practitioners together around Databricks. It is networking content, so it does not provide technical steps. For product documentation, use the official Databricks documentation linked below. Meetups like this one connect practitioners who are building on Databricks, which helps when you need peer advice on a real project. Practitioners who join meetups often exchange working patterns and code snippets, which can shorten the time you spend on your next project. 📚 Docs: https://www.databricks.com/dataaisummit 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #DevConnect #DataAISummit #DeveloperCommunity #NextGenLakehouse #Lakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/dataaisummit More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

How to Get a Discount on Data+AI Summit Registration

How do you get a discount on Data+AI Summit? This video explains how to access a registration discount for the conference. Discount terms and pricing change each year, so confirm the current offer on the official Summit page before you register. The Summit page also lists the dates, tracks and sessions. Registration offers change, so verify the current price and eligibility before you rely on any discount. Budget for the conference early, because the best sessions can fill up and the cost is easier to justify when you plan the agenda in advance. 📚 Docs: https://www.databricks.com/dataaisummit 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #DataAISummit #Conference #DataAI #NextGenLakehouse #DataAI

— Pinned comment —
Official reference for this topic: https://www.databricks.com/dataaisummit More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Data+AI Summit 2026 Invitation: Why You Should Attend

Why should you attend Data+AI Summit 2026? This invitation video sets out what the conference offers data engineers, analysts and AI practitioners, including technical sessions, hands-on content and product announcements. Use the official Summit page to check dates, registration options and the session catalog. If you want to plan your week, the session catalog and tracks on the Summit site are the best place to start. The invitation is a reminder to plan your attendance early, since the sessions and tracks shape how much value you get from the event. 📚 Docs: https://www.databricks.com/dataaisummit 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #DataAISummit #DataAI2026 #Conference #NextGenLakehouse #Lakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/dataaisummit More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Databricks Recruiting: Open Roles and How to Apply

How does recruiting at Databricks work? This video covers the hiring process and how candidates can find and apply for open roles. For current openings and the application process, use the official careers page, since roles change often. Check the careers page for the roles and locations that match your background before you apply. Job postings and team pages change frequently, so use the careers site as the source of truth before you contact a recruiter. Check the job description for the team and location, then tailor your application to the data or AI product that role supports. 📚 Docs: https://www.databricks.com/company/careers 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #Careers #Hiring #DataEngineering #NextGenLakehouse #DataAI

— Pinned comment —
Official reference for this topic: https://www.databricks.com/company/careers More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Boosting Software Engineer Performance at Databricks

How do software engineers grow at Databricks? This video discusses performance and development for software engineers working on the platform. It is about working culture rather than a product feature. The official careers page describes open roles and the company's teams. The discussion is about working culture, so pair it with the careers page if you want to see the teams and roles. Engineers who want to grow usually benefit from clear feedback and a visible path, so ask for both when you discuss your development plan. 📚 Docs: https://www.databricks.com/company/careers 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #SoftwareEngineering #Careers #EngineeringCulture #NextGenLakehouse #DataAI

— Pinned comment —
Official reference for this topic: https://www.databricks.com/company/careers More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Data+AI World Tour Paris: Event Recap

What was the Data+AI World Tour in Paris? This event video recaps the Paris stop of the Databricks world tour, with sessions and conversations with the local data and AI community. Use the official Summit page for the schedule of current events and the product announcements. Use the event recap to see what the community covered in Paris, and check the official site for upcoming events. Local events like the Paris world tour give French data engineers a chance to meet the Databricks team and see demos close to home. 📚 Docs: https://www.databricks.com/dataaisummit 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #WorldTour #Paris #DataAI #Community #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/dataaisummit More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Patrick Wendell (Databricks Co-founder) at NextGenLakehouse

A short clip featuring Databricks co-founder Patrick Wendell at NextGenLakehouse. Explore the channel playlists for technical Databricks walkthroughs. 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #PatrickWendell #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/dataaisummit More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Data and AI Platform Benefits for the Gaming Industry

What does the Databricks Data and AI Platform offer gaming companies? This video explains the benefits for the gaming industry, such as unifying player and telemetry data, running analytics at scale and applying AI to engagement and monetisation. The lakehouse product page summarises the platform's architecture and the governance that applies to gaming datasets. Map these benefits to your own player, telemetry and monetisation data to decide where the platform would help first. Studios that run live services often need near-real-time analytics, so check how the platform handles streaming and historical data together. 📚 Docs: https://www.databricks.com/product/databricks-lakehouse 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #Gaming #DataIntelligence #Lakehouse #GamingAnalytics #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/product/databricks-lakehouse More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Carly Taylor on Why Data Matters for the Gaming Industry

Why does data matter so much in gaming? Carly Taylor shares her view on how game studios use data to understand players, improve live operations and make product decisions. The discussion frames the Databricks platform as the foundation for those workloads. The product page explains the lakehouse capabilities behind the examples. Her perspective is useful when you present a data business case to game studio leadership. Gaming teams often use the same data to plan live events and to measure long-term retention, so the platform benefits compound over time. 📚 Docs: https://www.databricks.com/product/databricks-lakehouse 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #Gaming #GameData #DataIntelligence #Lakehouse #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/product/databricks-lakehouse More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

DataCamp at Data+AI Summit: Partner Session Recap

What did DataCamp present at Data+AI Summit? This partner video recaps DataCamp's session at the conference, covering how the learning platform approaches data and AI skills. For Databricks' own training, see the Databricks Academy catalog. Partner sessions show how the wider ecosystem learns and works with Databricks, so they complement the official training. Vendor sessions at Summit often show practical integrations, so note the ones that connect to tools your team already uses. Compare the partner's material with the Databricks Academy courses to choose the learning path that fits your skills. 📚 Docs: https://www.databricks.com/dataaisummit 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #DataCamp #DataAISummit #DataSkills #Partner #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/dataaisummit More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Behzad Nikzad at Data+AI Summit 2025: Interview

Who is Behzad Nikzad and what does he discuss? This interview from Data+AI Summit 2025 features Behzad Nikzad talking about his work with data and AI. Check the video for the specific topics covered, and use the Summit page for the sessions from the same event. Use the interview as background, then find the matching Summit session for the technical depth. Interviews from the conference are most useful when you note one idea to test in your own project after you watch them. 📚 Docs: https://www.databricks.com/dataaisummit 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #DataAISummit2025 #Interview #DataAI #Community #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/dataaisummit More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Derar Alhussein at Data+AI Summit 2025: Interview

What does Derar Alhussein talk about at Data+AI Summit 2025? This interview covers his perspective on data and AI work shared at the conference. The clip is a personal interview, so the Summit page is the place to find the related sessions and announcements. Interviews like this one give a personal view of the conference themes, which complements the official session recordings. Pick one point from the conversation that applies to your team, and check the related session notes or docs before you try it. 📚 Docs: https://www.databricks.com/dataaisummit 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #DataAISummit2025 #Interview #DataAI #Community #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/dataaisummit More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Kyjah Keys at Data+AI Summit 2025: Interview

What does Kyjah Keys discuss at Data+AI Summit 2025? This interview records a conversation at the conference about data and AI. Check the video for the topics covered, and use the Summit page to find the sessions from the same event. Pair the conversation with the Summit session catalog to find the technical talks behind the themes discussed. Conference interviews reveal how practitioners frame problems, so note the terms they use if you want to explain your own project more clearly. 📚 Docs: https://www.databricks.com/dataaisummit 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #DataAISummit2025 #Interview #DataAI #Community #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/dataaisummit More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Andrew Sitz at Data+AI Summit 2025: Interview

What does Andrew Sitz talk about at Data+AI Summit 2025? This interview captures his views on data and AI from the conference floor. The clip is a conversation rather than a product tutorial, so the Summit page lists the technical sessions from the same event. Use the interview to see how practitioners think about data and AI, then check the official sessions for implementation detail. Personal stories from the conference are a quick way to learn what peers struggle with, which can guide your next proof of concept. 📚 Docs: https://www.databricks.com/dataaisummit 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #DataAISummit2025 #Interview #DataAI #Community #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/dataaisummit More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Databricks Community: How to Get Involved

How do you get involved in the Databricks Community? This video explains the community spaces where users ask questions, share solutions and meet other practitioners. Getting started with the platform in the official documentation is a good first step before you join discussions. Ask a question in the community, and share a solution once you have one, since that is how the community grows. The community is a good place to learn from others' mistakes, so search existing threads before you post a new question. 📚 Docs: https://docs.databricks.com/aws/en/getting-started/ 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #DatabricksCommunity #DataEngineering #Learning #Community #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://docs.databricks.com/aws/en/getting-started/ More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Aarni Sillanpaa at Data+AI Summit 2025: Interview

What does Aarni Sillanpaa discuss at Data+AI Summit 2025? This interview captures a conversation from the conference about data and AI work. The video is a personal discussion, so the Summit page is the best source for the sessions and announcements from the same event. The personal view in this interview complements the technical sessions from the same conference. Use the interview to build a list of questions for the sessions you plan to attend, so the conference time is well spent. 📚 Docs: https://www.databricks.com/dataaisummit 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #DataAISummit2025 #Interview #DataAI #Community #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/dataaisummit More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Julia Forde at Data+AI Summit 2025: Interview

What does Julia Forde talk about at Data+AI Summit 2025? This interview records her views on data and AI from the conference. It is a conversation rather than a product walkthrough, so the Summit page lists the related technical sessions. Use the interview as context for the conference themes and then look up the technical sessions on the Summit site. Conversations like this one often surface the practical concerns of business and technical leaders, which helps when you plan a rollout. 📚 Docs: https://www.databricks.com/dataaisummit 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #DataAISummit2025 #Interview #DataAI #Community #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/dataaisummit More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Josue Bogran at Data+AI Summit 2025: Interview

What does Josue Bogran discuss at Data+AI Summit 2025? This interview covers his perspective on data and AI shared at the conference. The clip is a personal conversation, so use the Summit page to find the sessions from the same event. The interview is a good companion to the technical sessions, which cover the implementation details. Interviews from the event work well as short reference points when you explain a data or AI idea to a non-technical audience. 📚 Docs: https://www.databricks.com/dataaisummit 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #DataAISummit2025 #Interview #DataAI #Community #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/dataaisummit More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Data+AI Summit 2025 Highlights from the Conference

What happened at Data+AI Summit 2025? This video gives a general look at the conference, its themes and the atmosphere on site. For the product announcements and session list, use the official Summit page. Use the Summit site to look up the individual sessions and recordings that go deeper on each theme. Conference overviews are a good starting point, and each theme links to deeper sessions you can watch after the event. Take notes on the two or three themes that match your roadmap, then read the linked session pages before your next planning cycle. 📚 Docs: https://www.databricks.com/dataaisummit 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #DataAISummit2025 #Conference #DataAI #Lakehouse #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/dataaisummit More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Data+AI Summit 2025 Expo Floor Tour

What is on the expo floor at Data+AI Summit 2025? This video tours the expo hall, with partner booths and product demos from across the data and AI ecosystem. Use the Summit page to find the partners and the sessions linked to them. Walk the expo with the Summit agenda to find the booths and demos tied to the sessions you want to attend. Expo visits are useful for comparing partner tools side by side, so note the integrations that matter for your stack before you talk to vendors. 📚 Docs: https://www.databricks.com/dataaisummit 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #DataAISummit2025 #Expo #DataAI #Partners #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/dataaisummit More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Microsoft and Databricks CEOs at Data+AI Summit

What did the Microsoft and Databricks CEOs discuss at Data+AI Summit? This clip shows the two leaders on stage at the conference. For what Databricks offers on Azure, check the Azure Databricks documentation and the Summit page for the announcements from the event. The clip shows the partnership on stage, so check the Azure Databricks docs for the platform details. Partnership announcements are useful when you plan cloud decisions, so check which regions and services are available before you commit. 📚 Docs: https://www.databricks.com/dataaisummit 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #Microsoft #Azure #DataAISummit #Partnership #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/dataaisummit More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Data+AI Summit Day 2 Keynote Highlights

What was in the day-two keynote at Data+AI Summit? This video covers the second-day keynote, where Databricks shares product direction and customer stories. The Summit page links to the session recordings and the announcements from the keynote. Watch the keynote chapters to find the product announcements, then open the linked docs for the features you plan to try. Keynote announcements often precede docs and previews, so bookmark the product pages you care about and check them after the event. 📚 Docs: https://www.databricks.com/dataaisummit 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #DataAISummit #Keynote #DataAI #Announcements #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/dataaisummit More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Data+AI Summit Pre-Keynote: What to Expect

What does the Data+AI Summit pre-keynote cover? This video previews the keynote and the themes the conference will focus on. Check the Summit page for the agenda and the sessions once they are published. Use the preview to plan which keynote and sessions you want to follow when the agenda is published. Pre-keynote content helps you plan the first day, so decide which product tracks you want to follow before the agenda fills up. Save the agenda link now so you can compare the keynote announcements with the sessions you have marked. 📚 Docs: https://www.databricks.com/dataaisummit 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #DataAISummit #Keynote #DataAI #Conference #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/dataaisummit More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Data+AI Summit After Party: Event Recap

What was the Data+AI after party like? This event video shows the evening social during Data+AI Summit. It is community content rather than a technical walkthrough. The Summit page lists the sessions and announcements from the conference. The evening is social, so it is best used as networking context rather than as a source of product facts. Events like this one are where many practitioners reconnect after the sessions, so think about who you want to follow up with afterwards. 📚 Docs: https://www.databricks.com/dataaisummit 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #DataAISummit #Community #Networking #NextGenLakehouse #DataAI

— Pinned comment —
Official reference for this topic: https://www.databricks.com/dataaisummit More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Data+AI Summit 2025: Session Highlights

What were the highlights of Data+AI Summit 2025? This video gives a general overview of the conference and its sessions. The Summit page has the official agenda and recordings for the technical content. Use the Summit page to find the recordings for the sessions that match your data or AI roadmap. A general conference overview helps you plan the agenda, and the sessions linked on the Summit page go deeper into each topic. Note the sessions that match your roadmap, then open the linked product docs to check the features before you plan a pilot. 📚 Docs: https://www.databricks.com/dataaisummit 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #DataAISummit2025 #Conference #DataAI #Lakehouse #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/dataaisummit More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Data+AI Summit Day 2: Kicking Off the Second Day

What happened on day two of Data+AI Summit? This video opens the second day of the conference with the schedule and first sessions. Use the Summit page to find the day-two agenda and the recordings. Set your schedule from the Summit agenda so you do not miss the sessions most relevant to your work. Day-two content often builds on the keynote, so review the first day's announcements before you choose the sessions for the afternoon. Review the day-one recap before you choose the afternoon sessions, since the second day often builds on the first. 📚 Docs: https://www.databricks.com/dataaisummit 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #DataAISummit #Conference #DataAI #Keynote #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/dataaisummit More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Databricks Co-founder on Stage at DevConnect

What did the Databricks co-founder say at DevConnect? This clip shows a co-founder on stage at the DevConnect developer meetup, speaking to the community of engineers who build on Databricks. Use the official documentation for the technical details of the features discussed. Developer talks like this one are a good way to hear what the platform team is building next. Co-founder talks often explain why features were built, which gives useful context before you read the product documentation. 📚 Docs: https://www.databricks.com/dataaisummit 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #DevConnect #Cofounder #DeveloperCommunity #DataAISummit #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/dataaisummit More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Matei Zaharia at DevConnect: Developer Community Talk

What did Matei Zaharia talk about at DevConnect? This video features the Databricks co-founder at the DevConnect developer meetup, talking with the developer community about the platform and its open-source roots. The official documentation covers the features discussed in the talk. Check the open-source project pages for the code behind the features discussed in the talk. Talks about open-source roots help explain why the platform exposes open formats and APIs, which affects how you plan integrations. If you want to follow the open-source side of the platform, check the project repositories linked from the Databricks blog. 📚 Docs: https://www.databricks.com/dataaisummit 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #MateiZaharia #DevConnect #ApacheSpark #DeveloperCommunity #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/dataaisummit More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Data and AI Summit 2025: Event Recap

What happened at Data and AI Summit 2025? This recap video covers the conference from the NextGenLakehouse perspective, including sessions, the expo and community moments. The official Summit page lists the session catalog and announcements. Use the recap to see the highlights, then pick the sessions to watch in full on the Summit site. Recaps are a quick way to catch up on the conference, and the full session list on the Summit site helps you go deeper on the topics you missed. 📚 Docs: https://www.databricks.com/dataaisummit 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #DataAISummit2025 #Conference #DataAI #Lakehouse #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/dataaisummit More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Deploy a Databricks Workspace in Under Two Minutes: Setup Steps

How do you deploy a Databricks workspace quickly? This video shows creating a workspace with Unity Catalog enabled. The timing shown depends on your cloud account and permissions, so the workspace creation documentation is the right reference for prerequisites and each step. Set up Unity Catalog metastore access before you start building. Confirm your cloud account, networking and admin permissions first, since those decide how quickly the workspace is ready. Once the workspace exists, set up Unity Catalog and a first catalog so your team can start building right away. 📚 Docs: https://docs.databricks.com/aws/en/admin/workspace/create-workspace 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #UnityCatalog #Workspace #CloudData #Admin #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://docs.databricks.com/aws/en/admin/workspace/create-workspace More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

How Product Managers Prioritize Features at Databricks

How do you prioritize features as a product manager? This video shares how PMs at Databricks weigh customer input, strategy and engineering cost when choosing what to build. It is about product practice rather than a specific feature, so the Summit and careers pages are the official entry points. Use the discussion as a framework for weighing customer impact against effort when you set your own product priorities. Product prioritisation works best when you write down the assumption behind each feature, so you can check it after launch. 📚 Docs: https://www.databricks.com/company/careers 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #ProductManagement #ProductStrategy #Careers #TechCareers #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/company/careers More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Don't Be a Product Manager? Career Advice from a Databricks PM

Should you become a product manager? This video shares a candid view on the product management role, including the trade-offs and who it suits. It is career advice rather than product content. The careers page lists current openings and the teams hiring. Talk to current product managers and weigh the trade-offs before you decide whether the path suits you. Product roles differ across companies, so compare the day-to-day work with your strengths before you choose a path. Compare this advice with a few current PM job descriptions to see how the role is described across teams. 📚 Docs: https://www.databricks.com/company/careers 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #ProductManagement #CareerAdvice #Careers #TechCareers #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/company/careers More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Why Product Management Is So Difficult

Why is product management so difficult? This video discusses the trade-offs, ambiguity and stakeholder work that make product management hard. It is an opinion piece on the role rather than product documentation. The careers page covers the teams and open positions. Use the discussion to set expectations if you are moving into product management, especially around stakeholder work. Product work often involves saying no, so think about how you would handle competing priorities from sales, engineering and customers. 📚 Docs: https://www.databricks.com/company/careers 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #ProductManagement #ProductLeadership #Careers #TechCareers #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/company/careers More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Why Databricks Needs Product Managers

Why does Databricks need product managers? This video explains the role of product managers in a fast-moving data and AI platform, where they translate customer problems into product direction. The careers page lists current PM openings. Read the open PM roles on the careers page to see which product areas Databricks is investing in. Product managers connect engineering, customers and go-to-market teams, so the role rewards people who can explain trade-offs clearly. Compare the PM roles you find with the skills in this video to see where your experience fits. 📚 Docs: https://www.databricks.com/company/careers 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #ProductManagement #Careers #DataAI #TechCareers #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/company/careers More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Why Product Management Is All About Customer Exchanges

Why is product management about customer exchanges? This video argues that good product decisions come from regular conversations with customers and users. It is a view on the PM role rather than a product feature. The careers page covers the teams hiring PMs. Build a habit of regular customer conversations, and write down what you learn so it feeds into product decisions. Customer exchanges work best when you ask about the problem before you discuss the solution, and then check what you heard. 📚 Docs: https://www.databricks.com/company/careers 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #ProductManagement #CustomerFeedback #Careers #TechCareers #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/company/careers More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

What Is the Role of a Product Manager at Databricks?

What does a product manager do at Databricks? This video describes the PM role, from working with customers and engineering to shaping the roadmap for data and AI products. The careers page lists current PM roles and the teams that hire them. Read the PM job descriptions on the careers page to see the skills the role expects. Product managers in data platforms often work across governance, pricing and developer experience, so the role covers more than features. 📚 Docs: https://www.databricks.com/company/careers 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #ProductManagement #Careers #DataAI #TechCareers #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/company/careers More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Advantages of a PhD for a Data and AI Career at Databricks

Is a PhD an advantage at Databricks? This video discusses the value a PhD brings to data and AI work, and where practical experience matters more. It is career advice rather than product content. The careers page lists current openings and the research and engineering teams. Focus on the skills that matter in the role, such as applied experience and clear communication, whatever your academic background. Doctoral training can help with research-heavy roles, while many product and engineering roles value shipped work and clear communication. 📚 Docs: https://www.databricks.com/company/careers 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #PhD #Careers #DataScience #AIResearch #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/company/careers More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

What Makes a Leader? Arsalan Tavakoli on Leadership

What makes a leader according to Arsalan Tavakoli? In this podcast clip, Arsalan Tavakoli, a Databricks leader, describes the qualities he looks for in leaders and how they build teams. It is leadership content rather than a product walkthrough. The careers page covers the roles hiring at Databricks. Apply the leadership ideas in your own team by setting clear expectations and giving regular feedback. Leadership ideas are easier to apply when you pick one habit, such as weekly one-to-ones, and review its effect after a month. 📚 Docs: https://www.databricks.com/company/careers 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #Leadership #Podcast #Careers #TechLeadership #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/company/careers More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

The Idea Behind Open Source at Databricks

What is the idea behind open source at Databricks? This video explains why Databricks releases open-source projects, using examples such as Apache Spark and Unity Catalog. The blog post on open-sourcing Unity Catalog describes the reasoning and what the project includes. Look at the open-source projects on the Databricks GitHub and blogs to see how the community shapes the platform. Open-source contributions shape which features are built, so reading issues and design notes can help you anticipate the next release. 📚 Docs: https://www.databricks.com/blog/open-sourcing-unity-catalog 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #OpenSource #ApacheSpark #UnityCatalog #Lakehouse #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/blog/open-sourcing-unity-catalog More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

I Work at Databricks, of Course: Life on the Team

What is it like to work at Databricks? This short video is a light take on life on the team at Databricks. It is not product content. The careers page lists open roles and the teams hiring. Look at the careers page if this sparks your interest in joining the team. Team stories like this one show the day-to-day culture, which can help you decide whether a role at Databricks fits your goals. Use the careers page to compare the roles and locations before you decide what to explore next. 📚 Docs: https://www.databricks.com/company/careers 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #Careers #WorkingAtDatabricks #TechCareers #DataAI #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/company/careers More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Tom Evans from ASOS: Customer Story on Databricks

How does ASOS use Databricks? In this customer interview, Tom Evans from ASOS shares how the retailer uses the platform for data and analytics. Customer stories show the workloads in context, and the Summit page links to the related sessions from the event. Use the customer story as a reference when you present a similar retail or e-commerce use case internally. Retail teams often start with forecasting or customer analytics, so the story can help you prioritise a first use case for your own business. 📚 Docs: https://www.databricks.com/dataaisummit 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #ASOS #CustomerStory #Retail #DataAI #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/dataaisummit More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

NextGenLakehouse Newsletter: Celebrating Team Growth

What is NextGenLakehouse celebrating? This video is an update on the NextGenLakehouse newsletter and the growing team behind the channel. It is channel news rather than product content. Subscribe to the newsletter for the Databricks coverage and the new videos as they are published. Subscribe to the newsletter to get the next Databricks updates in your inbox. Community growth is part of how the channel keeps delivering practical Databricks content, so thanks to viewers for the support. Newsletter subscribers get new videos and Databricks updates first, so sign up through the channel links. 📚 Docs: https://www.databricks.com/dataaisummit 🔗 More: https://www.nextgenlakehouse.com/content #NextGenLakehouse #Newsletter #Databricks #DataCommunity #Lakehouse #DataAI

— Pinned comment —
Official reference for this topic: https://www.databricks.com/dataaisummit More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

David Sykes from Octopus Energy: Customer Interview

How does Octopus Energy use data? In this customer interview, David Sykes shares how the energy company works with its data and platform. Customer stories show real workloads, and the Summit page links to the related sessions. Use the customer story to compare how an energy company approaches data on the platform. Energy and utility data often combines meter readings, pricing and customer records, so the customer story is a useful reference for similar workloads. Read the customer story alongside the Databricks docs for the features the team relies on most. 📚 Docs: https://www.databricks.com/dataaisummit 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #OctopusEnergy #CustomerStory #Energy #DataAI #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/dataaisummit More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Subscribe to NextGenLakehouse on YouTube

Should you subscribe to NextGenLakehouse? This short video asks viewers to subscribe for weekly Databricks content, covering product features, events and interviews. Subscribe to get new videos on Lakeflow, Unity Catalog, Genie and Lakebase as they are published. The Databricks getting started documentation is a good place to begin. Turn on notifications so you do not miss the next episode. New videos cover Lakeflow, Unity Catalog, Genie and Lakebase, so subscribing helps you keep up with the platform as it changes. 📚 Docs: https://docs.databricks.com/aws/en/getting-started/ 🔗 More: https://www.nextgenlakehouse.com/content #NextGenLakehouse #Databricks #YouTube #DataEngineering #Lakehouse #AI

— Pinned comment —
Official reference for this topic: https://docs.databricks.com/aws/en/getting-started/ More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Databricks vs the Market: How the Lakehouse Platform Compares

How does Databricks compare to the rest of the market? This short video compares the Databricks lakehouse with other data platforms and tools. Comparisons depend on workload, cloud and team skills, so verify claims against your own requirements. The product page describes the lakehouse architecture and its core components. Use the comparison as a starting point and test the platforms against your own workload, cost and team skills. When you present a comparison to stakeholders, show the workload, the cost model and the governance needs, not only feature checklists. 📚 Docs: https://www.databricks.com/product/databricks-lakehouse 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #Lakehouse #DataPlatform #Comparison #DataEngineering #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://www.databricks.com/product/databricks-lakehouse More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content

Why AtScale? Semantic Layers Compared with Databricks Metric Views

Why choose AtScale, and how does it compare with Databricks? This video looks at the semantic layer vendor AtScale and the choice between an external semantic layer and a native one. Databricks metric views define measures and dimensions in Unity Catalog, and the tutorial below shows how to build one. Decide where your metric definitions should live before you choose a semantic layer, since that choice affects every BI tool. Semantic layers affect every BI tool and AI agent that asks for a metric, so the choice has a long-term impact on governance and consistency. 📚 Docs: https://docs.databricks.com/aws/en/uc-semantics/metric-views/tpch-example 🔗 More: https://www.nextgenlakehouse.com/content #Databricks #AtScale #SemanticLayer #MetricViews #BI #NextGenLakehouse

— Pinned comment —
Official reference for this topic: https://docs.databricks.com/aws/en/uc-semantics/metric-views/tpch-example More NextGenLakehouse videos on this topic: https://www.nextgenlakehouse.com/content