Machine Learning & MLflow

Yes. AI Runtime is Databricks' fully managed serverless GPU compute for deep learning training and fine-tuning. The docs list on-demand A10s and H100s, and you can launch workloads from the CLI, notebooks or Jobs without provisioning GPU infrastructure. Ray Core, Ray Data, Ray Train and Ray Tune can run on AI Runtime.

MLflow Tracing records the inputs, outputs, latency, token usage, and cost of every intermediate step in an agent or LLM request. It can store traces as Unity Catalog Delta tables with schema- and table-level access control, and you can query them directly with SQL.

Yes. MLflow Tracing can store traces as Unity Catalog Delta tables with schema- and table-level access control, and the traces support direct SQL access. Each trace records the inputs, outputs, latency, token usage, and cost of every intermediate step in an agent or LLM request. Teams use these traces to debug agent behavior and track quality and cost over time, with governance inherited from Unity Catalog.

Databricks Model Serving hosts custom Python models registered in Unity Catalog or the model registry, along with Databricks-hosted foundation models. Base models such as Meta-Llama-3.3-70B-Instruct and GTE-Large are available on pay-per-token pricing, and fine-tuned variants can use provisioned throughput. You can also govern external models hosted outside Databricks, such as GPT-4 from OpenAI, from a central endpoint. Endpoints scale up or down automatically as demand changes.

Model Serving deploys models behind serverless, autoscaling REST endpoints for real-time inference, so you do not manage servers. Endpoints serve custom MLflow-format Python models, including scikit-learn, XGBoost, and PyTorch models, from Unity Catalog or the workspace model registry. Endpoints add and remove capacity as demand changes, which helps keep latency low.

Yes. Model Serving hosts custom MLflow models, such as scikit-learn, XGBoost, PyTorch, and Hugging Face transformers models, on autoscaling serverless endpoints. Models can come from Unity Catalog or the workspace model registry. For fine-tuned foundation models, provisioned throughput supports fine-tuned variants of base models.

Data scientists use notebooks in Python, R, and SQL, track experiments with managed MLflow, and train models on AI Runtime serverless GPU compute. MLflow logs parameters, metrics, artifacts, and code versions for each run, and the MLflow Model Registry in Unity Catalog stores governed model versions. Models can then be deployed to Model Serving endpoints.

Managed MLflow tracks parameters, metrics, artifacts, and code versions for each experiment run, so results are reproducible. You can log and manage these with the open-source MLflow APIs and view them in the managed Databricks service. Models can be registered in the Unity Catalog model registry, which gives governed versions you can share across workspaces.

MLflow manages models from experiment tracking through the Unity Catalog model registry and into deployment on Model Serving. Models registered in Unity Catalog or the workspace registry can be deployed to serverless, autoscaling REST endpoints, and Unity Catalog controls who can call them. MLflow Tracing and LLM-judge evaluation help you assess GenAI models and agents before you promote them.

AI Runtime provides serverless GPU compute with on-demand A10 and H100 GPUs for training, fine-tuning, and deep learning workloads. It includes pre-built environments, dataloaders, MLflow experiment tracking, and job scheduling for GPU workloads. You can use it through the Databricks CLI, notebooks, jobs, and Ray.

AutoML orchestrates distributed model training and hyperparameter tuning across multiple algorithms, including scikit-learn, XGBoost, and LightGBM models. You can also track your own tuning trials with MLflow experiment tracking to compare runs and pick the best model. Databricks handles the distributed compute for these runs.

Yes. MLflow is the open-source AI engineering platform for agents, LLMs, and ML models, and Databricks offers a managed MLflow service that adds enterprise security, disaster recovery, and scalability. You can track parameters, metrics, artifacts, and code versions with the same APIs. Models registered in the Unity Catalog model registry can be shared across workspaces.

MLflow is the open-source AI engineering platform created by Databricks, used for experiment tracking, model management tracing. Databricks offers it as a managed service that adds enterprise security, disaster recovery, and scalability. MLflow Tracing records each step of an agent or LLM request, LLM-as-a-judge evaluation adds quality feedback, and models in the Unity Catalog registry are governed and can be deployed to Model Serving.

Databricks Feature Store is a Unity Catalog-governed registry where teams define features once and reuse them for training and inference. Feature Views are the recommended way to define features for new use cases, with Databricks managing the pipelines, while feature tables are Delta tables with a primary key that you populate. Models record lineage to the features they use, and Model Serving looks up current feature values from an online store, powered by Lakebase, at inference time.

Synced tables copy Unity Catalog Delta tables, Iceberg tables, and views into a read-only Lakebase Postgres table so applications can query them with low latency. They support snapshot, triggered, and continuous sync modes, and synced tables are generally available. Feature Store online stores are also powered by Lakebase.

Databricks Runtime for Machine Learning is a prebuilt compute image with common ML and deep learning infrastructure, so data scientists can skip most environment setup. It supports GPU instance types, and each release refreshes its libraries under the runtime maintenance policy; the release notes list the full set of preinstalled libraries. For fine-tuning and custom deep learning at scale, Databricks recommends AI Runtime, its serverless GPU offering.

Feature Views in Databricks Feature Store define features declaratively and compute them, including time-windowed aggregations and streaming features. Features can be published to the Online Feature Store, which is powered by Lakebase.

Point-in-time correctness creates a training dataset that reflects feature values as of when each label was recorded, which keeps future values out of training. To enable it for time-series feature tables, declare timeseries_columns (Unity Catalog) or timestamp_keys. The Feature Store docs also say on-demand computation helps prevent training and serving skew.

The Online Feature Store is powered by Databricks Lakebase. publish_table syncs an offline feature table in TRIGGERED mode (the default, updated through the API or on a schedule), CONTINUOUS mode (a streaming pipeline), or SNAPSHOT mode (a one-time copy). The docs describe low-latency access at high scale but give no latency figures.

OpenSharing lets a provider share a model together with all of its dependencies, such as feature tables and Unity Catalog functions, in one share that the recipient mounts as a read-only catalog. Real-time serving still needs an online store and a Model Serving endpoint in each region where the model is used.

MLflow 3 Tracing records each step of a request, including LLM and tool calls, with inputs, outputs and metadata. Scorers, which are LLM judges and code-based checks, attach quality assessments to traces and can be run with mlflow.genai.evaluate. Deployment jobs manage the model lifecycle through evaluation, approval and deployment, with governance through Unity Catalog.

Deployment jobs manage the model lifecycle, including evaluation, approval and deployment, with governance through Unity Catalog and an activity log. The Model Registry integrated with Unity Catalog centralizes models and artifacts, supports access across workspaces, and tracks lineage.

Start custom models on CPU and move to GPU only when measured latency requires it. Size provisioned concurrency as target QPS multiplied by average latency in seconds, so 100 QPS at 200 ms needs 20 concurrency. Load test to measure latency and concurrency. Model Serving can support over 25K queries per second with overhead latency of less than 50 ms.

Lakebase backs the Online Feature Store. publish_table syncs an offline Delta feature table in TRIGGERED, CONTINUOUS or SNAPSHOT mode. Feature Views can compute time-windowed aggregations declaratively for serving.

Databricks AI Runtime is serverless GPU compute for training, fine-tuning and batch inference on custom deep learning models, with on-demand A10 and H100 GPUs.

Send a POST request to https://<databricks-instance>/serving-endpoints/<endpoint-name>/invocations with a JSON body in dataframe_split format, which Databricks recommends, or in dataframe_records format. Authenticate with a Databricks API token, and use machine-to-machine OAuth tokens for production workloads. Using plain HTTP requests means any application or workflow tool can call the endpoint without a platform-specific client library.

Databricks runs PySpark, MLflow, and Feature Store on one platform, with features stored in Unity Catalog and served in production from the same governance layer. Model Serving deploys custom models and LLMs as REST endpoints that scale automatically, and AI Runtime provides serverless GPU compute for deep learning. The Databricks documentation does not describe Snowpark, so check Snowflake's documentation for its behavior.

Genie lets business users ask data questions in natural language, grounded in Unity Catalog data. Data teams configure Genie Agents with trusted data, metrics, business rules, sample queries, and verified answers. The Genie documentation does not compare Genie with Snowflake Cortex.

Databricks Model Serving can host multiple served models on one endpoint with a traffic split, but all models on that endpoint must be the same type (custom models, provisioned throughput Foundation Model APIs, or external models); types cannot be mixed. For governance across providers, AI Gateway covers Foundation Model APIs, including GPT, Claude, and Gemini, and external providers such as OpenAI and Anthropic, with rate limits, budgets, usage tracking through system tables, guardrails, and inference-table audit logs.

AI Runtime provides serverless GPU compute for deep learning training and inference, so teams can run GPU workloads without provisioning GPU clusters themselves. Model Serving also offers GPU support for deployment. Training runs are tracked with MLflow, and the data and models they use are governed by Unity Catalog.

Genie Agents are configured by data teams with trusted data, metrics, business rules, sample queries, and verified answers, so Genie One responses reflect the organization's own definitions. Business users get answers governed by the same Unity Catalog permissions as the underlying tables. The Genie documentation does not compare Genie with Microsoft Fabric.

The AI Gateway, which Databricks describes as its governance solution for enterprise AI, gives teams one place to set policy and track spend for model access. It provides consumption limits, budgets with per-user thresholds, usage tracking by service, model, principal, and tag, guardrails, and inference-table audit logs. It covers Foundation Model APIs (GPT, Claude, and Gemini) and external providers such as OpenAI and Anthropic.

Databricks AI Search (formerly Vector Search) keeps Delta Sync indexes updated automatically as the source Delta table changes, supports hybrid keyword-similarity search that merges vector and BM25 results, and is governed by Unity Catalog. The documentation does not compare it with Weaviate or Pinecone.

AI Runtime supports distributed training with Ray. Ray Core, Ray Data, Ray Train and Ray Tune can run on AI Runtime, and multi-node GPU jobs can be submitted from the CLI with YAML job configs.

The MLflow 3 Prompt Registry, which is in Beta, stores immutable prompt versions with auto-incrementing numbers, and each version can carry a commit message. Aliases are mutable pointers to versions, such as production or staging, set with set_prompt_alias, so you can run gradual rollouts and A/B tests by pointing an alias at a different version.

ai_query calls a Model Serving endpoint from SQL, including custom endpoints for traditional ML models, for real-time or batch inference. For custom endpoints the request can be a single column or a STRUCT, and returnType is inferred from the model schema on Runtime 15.2 or above. The definer must have CAN QUERY permission on the endpoint.

MLflow 3 tracks models and agents with metrics, parameters and traces. The Model Registry integrated with Unity Catalog centralizes models and artifacts and supports access across workspaces. Deployment jobs manage the evaluation, approval and deployment steps of the model lifecycle.

Package a custom model as an MLflow pyfunc, register it in Unity Catalog, and deploy it to a Model Serving endpoint. Provisioned throughput is for workloads that need performance guarantees. Clients can query the endpoint directly or call it through ai_query in SQL.

Databricks recommends installing ML libraries at the compute level, and for Unity Catalog on Runtime 13.3 LTS or later it recommends using Unity Catalog volumes as the library source with standard access mode. Storing libraries in the DBFS root is deprecated and disabled by default from Runtime 15.1.

Models can be logged with MLflow and registered in the Unity Catalog model registry, which centralizes models and artifacts and supports access across workspaces. Deployment jobs manage evaluation, approval and deployment steps for the model lifecycle.

AI Runtime is Databricks' fully managed serverless GPU compute for training and fine-tuning deep learning models, including multi-node distributed jobs launched from the CLI. The docs list on-demand A10s and H100s, and Ray Train can run on AI Runtime for distributed workloads.

Genie Code is a Databricks agent that helps with data exploration and model building, and the ML overview lists it in the getting-started area. The docs checked do not say that it analyzes AI Runtime training logs, so validate that before relying on it.

AI Runtime runs continuous automated health checks that help detect and isolate failing GPU nodes, and Databricks manages GPU health, whereas classic compute requires users to validate it themselves.

Databricks supports distributed PyTorch training on Spark clusters with TorchDistributor, an open-source module in PySpark, and offers a DeepSpeed distributor for scaling deep learning workloads. The same page covers Ray and Spark ML, and Spark ML on Standard compute lets you train without managing a full cluster. Choose the distributor that matches your framework and cluster layout.

Databricks Feature Store serves features to real-time models through the Online Feature Store, which is powered by Databricks Lakebase. At inference time, the model automatically looks up the latest feature values. For Feature Views, Databricks creates and manages the feature pipelines, while for feature tables you own the pipeline that populates and refreshes the table.

Log custom models trained with scikit-learn, XGBoost, PyTorch, Hugging Face transformers, or other standard ML libraries with MLflow, then register them in Unity Catalog, the recommended registry. Create a Model Serving endpoint to deploy and query the model, and endpoints scale up or down to meet demand changes.

Databricks AutoML runs regression, classification, and forecasting experiments through a low-code UI or the Python API, and orchestrates distributed training and hyperparameter tuning across multiple algorithms. Each trial is tracked in MLflow, and its generated notebook is saved as an MLflow artifact or imported into your workspace. Classification and regression notebooks include SHAP (Shapley value) code that is off by default; set shap_enabled = True and re-run the notebook to enable it.

Databricks serves foundation models including Claude, GPT, Gemini, Gemma, Llama, Qwen, and DeepSeek, with pay-per-token access that does not require provisioning dedicated serving capacity. Provisioned throughput options are also available, with on-demand capacity that has no term commitment or reserved capacity for a fixed term. For batch inference over tables, use ai_query as described in the AI Functions guidance to select supported models and serving options.

Databricks Machine Learning covers data preparation, distributed training, experiment tracking, model governance, and model serving on one platform. Distributed deep learning uses Ray, TorchDistributor, and DeepSpeed, and clusters come pre-configured with scikit-learn, XGBoost, and MLflow. MLflow tracks experiments, Models in Unity Catalog provide centralized governance, Model Serving deploys models as REST endpoints with automatic scaling and GPU support, and the Feature Store manages features in Unity Catalog. AI Runtime provides serverless GPU compute for deep learning training and inference.

Model Serving can serve multiple models on one endpoint and split traffic between them with traffic_config routes that set percentages, such as sending 10% of traffic to one served model and 90% to another. This supports A/B testing, which lets you compare the performance of different models. Shifting traffic is a manual update to the endpoint configuration, so plan your rollback steps in advance.

AI Runtime provides on-demand NVIDIA A10 and H100 GPUs for distributed training, launched from the CLI, notebooks, or Lakeflow Jobs. Ray Train and Ray Tune support distributed training and hyperparameter tuning, and MLflow tracks experiments, metrics, and artifacts for every run. Trained models can be registered in the MLflow Model Registry, which provides a centralized repository for managing model versions.

Lakehouse Monitoring inference profiles track model inputs, predictions, and optional ground-truth labels, comparing model performance and data quality across time windows. Drift metrics measure how the data changes against a baseline table, such as training or validation data, or across successive time windows. Results are written to Delta metric tables with a customizable dashboard that you can use to track model-serving endpoints.

Ingest sensor data continuously with Structured Streaming and Auto Loader from cloud storage into Delta tables, which use checkpoints for fault tolerance. Train failure-prediction models on the historical data with AI Runtime and track them in MLflow, then serve the registered models through Databricks Model Serving.

Unity Catalog records data access and system activity in the audit log system table (system.access.audit), which is in Public Preview. You can query it with SQL to review access, but the documentation does not describe automated detection of unusual access, so any anomaly detection is something you would build on top of that table.

The MLflow Model Registry is a centralized repository and set of APIs for managing the model deployment process, with model versions tracked there. Unity Catalog governs access to registered models and tracks their lineage. MLflow experiment tracking records parameters, metrics, artifacts, and code versions for every run, so each model can be traced to how it was built.

Databricks provides an ML platform for the full lifecycle: Databricks Runtime for ML, MLflow experiment tracking, Feature Store, serverless GPU compute through AI Runtime, and Model Serving endpoints for custom models and LLMs, all governed by Unity Catalog. The cited documentation does not compare Databricks with Snowflake, so evaluate both against your workloads.

Databricks offers Model Serving for custom, Foundation Model and external models, AI Search for vector retrieval, and Unity Gateway for identity-aware access control and audit logs across models, MCP servers and tools. These docs do not compare Databricks with Snowflake, so check Snowflake's documentation for Cortex.

Databricks provides Databricks Runtime for ML, MLflow experiment tracking, AI Runtime serverless GPU compute for deep learning, and Model Serving endpoints for custom models and LLMs, all governed by Unity Catalog. The cited documentation does not cover BigQuery ML, so compare it against Google's documentation before deciding.

Databricks provides Databricks Runtime for ML, MLflow experiment tracking, serverless GPU training through AI Runtime, and Model Serving endpoints for custom models and LLMs, all governed by Unity Catalog. The cited documentation does not compare Databricks with Redshift ML, so check Amazon's documentation for Redshift ML's behavior.

Databricks offers an ML platform with AI Runtime, Feature Store, MLflow, Model Serving and Foundation Model APIs, and Unity Gateway governs access to models, MCP servers and tools. The docs do not compare Databricks with Microsoft Fabric, so check Microsoft's documentation before deciding.

PySpark on Databricks lets you use Python directly on Apache Spark, the engine behind Databricks compute. The same PySpark documentation covers Spark SQL and DataFrames, Structured Streaming, the Pandas API on Spark, and MLlib. This page does not describe Snowpark, so check Snowflake's documentation for its behavior.

Databricks offers an ML platform with AI Runtime, Feature Store, MLflow, Model Serving and Foundation Model APIs, plus AI Search for vector retrieval. The docs do not name or compare other platforms, so compare Snowflake and BigQuery against their own documentation for your use case.

Databricks provides Databricks Runtime for ML, MLflow experiment tracking, serverless GPU training through AI Runtime, and Model Serving endpoints for custom models and LLMs, all governed by Unity Catalog. The cited documentation does not compare Databricks with Cloudera, so check Cloudera's documentation before deciding.

Databricks retired the hosted DBRX endpoints under its published model retirement policy: pay-per-token serving ended April 30, 2025, and provisioned throughput ended December 19, 2025. The retirement table names a recommended replacement, which for pay-per-token DBRX is Meta-Llama-4-Maverick. Migrate hosted workloads to that replacement before the retirement dates.

Databricks retired the hosted DBRX serving endpoints under its model retirement policy: pay-per-token ended April 30, 2025, and provisioned throughput ended December 19, 2025. The policy works in stages, with deprecation first, a warning in the UI, and a retirement date announced at least one month in advance. Once retired, a model is no longer accessible, so migrate to the recommended replacement before the date.

Foundation Model APIs offers models from several providers, including GPT and GPT OSS (OpenAI), Gemini and Gemma (Google), Claude (Anthropic), Llama (Meta), and Qwen, DeepSeek, and GTE embedding models, with pay-per-token and provisioned throughput options. The roster changes and some models are retired on dates listed on the supported-models page, so check it before you build.

Yes. Databricks' model retirement policy has a deprecation stage, during which a warning appears in the UI, followed by retirement. Retirement dates for deprecated models are announced at least one month in advance, and Databricks recommends migrating to a named replacement model before the retirement date. Plan migrations against that window.

Yes. Databricks provides a fully managed, hosted version of MLflow, with a Unity Catalog model registry for centralized governance of models. The managed service covers experiment tracking, evaluation powered by MLflow, and workflows for developing, evaluating, and monitoring agents and LLM applications.

Yes. Unity Catalog governs ML models: Models in Unity Catalog is the recommended way to register, version, tag, and alias models, such as Champion. You control access with Unity Catalog privileges, and access and lineage are governed alongside your other data and AI assets. Models are then deployed to Model Serving endpoints.

Databricks Feature Store is a central registry for the features used in AI and ML models. When features and models are registered in Unity Catalog, you get built-in governance, lineage, point-in-time joins, and cross-workspace feature sharing and discovery. The docs recommend Feature Views for most new projects, and feature tables for workloads that need Delta Sharing or a third-party online store. The example notebooks walk through creating a feature table, training a model, and running batch scoring with automatic feature lookup.

You can serve multiple models from one Model Serving endpoint and set a traffic split between them, which is designed for comparing model performance and running A/B tests. The custom-model example routes 90% of traffic to a current served model and 10% to a challenger, and you can change the split later, for example to 50/50, with the Edit configuration button or a PUT request. Endpoints cannot mix model types, so keep models of the same type on each endpoint.

MLflow on Databricks covers both. It is a fully managed, hosted version of open source MLflow, which Databricks describes as the largest open source AI engineering platform for agents, LLMs, and ML models. For traditional ML it provides experiment tracking, model evaluation, a model registry, and model deployment tools. For agents and LLM applications it adds tracing, evaluation, and prompt management to debug and improve production-quality AI applications.

Register models in Unity Catalog using a three-level name in the form catalog.schema.model, then use aliases to mark deployment status. Aliases are mutable named references to a specific model version, so a Champion alias can move to a new version without changing the workloads that load it. Registered models are a FUNCTION securable, so you grant access with GRANT ON FUNCTION.

After you run an agent with tracing enabled, open the experiment's Traces tab to view its trace. The span tree shows the path from the agent through each LLM call and tool call, so you can see where time and tokens are spent. Traces can be stored as Unity Catalog Delta tables, and you can ask Genie Code questions about trace data in natural language.

MLflow on Databricks is the open source MLflow platform, fully managed and hosted by Databricks and made more robust and scalable for enterprise use. It supports developing, evaluating, and monitoring AI agents, LLM applications, and ML models. Its core capabilities include experiment tracking, a Model Registry integrated with Unity Catalog for centralized governance and lineage, tracing, and agent evaluation.

Workspace experiments are not tied to any notebook, so any notebook can log runs to one by using its experiment ID or name. Notebook experiments are tied to a single notebook and share its name and ID. If a run starts without an active experiment, Databricks automatically creates a notebook experiment for you, so logging works out of the box.

Right-click the folder, select Create > MLflow experiment, then enter a name and an optional artifact location. You can also create one from the Experiments page using the Custom option, and log runs to it with mlflow.set_experiment() using the experiment path.

By default, artifacts are stored in MLflow-managed storage at dbfs:/databricks/mlflow-tracking/<experiment-id>. Databricks recommends setting the artifact location to a Unity Catalog volume instead. S3 is also supported but not recommended, and artifacts stored in S3 do not appear in the MLflow UI.

Databricks autologging works in interactive Python notebooks and supports 13 ML and generative AI frameworks, including scikit-learn, Apache Spark MLlib, TensorFlow, XGBoost, and LangChain. It automatically captures model parameters, metrics, files, and lineage information during training. Each training session is recorded as an MLflow tracking run, so your experiment history builds up without extra code.

On serverless compute, autologging is not enabled automatically; call mlflow.autolog() explicitly in your notebook, and training sessions are then recorded as MLflow tracking runs.

In a notebook, call mlflow.autolog(disable=True). Workspace administrators can turn autologging on or off for all interactive notebook sessions from the Advanced tab of the admin settings page. Cluster-level changes take effect after the clusters are restarted, so plan the restart when you change the setting.

Models are registered in Unity Catalog, which Databricks hosts as a version of the MLflow Model Registry, and each model is addressed by a three-level name in the form catalog.schema.model. You can register a model with register_model(), with log_model() by setting registered_model_name, or from the experiment run page in the UI. Registering in Unity Catalog gives centralized governance, lineage tracking, and access across workspaces.

Yes. In the Unity Catalog training examples, each training run that registers a model creates a new model version under the registered model. You can promote a version by copying it across registered models, for example from a staging model to a production model. Unity Catalog models also support adding descriptions, renaming, and deleting versions.

Aliases are mutable names that point to a specific model version and indicate its deployment status. The Champion alias is commonly used to mark the production version, and you can reassign it to a different version. Batch jobs that load a model by alias (models:/<name>@Champion) pick up the change on their next run.

Key-value tags on registered models and model versions help label and categorize them, for example by task or validation status. Setting and deleting tags requires the APPLY TAG privilege on the model or ownership of it. Granting APPLY TAG lets collaborators label models without taking ownership.

New model versions in Unity Catalog should include a model signature, which documents the expected inputs and outputs of the model. With a signature, the model gets automatic input enforcement at inference time. Versions registered without a signature have no automatic input enforcement and must handle unexpected inputs themselves.

Model Serving deploys custom Python models in MLflow format registered in Unity Catalog or the workspace registry, including scikit-learn, XGBoost, PyTorch, and Hugging Face transformers models, as well as agents. It also serves Databricks-hosted foundation models and externally hosted models through external model endpoints. Each served model is exposed as a REST API that you can integrate into web and client applications.

Call a served model from SQL with ai_query, which supports batch inference in production workflows such as Lakeflow pipelines, Databricks workflows, and Structured Streaming. Real-time inference runs through REST endpoints, and both paths use the same served models.

In the Serving UI, open the kebab menu at the top of the endpoint page and select Delete. Through the REST API, send DELETE /api/2.0/serving-endpoints/{name}, or call delete_endpoint with the MLflow Deployments SDK. Deleting an endpoint disables usage and deletes all data associated with it, so confirm you no longer need it because this cannot be undone.

Endpoint configuration updates can take 10 to 25 minutes, depending on the details of the update. The existing configuration keeps serving until the new one is ready, and a failed update leaves the existing configuration in effect. You can cancel an in-progress update from the Serving UI.

Databricks does not recommend scale to zero for production endpoints, because capacity is not guaranteed when an endpoint is scaled to zero. For production, keep endpoints provisioned so capacity is ready when requests arrive. Scale to zero is controlled by the scale_to_zero_enabled field in the endpoint configuration, and it is a trade-off between idle cost and availability.

When an endpoint scales back up after scaling to zero, requests incur additional latency that Databricks calls a cold start. Only requests that arrive while the endpoint is scaling up see the slower response. Teams that need predictable latency should keep endpoints scaled up so every request gets the warm-endpoint experience.

Custom model endpoints scale out by the number of requests a served model can process at the same time: Small handles 0 to 4 requests, Medium 8 to 16, and Large 16 to 64. Size the endpoint to your expected concurrency, which the docs estimate as roughly QPS multiplied by model run time.

Model Serving accepts up to 16 MB per request, and agent endpoints accept up to 4 MB. Requests or responses over 1 MB are not logged, so keep payloads under that size if you rely on logging. Staying within these limits keeps your endpoint logs complete.

Each request can run for up to 597 seconds of model execution. Design long-running inference to fit inside this window, or use batch inference for very long workloads. Batch inference with ai_query is the natural path for large offline jobs.

With the custom option and route optimization, provisioned concurrency is up to 1024 per model, and it is up to 4096 per workspace. Your Databricks account team can increase the workspace limit if you need more. Set provisioned concurrency to match your expected traffic so serving capacity is ready when requests arrive.

With route optimization, the documented limit is 300,000 queries per second per endpoint, which makes route optimization the path for high-throughput endpoints. Without route optimization, the per-workspace limit is 200 queries per second, which the docs recommend for small development use cases. Enable route optimization at endpoint creation when you need production throughput at scale.

An inference table automatically logs incoming requests and outgoing responses for a model serving endpoint, along with HTTP status codes, model execution times, and the request and response JSON. Rows include fields such as databricks_request_id, status_code, execution_time_ms, request, response, and request_metadata. A batch of inputs is logged as one row. The inference tables page is marked retired, and Databricks recommends AI Gateway-enabled inference tables for new work.

For the legacy inference table feature, records are saved to a Delta table in Unity Catalog with the default name <catalog>.<schema>.<endpoint-name>_payload, and you can add a custom prefix. This page is retired, and AI Gateway tables use a different schema and are recommended for new work.

Inference tables on serving endpoints are a legacy feature: their documentation is retired, and Databricks recommends AI Gateway-enabled inference tables for new work. For the legacy feature, select Enable inference tables and choose a catalog and schema when creating an endpoint, or open Edit configuration on an existing endpoint, enable the option, and update. Unity Catalog must be enabled in the workspace.

For the legacy inference table feature, the endpoint creator and any user who modifies the endpoint need Can Manage on the endpoint, plus USE CATALOG, USE SCHEMA, and CREATE TABLE in the target catalog and schema. Permissions are checked for the user who creates or modifies the endpoint, so set them up before enabling the feature.

For the legacy inference table feature, changing the table's schema or name, deleting it, or losing permissions on its catalog or schema can corrupt it. The endpoint's auto-capture configuration then shows FAILED, and you must create a new endpoint to resume logging.

Migration is one-way: after an endpoint migrates to AI Gateway inference tables, it cannot switch back to the legacy table. Databricks recommends AI Gateway-enabled inference tables, so plan the migration with that in mind. Logging is best effort, and logs are typically available within an hour.

Define multiple served entities on the endpoint, each with a name such as current or challenger, and set a traffic_config that maps those names to traffic percentages. The Databricks example sends 90% of traffic to the current model and 10% to the challenger. This lets you test a new model version while production stays on the current one.

Yes. You can change the split later with a PUT request to the endpoint configuration, or with the Edit configuration button on the Serving tab, for example to move from 90/10 to 50/50. Querying a specific served model directly bypasses the traffic settings. The feature is designed for comparing model performance and running A/B tests.

Route optimization is available for custom model serving endpoints and feature serving endpoints. It changes the network path for inference requests to reduce overhead latency and raise throughput. Endpoints that use Foundation Model APIs or external models are not supported.

Route optimization is set when you create the endpoint, because existing endpoints cannot be updated to become route optimized. Databricks in-house OAuth tokens are the supported authentication for route-optimized endpoints, and personal access tokens are not supported. Route-optimized endpoints must still meet the same requirements as non-route-optimized endpoints.

Route optimization shortens the network path for inference requests, which raises supported queries per second and gives more stable, lower latency with lower overhead latency. It applies to custom model and feature serving endpoints and can only be enabled at endpoint creation.

You can call the REST API at POST /serving-endpoints/{name}/invocations, use the MLflow Deployments SDK predict() function, use the OpenAI client with the endpoint name as the model input, or use the Query endpoint option in the Serving UI with JSON input. The REST API, MLflow Deployments SDK, and OpenAI client need a Databricks API token. Databricks recommends machine-to-machine OAuth tokens for production.

MLflow Tracing records the inputs, outputs, latency, token usage, and cost of every intermediate step in an agent run. Each step, such as an LLM call or a tool call, appears as a span in the trace. Traces help you debug, evaluate, and monitor agents.

Call mlflow.langchain.autolog() before running the agent, and autologging captures a trace with a span for each step of the LangChain run. LangGraph is a separate integration on the automatic tracing page, so enable its integration as well if your agent uses it. Autologging does not cover custom code, which you can instrument with custom spans.

Autologging covers the framework calls it knows about, so add custom spans for your own functions and steps that it does not cover. Custom spans let you instrument your own code alongside the automatically traced framework calls. The MLflow Tracing guide describes manual and custom tracing for this case.

Yes. Model Serving can deploy any Python model or custom code as a production-grade API on CPU or GPU compute. Custom pyfunc models let you deploy arbitrary Python code or ship additional code alongside your model, using code_path for helper files. For pyfunc models, add dependencies explicitly with pip_requirements, conda_env, or extra_pip_requirements.

Databricks documents TorchDistributor for distributed PyTorch training, the DeepSpeed distributor for large models limited by memory, Ray for parallel compute in ML workflows, and Spark ML distributed training through the pyspark.ml.connect module. These options scale training beyond a single machine. The docs advise training on a single machine when possible and moving to distributed training when the model or data no longer fits in memory.

TorchDistributor is an open-source module in PySpark for distributed PyTorch training on Spark clusters. It launches PyTorch training jobs as Spark jobs and uses torch.distributed.run to run training across worker nodes. It is one of the distributed training options included in Databricks Runtime for Machine Learning, and the DeepSpeed distributor is built on top of it.

With Ray 2.3.0 and above, you can create Ray clusters and run Ray applications on Apache Spark clusters with Databricks. Ray clusters are managed in the same execution environment as the running Spark cluster, so there is no separate environment to set up. Ray on Spark runs on dedicated access mode, no-isolation shared access mode, and jobs clusters.

Ray is well suited to dynamic, compute-intensive parallel workloads. On Databricks, the Ray guidance covers distributed deep learning (PyTorch, TensorFlow, Hugging Face, Keras), hyperparameter tuning with Ray Tune, and traditional ML training and batch inference such as scikit-learn and XGBoost models. It also lists high-performance computing, time series forecasting, feature engineering, and audio, image, and video processing.

ai_query is the general-purpose AI Function for task and model flexibility. You provide a prompt and choose any supported Foundation Model API. Databricks recommends it for large-scale batch inference in production workflows such as Lakeflow pipelines, Databricks workflows, and Structured Streaming.

ai_query is billed for the Model Serving endpoint you specify in the call. Databricks-hosted foundation model endpoints are billed like the task-specific AI Functions, while custom model and provisioned throughput endpoints run on their own dedicated serving compute. Choosing the endpoint is therefore the main lever for planning the cost of a batch inference job.

Databricks documents AWS GPU instances across several families: P5 (H100), P4d and P4de (A100), G6e (L40S), G6 (L4), G5 (A10G), G4dn (T4), and P3 (V100). The P3 family is being deprecated, so prefer a newer family for new clusters.

To reduce communication overhead in distributed training, Databricks recommends setting spark.task.resource.gpu.amount to the number of GPUs per worker node, which creates one Spark task per worker with all of that worker's GPUs assigned to it.

MLflow 3 evaluation asks whether a response is correct, grounded, safe, and complete. It scores traces with scorers, which are LLM judges or code-based checks, and each score is attached to the trace as feedback. You can run scorers from the UI to get a first quality read before moving to code.

The UI offers pre-built LLM-as-a-judge scorers such as Correctness, Completeness, and Fluency. You select traces, choose Actions and then Evaluate, and run the scorer to get an initial quality read. Each result is attached to its trace as feedback and recorded under the Evaluation runs tab.

Run mlflow.genai.evaluate() on an evaluation dataset to score a whole app version at once and compare versions. Start with UI scoring on a few traces, then move to this repeatable loop as your app matures. Repeatable scorers can move from the UI into the code loop, so your quality checks stay consistent as you iterate.

The Prompt Registry tracks prompt changes with Git-like versioning, including commit messages and rollback. Each version is an immutable snapshot with an auto-incrementing number, so you always have a stable record of each change. The registry is currently in Beta.

Create an online store with fe.create_online_store, then publish your feature table to it with publish_table. Prerequisites: a primary key constraint with non-nullable key columns, and Change Data Feed for TRIGGERED and CONTINUOUS modes. The online store is powered by Lakebase.

TRIGGERED is the default and incrementally updates the online table on demand or on a schedule. CONTINUOUS streams new writes from the offline table as they arrive. SNAPSHOT performs a one-time full copy of the offline table. Pick the mode that matches how fresh your real-time features need to be.

The feature table needs a primary key constraint, and primary key columns cannot contain NULL values. Change Data Feed must be enabled for the TRIGGERED and CONTINUOUS publish modes. Once these prerequisites are met, the feature table can be published to the online store.

For feature tables with a time series key, the online store keeps the latest value per entity plus all prior time-series values. Without a time series key, the online store keeps only the latest value per entity and removes duplicate rows during publishing; some exact-match historical lookup cases publish all rows without deduplication.

Define each feature with a FeatureLookup that names the feature table, the features to retrieve, and the lookup key. Build the training set with FeatureEngineeringClient.create_training_set, passing your DataFrame, the lookups, and the label column. Feature Engineering in Unity Catalog uses this client for these steps.

Train your model on the DataFrame returned by TrainingSet.load_df. Doing so guarantees that the joined features match what the model was trained on. Changes you make to that DataFrame afterward are not applied at inference, which can reduce model performance, so keep the training DataFrame as loaded.

Log the model with the log_model method of FeatureEngineeringClient and pass the training set. The model retains references to the features it was trained with. At batch scoring time, score_batch automatically retrieves those stored features before scoring.

Specify a timestamp_lookup_key in the FeatureLookup. Feature Store then retrieves the most recent feature value at or before each timestamp in that DataFrame column, matched on the primary key. Any FeatureLookup on a time series feature table must be a point-in-time lookup.

The lookback_window parameter excludes feature values that are older than the window from training and batch inference. Online serving always uses the latest value regardless of the lookback window. Point-in-time lookups do not skip rows with null feature values stored in the time series feature table, so handle nulls explicitly in your pipeline.

Databricks AutoML supports classification, regression, and forecasting. You can run experiments through a low-code UI for each problem type or through the Python API. AutoML cleans and prepares your data, then orchestrates distributed training and hyperparameter tuning across multiple algorithms to find the best model.

For classification and regression experiments, the data exploration notebook and the best trial notebook are imported into your workspace automatically, and notebooks for other trials are saved as MLflow artifacts. For forecasting experiments, notebooks for all trials are imported automatically. The generated notebooks let you review, modify, and reuse the training code.

Call automl.regress with a training DataFrame, the target_col that the model should predict, and a timeout_minutes value such as 30. When the run completes, follow the link to the best trial notebook, which contains the training code and a feature importance plot. For inference, load the model from summary.best_trial.model_path.

Data profiling produces a profile metrics table with summary statistics and a drift metrics table with statistics about how the data changes over time. Metrics are computed for the whole table and for the time windows and data slices you define. The metric tables are Delta tables in a Unity Catalog schema you choose, and a dashboard is created automatically for each profile.

You can specify an optional baseline table as a reference for measuring drift, and drift metrics then describe how the monitored table differs from that baseline over time. For inference profiles, the docs suggest your model's training or validation data as a good baseline. A representative baseline makes drift signals more meaningful.

Yes. Data profiling can track the performance of machine learning models and model-serving endpoints by profiling their inference tables. Inference profiles compare model performance and data quality metrics across time-based windows of the request log. This lets you spot degradation in a deployed model over time.

Register the agent in Unity Catalog, then call agents.deploy with its UC model name and version. The recommended MLflow 3 path requires MLflow 3.1.3 or later and databricks-agents 1.1.0 or later; the older MLflow 2 path uses MLflow 2.13.1 or later and databricks-agents 0.12.0 or later. For new agents, the docs recommend the Agent Bricks CLI, which deploys to Agent Runtime. Read query_endpoint from the deployment result for the query URL.

Deployment creates a scalable REST endpoint with load balancing and a Review App where stakeholders can give feedback. It also writes real-time traces to the MLflow experiment (also stored in AI Gateway inference tables) and offers production monitoring in beta. Deployments can take up to 15 minutes. For new agents, Databricks recommends the Agent Bricks CLI.

Databricks recommends Delta Lake tables for storing deep learning training data. For datasets too large to fit in memory, use streaming approaches such as PyTorch IterableDataset, Hugging Face datasets streaming, or Ray Data. This keeps data loading scalable as your datasets grow.

Databricks recommends starting development on a Single Node GPU cluster, which the docs describe as typically the fastest and most cost-effective option for deep learning model development. It suits fast, iterative development on small- to medium-size data. Scale to distributed training when the workload requires it.

A common rule of thumb is that when you increase the batch size by a factor n, you increase the learning rate by the square root of n. When tuning manually, change the batch size by a factor of 2 or 0.5, then keep tuning manually or with an automated tool such as Optuna.

Databricks Runtime for Machine Learning includes common ML and deep learning libraries such as XGBoost, LightGBM, PyTorch, TensorFlow, Optuna, and Hyperopt, and the release notes list the full set. For distributed training, the distributed packages for TorchDistributor, DeepSpeed, and Ray are included in the runtime. For GPU deep learning, Databricks recommends AI Runtime.

On serverless compute, autologging is not enabled by default, so call mlflow.<library>.autolog() explicitly for each integration you want to trace, such as mlflow.openai.autolog() or mlflow.langchain.autolog(). The automatic tracing page's headline says 30+ frameworks and providers, and its listings name 29 integrations: 9 LLM providers, 16 agent frameworks (LangChain and LangGraph listed separately), and 4 utilities. Each traced LLM call, tool call, and agent step is captured as a span.

MLflow Tracing records the inputs, outputs, latency, token usage, and cost of every intermediate step. This maps the full execution flow of an agent or chain, so you can find the cause of an issue instead of only seeing the final answer. Databricks says you use the standard MLflow SDK and OpenTelemetry, and you do not need to run a tracking server or maintain a version yourself.

Databricks documents mlflow.genai.evaluate() for scoring a whole app version on an evaluation dataset so you can compare versions. Pre-built LLM-as-a-judge scorers such as Correctness, Completeness, and Fluency are available, and you can reuse registered scorers or create new ones. The same scorers can run automatically on a sample of production traffic, and the docs recommend moving them into a repeatable evaluate loop as the app matures.

No. A model alias is a mutable, named reference to a registered model version, and Databricks uses names such as Champion for this. Aliases are not updated automatically: you move the alias with set_registered_model_alias, and batch workloads that load models:/<name>@Champion pick up the new version on their next run. The Models in Unity Catalog documentation does not describe a serving endpoint following an alias automatically, so plan an endpoint update when you change the production version.

ai_forecast is a table-valued Databricks SQL function that extrapolates historical time-series data into a future horizon, returning forecast values with upper and lower prediction bounds. You pass the observed table, a right-exclusive horizon, the time and value columns, and an optional group_col to forecast each group independently. Version 1 is the Public Preview default; version 2, in Beta, is recommended and requires the Predictive AI Functions preview. It needs a Pro or Serverless SQL warehouse.

Use ai_detect_anomalies, a Beta table-valued SQL function. For each point it estimates an expected value and prediction interval from the preceding history, then returns expected, lower_bound, upper_bound, anomaly_score, and is_anomaly columns. Pass the table as observed, name the time column and one value column per call, and add group_col to analyze separate series independently. The default anomaly threshold is 0.95. It requires a Pro or Serverless SQL warehouse and an admin-enabled Predictive AI Functions preview.

ai_top_drivers is a Beta table-valued function for contribution analysis. It compares a numeric metric between control rows (is_test = FALSE) and test rows (is_test = TRUE) to show which dimension segments contribute to the difference. Each result includes a contributor column plus metric_control, metric_test, change, relative_change, and support. If you omit dimensions, it considers the top 20 correlated columns. It requires Databricks Runtime 16.1 or later, a Photon-enabled Pro or Serverless warehouse, and the Predictive AI Functions preview.

Use ai_predict_class. It trains a classifier on rows where the target column is non-NULL and predicts the rows where it is NULL, returning only the scored rows with a <target>_prediction column and a 0-to-1 <target>_confidence column. Each call trains a new model and does not save it. You pass a constant target column and a constant array of feature columns, and the function handles one target per call. It is Beta and requires the Predictive AI Functions preview.

Use ai_predict_value, a Beta Databricks SQL function. It trains a regression model on rows where the target column is not NULL and predicts the target for rows where it is NULL, returning those rows with a <target>_prediction DOUBLE column. Like ai_predict_class, it trains a fresh model on each call without saving it, and it handles one target per call. It requires either a Pro or Serverless SQL warehouse or a cluster on Databricks Runtime 19 or later, and the Predictive AI Functions preview.

A LoggedModel is an MLflow 3 object with its own unique ID that links a model's artifacts, metadata, parameters, metrics, and the code used to generate it. It is created when you call mlflow.<model-flavor>.log_model(), persists across environments, and can be registered to the Unity Catalog model registry. You can search and filter LoggedModels with mlflow.search_logged_models().

The Databricks AI Runtime SDK, installed as databricks-sdk-air, replaces serverless_gpu. Starting with GPU environment version 6 it takes that role, and version 6 is the last GPU environment version that includes serverless_gpu. The distributed training documentation includes a migration guide for moving existing code over.

The Logs tab for job-backed AI Runtime workloads lets you search for a term, filter by node, log level (info, warning, or error), and time range, and view a single node in multi-node runs. Active runs stream new output live, and you can copy or download matching lines. Logs stay available on the Logs tab for 60 days, and runs created earlier keep their logs on the deprecated Artifacts tab.

Start with MLflow, which provides experiment tracking, model evaluation, and a production model registry. Model Serving is a distinct component for deploying models, and it relies on the MLflow Model Registry, so the two are closely integrated but not interchangeable. You only need the agent and generative AI products when your project calls for them.

For a beginner, start with the AI and machine learning get-started tutorial 'Train and deploy an ML model', which builds a scikit-learn classification model, tracks it with MLflow, and uses Hyperopt for tuning. Then try the deep learning tutorial that uses MLflow 3 tracking, or the no-code LLM and agent tutorial in the AI Playground.