GenAI, Agents & Vector Search

Databricks AI Search (formerly Vector Search) is a managed search service that builds indexes from Delta tables, streaming tables, materialized views, and managed Iceberg v3 tables. Indexes sync automatically as the source data updates, Databricks can compute embeddings for you or accept precomputed ones, and the indexes appear in and are governed by Unity Catalog. It supports retrieval-augmented generation and similarity search over data that already lives in your lakehouse. Row- and column-level permissions are not enforced on indexes, so apply application-level filtering where different users must see different results.

DBRX was Databricks' open large language model. Databricks no longer serves it: the hosted DBRX endpoints were retired, with pay-per-token ending April 30, 2025 and provisioned throughput ending December 19, 2025. Its architecture and weight-release details are not on the retirement policy page, so cite Databricks' original DBRX announcement for those.

Foundation Model APIs give pay-per-token access to Databricks-hosted base models, with priority pay-per-token for latency-sensitive applications and provisioned throughput for dedicated capacity. Models are served through Databricks Model Serving, which scales automatically as demand changes. For batch workloads, AI Functions optimized models are recommended for batch inference.

Yes. AI Search indexes support retrieval-augmented generation, so an agent can retrieve relevant context from an index and pass it to a model for grounded answers. Indexes sync automatically from Delta source tables as the data changes, and Databricks can compute embeddings for you or accept precomputed ones. Row- and column-level permissions are not supported on indexes, so apply application-level filtering when different users must see different results.

Yes. Genie lets you ask natural-language questions over data governed by Unity Catalog, and every answer is grounded in your organization's data and governed through Unity Catalog. For unstructured content, AI Search indexes built from Delta tables provide semantic retrieval for RAG and search applications, and those indexes appear in and are governed by Unity Catalog.

Agent Bricks is Databricks' agent developer platform for building, deploying, and governing AI agents. Agent Runtime hosts agents, and agent memory and sessions are backed by Lakebase. MLflow Tracing records each step of an agent's run and can store those traces in Unity Catalog, retrieval uses AI Search indexes, and model access is governed through Unity Gateway, which applies guardrails, rate limits and usage tracking.

Foundation Model APIs provide pay-per-token access to Databricks-hosted base models, with priority pay-per-token for latency-sensitive applications and provisioned throughput for dedicated capacity. The page does not name specific model families, so check its supported-models list for what is currently available.

Yes. Foundation Model APIs give pay-per-token access to Databricks-hosted models, and provisioned throughput gives dedicated capacity for fine-tuned variants. For batch inference, the docs recommend AI Functions optimized models. Start on pay-per-token and move to provisioned throughput when a workload needs reserved capacity.

Build retrieval-augmented generation by combining Databricks AI Search (formerly Vector Search) for retrieval with a model endpoint such as Foundation Model APIs for generation, all under Unity Catalog governance. AI Search builds a managed vector index from a Delta table and can sync automatically when the source table changes. Knowledge Assistant can also build a RAG chatbot over your documents and expose it as an endpoint.

Use the ai_query function in a SQL SELECT to call a Foundation Model API endpoint, a custom model endpoint, or an external model endpoint over a table of prompts or documents. It supports real-time and batch inference. Setting failOnError to false (Databricks Runtime 15.3 or above) makes the function return a STRUCT with the parsed response in a response field and any error in an errorMessage field, instead of failing the query, so a batch job can keep processing past individual row failures. The function requires Databricks Runtime 15.4 LTS or above and a supported Model Serving region.

Agent Bricks is Databricks' platform for building, evaluating, and deploying agents, with Unity Catalog governance. Agents built with any framework run on Agent Runtime, and MLflow Tracing records each step an agent takes so you can debug it. You can measure agent quality with LLM judges, custom scorers, and expert feedback before you ship.

Databricks AI Search (formerly Vector Search) builds a managed vector index from a Delta table, so you can add semantic search to an application without running separate vector infrastructure. The index can sync automatically when the source table changes. Embeddings can come from a model you choose, served through Foundation Model APIs or your own endpoint, or from precomputed vectors, and you query the index through a REST API or the Python SDK.

Foundation Model APIs give you pay-per-token access to Databricks-hosted foundation models, which you can call from SQL with the ai_query function or from Python. Provisioned throughput is available for production workloads that need dedicated capacity or fine-tuned variants of base models. Unity Gateway governs access to models, including external providers, and applies guardrails, rate limits and usage tracking.

Agent Bricks lets you measure agent quality with LLM judges, custom scorers, and expert feedback. MLflow Tracing records each step an agent takes, so you can see where a response went wrong. Use the results to compare agent versions and decide what to change before you ship.

Yes. Provisioned throughput supports fine-tuned variants of base models, including models trained on your proprietary data, and reserved capacity for production workloads. You can serve fine-tuned models through Model Serving with governance and monitoring. Check the Foundation Model APIs docs for the current fine-tuning workflow and supported models.

Databricks AI Search (formerly Vector Search) can sync its index automatically when the source table is updated, so semantic search and RAG results stay current without manual refreshes. Sync modes depend on the index type: storage-optimized endpoints do not support continuous sync, so check the sync mode for your endpoint. Direct Vector Access indexes are updated through the REST API.

Agent Bricks lets you build agents with any framework and deploy them on Agent Runtime, with AgentKit providing durable execution so runs survive restarts. Agents can connect to tools and MCP servers, retrieve content from AI Search, and use memory and sessions. Unity Catalog stores and governs the traces at scale.

Databricks SQL exposes vector_search, which queries an AI Search index from SQL, and ai_search, which is Beta. AI Search supports similarity search with metadata filtering, hybrid keyword-similarity search, and reranking.

Storage-optimized AI Search endpoints hold over one billion vectors at 768 dimensions, while Standard endpoints have a capacity of 320 million vectors at that dimension. The docs say storage-optimized queries add about 250 ms of latency, so Standard endpoints suit high-QPS, latency-sensitive serving.

With MLflow 3 you score traces with scorers: built-in LLM judges (for example Correctness, Completeness and Fluency), reusable registered scorers, or code-based checks. Run them over an evaluation dataset with mlflow.genai.evaluate() to score a whole app version and compare versions, then run the same scorers on a sample of production traffic to monitor quality. MLflow Tracing autologs supported frameworks such as LangChain, LangGraph and OpenAI, so each step of a run is visible.

Knowledge Assistant in Agent Bricks adjusts the agent's behavior from natural-language expert feedback. Experts add questions in the Examples tab and attach guidelines, and the guidelines apply right after they are saved. The docs label Knowledge Assistant a legacy offering.

Zepto's results are a customer case study that the docs do not cover. The documented building blocks are MLflow Tracing, which records the inputs, outputs and metadata of each step, and scorers, which are LLM judges and code-based checks that attach an assessment to each trace.

Databricks documents RAG as a pattern on its platform and suggests prototyping your own RAG agent in AI Playground. The docs do not compare RAG with fine-tuning.

Build RAG on Databricks with AI Search for retrieval, which was formerly called Vector Search, and Foundation Model APIs for managed LLM access. Evaluate with MLflow 3 GenAI evaluation, which uses mlflow.genai.evaluate with scorers.

Pair AI Search retrieval with structured attributes from Databricks Feature Store. Model Serving can look up feature values automatically at inference time, and feature serving exposes features to applications outside Databricks.

Databricks SQL offers AI Functions for LLM calls and document processing. ai_query is the general-purpose function for custom prompts against Foundation Model APIs and other serving endpoints. ai_parse_document parses PDFs and other documents, including PDF, JPG, PNG, TIFF, DOC/DOCX and PPT/PPTX, into structured elements such as text, tables and headers.

, Model Serving scales endpoints up or down with demand on serverless compute. It serves custom Python MLflow models, Databricks-hosted foundation models and externally hosted foundation models. The docs fetched do not describe pod-level autoscaling internals, GPU engines such as vLLM or Triton, or the telemetry export path.

ai_classify classifies text against custom labels. Its version 2.1 labels parameter accepts 2 to 500 unique labels. For taxonomies larger than 500 labels, the docs recommend embedding the documents and labels, retrieving the top k labels, and running ai_classify on that subset.

ai_prep_search is a Beta SQL function that prepares ai_parse_document output or plain text/Markdown for RAG by splitting it into chunks enriched with document context, and it requires Databricks Runtime 18.2 or above.

Yes. ai_query calls a Databricks model from SQL and can constrain output to a JSON schema with responseFormat, which requires Runtime 15.4 LTS or above and chat foundation models. AI Functions are not available on Databricks SQL Classic, so run them on a supported warehouse type.

Use ai_query rather than ai_gen when you need to set sampling. ai_gen takes only a prompt, while ai_query accepts model parameters through modelParameters (Runtime 15.3 or above), such as temperature, which defaults to 0.0 in ai_query. Test a few rows before scaling up, and combine with responseFormat to keep the JSON schema.

The Agent Bricks docs describe Genie Agents as the recommended low-code agent for governed data and documents, and the Agent Bricks CLI as the code-first path for custom agents. The CLI can run agents on Agent Runtime and serve them through Agent Server. The docs label Supervisor Agent and Knowledge Assistant as legacy offerings. The Genie One MCP server gives agents natural-language access to governed business data.

Unity Gateway provides identity-aware access control, usage tracking and budgets, and audit logs of AI activity for models, agents, MCP servers and tools. It can log requests and responses to Unity Catalog Delta tables for monitoring and debugging. MLflow deployment jobs manage the model lifecycle through evaluation, approval and deployment steps, with governance through Unity Catalog.

Databricks recommends MLflow ResponsesAgent for building agents, and it can wrap agents built with any third-party framework, including LangGraph. For new agents, the Agent Bricks CLI deploys to Agent Runtime on Databricks Apps using DurableAgentServer. MLflow Tracing records each step of an agent run.

The Supervisor Agent in Agent Bricks coordinates Genie Agents, Knowledge Assistant endpoints, model serving endpoints, Unity Catalog functions, AI Search indexes, MCP servers and custom agents. It can route to up to 50 agents and tools and respects end-user permissions. The docs label Supervisor Agent a legacy offering that is not recommended for new systems.

Omnigent is a Beta meta-harness that provides a common layer over Claude Code, Codex, Cursor, Pi and agents you write yourself. You can swap one line to change its harness or model while your tools, prompts, skills and policies stay the same.

MLflow on Databricks evaluates GenAI agents by running scorers, which can be LLM judges or code-based checks, that attach feedback to traces. You can reuse registered scorers or create new ones, and evaluation datasets let you compare agent behavior across test cases. For production, the same scorers can run automatically on a sample of production traffic to monitor quality over time. Traces record the execution path of the agent for debugging.

AI Search supports service principal and personal access token authentication, and the docs recommend service principals for production. Permission to create and manage endpoints is set through access control lists. Cross-workspace requests are possible through the Databricks SDK or REST API, but an index cannot be cloned into a different workspace. Row- and column-level permissions are not supported.

Parse documents with ai_parse_document, which returns text, tables and layout elements. Then use ai_extract with a JSON schema to pull structured fields into a Delta table that you can query with SQL.

Use ai_parse_document to parse PDFs, then ai_extract with a version 2.0 or later JSON schema to pull the specification fields. In version 2.1, a field's value is null when it cannot be extracted, so missing data is returned as NULL. ai_extract is available only in some regions.

Parse PDFs and other documents with ai_parse_document, then extract structured fields with ai_extract using a JSON schema. The output is a table you can query with SQL.

ai_query runs a prompt against a Foundation Model API or custom serving endpoint from SQL, so unstructured technician notes can be classified or standardized in SQL. Use responseFormat with a JSON schema to constrain the output (Runtime 15.4 LTS or above).

H3 cell IDs let you match locations on a shared hexagonal grid, and joining indexed datasets on cell IDs is a spatial join that does not need a spatial predicate.

AI Runtime is Databricks' fully managed serverless GPU compute for deep learning training and fine-tuning. The docs list on-demand A10s and H100s, support launching GPU jobs from the CLI, notebooks or Jobs, and allow Ray Core, Ray Data, Ray Train and Ray Tune to run on it.

Use an AI Search Delta Sync index with Databricks-computed embeddings to find similar supplier names, and query it with similarity search through the REST API or Python SDK. Then call ai_query with a JSON schema (responseFormat) to map each noisy name to a predefined category. The docs do not report accuracy for this approach.

Yes. AI Runtime is Databricks' fully managed serverless GPU compute for training and fine-tuning deep learning models. You can launch jobs from the CLI, notebooks or Jobs, and run Ray Train for distributed training. Data and models stay in your Databricks workspace.

Agent Bricks is the Databricks agent developer platform for building, deploying, querying, and monitoring agents. Agents can connect to Databricks-managed, custom, and external MCP servers, and the Genie One MCP server lets them ask natural-language questions over governed data. Databricks Sandbox gives agents an isolated environment to run the code they write with scoped access to governed data, and MLflow Tracing records each step an agent takes.

Genie is the Databricks AI experience for asking data questions in natural language, and every answer is grounded in your organization's data and governed through Unity Catalog. Data teams use Genie Agents to configure trusted data, metrics, and business rules for a domain. Genie One gives business users a simplified single entry point to ask data questions, explore AI/BI dashboards, and use Genie Agents. Together these let you answer ad-hoc analytics questions without building a custom chatbot.

Knowledge Assistant builds a chatbot that answers questions about your documents, using files in a Unity Catalog volume, a table with a file column, or an AI Search index, and it exposes the result as an agent endpoint for your applications. Ingestion skips files larger than 100 MB, and PDF, DOC, DOCX, PPT, and PPTX files with more than 500 pages, so split very large documents into smaller files. TXT and MD files have no page limit, and each PowerPoint slide counts as one page. Databricks labels Knowledge Assistant a legacy offering and recommends Genie Agents for new low-code agents.

Databricks AI Search (formerly Databricks Vector Search) builds indexes from a source table for retrieving embeddings. The docs describe three options: Delta Sync with Databricks-computed embeddings, Delta Sync with self-managed embeddings, and Direct Vector Access. Both Delta Sync options keep the index synced with the source table as it updates, which keeps retrieval current with little extra work.

Delta Sync indexes stay synced with their source table automatically. Direct Vector Access indexes give you direct control: you update them through the REST API when the embeddings table changes. Choose between the two based on how much control you want over index updates.

Yes. AI Search supports hybrid keyword-similarity search, which combines vector-based embedding search with traditional keyword techniques. You select Hybrid when you create an index to support both semantic and keyword search on the same index. Keyword-only queries are also available with query_type FULL_TEXT, which is a Beta feature.

Yes. AI Search supports filtering, and a query can define filters on indexed columns. If you leave the column list blank when you create an index, all columns from the source table are indexed. That lets you narrow vector results by any attribute you include in the index.

Only indexed columns can be returned in search results or used as filters. The primary key and embedding columns are always indexed, and indexing all source columns is the default when you leave the column list blank. Include every attribute you need to filter on in the index so it is available to your queries.

The best practices guide recommends keeping num_results in the range of 10 to 100 unless your application specifically requires more. The guide notes that a tenfold increase can roughly double latency, so this range keeps queries fast. It is a tuning guideline rather than a hard limit.

For real-time production use cases, the best practices guide recommends avoiding model endpoints that scale to zero, because cold starts can delay responses. Keeping embedding endpoints provisioned gives your application consistent latency. The guidance covers real-time production; batch and non-latency-sensitive workloads are not covered by this advice.

The best practices guide recommends standard endpoints when latency is critical and the index is well under 320M vectors. That guidance ties endpoint choice to both latency requirements and index size. Use the best practices page to choose the endpoint type for your index and latency target.

The best practices guide advises choosing the smallest dimensionality that preserves retrieval quality for your use case. Smaller dimensions improve performance: the guide notes that halving dimensions from 768 to 384 typically improves QPS by about 1.5x and reduces latency by about 20%. Test retrieval quality at a few dimensionalities to find the right trade-off for your data.

Self-managed embeddings mean you compute embeddings in advance and store the precomputed vectors in the source Delta table. This is one of the two Delta Sync options: a Delta Sync index can use Databricks-computed embeddings or self-managed embeddings, and the self-managed option applies when your source already holds precalculated vectors. The best practices guide says this approach enables the fastest retrieval.

The vector_search SQL function queries an AI Search index with SQL and returns a table of the top matching records, including all of the index columns. The index argument is the fully qualified name of an existing AI Search index in the same workspace. The function is in Public Preview and is available in regions where AI Search is supported.

The num_results argument is optional and defaults to 10, which returns the top 10 matching records. You can set it explicitly to return more or fewer matches. During the preview, num_results values above 100 are not supported, so set it within that range.

Databricks Runtime 15.3 or above lets you use query_text to search for a specific string of text in the embedding source column. Runtime 15.2 and below use the query argument instead. During the preview, filters_json and columns are not supported and Direct Vector Access indexes cannot be queried with this function, so check those limits when you write SQL against it.

ai_query invokes an existing Databricks Model Serving endpoint, parses its response, and returns the result. Its endpoint argument can name a Foundation Model serving endpoint, an external model serving endpoint, or a custom model endpoint. The function supports both real-time and batch inference, so one function covers interactive and large offline workloads.

The failOnError option is an optional boolean that defaults to true, which keeps endpoint errors visible in your results. When you set it to false, the function returns a struct containing the parsed response and an error message string, so a batch job can keep processing rows past individual failures. The option requires Databricks Runtime 15.3 or above.

Yes. The ai_query reference lists external model serving endpoints as a valid endpoint argument. Querying custom model endpoints and external model endpoints is enabled by default in supported regions. If a permission error occurs, a workspace admin can check the setting in Manage previews.

The labels argument of ai_classify takes between 2 and 500 labels, which covers most classification taxonomies. It is a STRING expression, and in version 2.x each label can include an optional description. For taxonomies with more than 500 labels, the docs describe an embedding-and-retrieval approach in their tutorial.

In version 2.x, the content argument of ai_classify accepts a VARIANT or STRING expression. That means you can classify raw text or the output of another AI function such as ai_parse_document. Version 2.1 can also return confidence scores and rationales when you enable them.

ai_extract pulls structured data from text and documents according to a schema you provide, and it accepts simple field lists as well as nested schemas with types and descriptions. In version 2.x it returns a VARIANT whose response field contains the extracted data, along with error_message and metadata. Optional settings such as citations and confidence scores are available through the options map.

ai_gen takes a single prompt argument of type STRING and returns a STRING. The function is in Public Preview, and its underlying model is tuned for English. The AI Functions overview lists ai_query as the general-purpose function for custom prompts and model choice.

Databricks SQL includes AI Functions grouped by purpose. ai_query is the general-purpose function for custom prompts and any supported Foundation Model API. Document-processing functions include ai_parse_document, ai_extract, and ai_classify, and ai_gen covers content generation. Text transformation functions include ai_fix_grammar, ai_translate, ai_summarize, and ai_mask, and vector_search is listed under vector search.

The AI Functions page introduces ai_query as the general-purpose function for task and model flexibility. You provide a prompt and choose any supported Foundation Model API, which makes it the choice for custom tasks. Specialized functions such as ai_classify and ai_extract cover narrower tasks, so pick the most specific function that fits your job.

Pay-per-token is the easiest way to start with Foundation Model APIs and is recommended for beginning your journey. For production, the docs recommend provisioned throughput for all production workloads, especially those that need high throughput, performance guarantees or fine-tuned models. For latency-sensitive production traffic on pay-per-token, priority pay-per-token is recommended: you opt in per request with service_tier set to priority, it needs no commitment, and it is billed at a higher per-token rate.

The Foundation Model APIs documentation recommends provisioned throughput for all production workloads, especially those that need high throughput, performance guarantees, or fine-tuned models. It gives you dedicated capacity for production traffic. The docs also recommend priority pay-per-token for latency-sensitive production workloads, so you can match the mode to each workload.

Provisioned throughput comes in two options. On-demand gives you dedicated capacity with no term commitment, so you can scale without a long-term contract. Reserved reserves dedicated capacity for a fixed term, with a 1-month or 3-month term, which suits planned, committed usage.

External models support OpenAI (including Azure OpenAI), Anthropic, Cohere, Amazon Bedrock, Google Cloud Vertex AI, and Databricks Model Serving endpoints with compatible schemas. A custom provider option covers alternate providers or models behind OpenAI API compatible proxies. AI21 Labs also has its own configuration section, so you can connect several providers from one place.

External models are third-party models hosted outside of Databricks, and they are configured on Model Serving endpoints. This lets you manage multiple LLM providers, such as OpenAI and Anthropic, through one consistent setup. The docs also describe storing API keys in one secure location, which strengthens your security posture.

On an AI Gateway (legacy) endpoint, select Rate limits and specify queries per minute (QPM) or tokens per minute (TPM). The Endpoint field sets a global maximum for all traffic to the endpoint, and you can add per-user defaults and custom limits for individual users, service principals, or groups. The newer Unity Gateway experience is generally available and is the enterprise control plane for governing LLM endpoints and coding agents.

Each endpoint can hold up to 20 rate limits, including up to 5 group-specific limits, so you can tailor throughput policies to teams. When both QPM and TPM limits apply, the more restrictive one is enforced. TPM limits cannot be applied to endpoints that serve custom models or agents.

Updates to AI Gateway (legacy) configurations take about 20 to 40 seconds to apply, and rate limiting updates can take up to 60 seconds. To change features on an existing endpoint, click Edit AI Gateway in the Gateway section of the endpoint page. Plan changes with that short delay in mind.

Databricks recommends restricting endpoint creation and CAN MANAGE to admins, which keeps guardrails and throughput limits enforced for end users. Give other users query permission on approved endpoints so they can use them without changing governance settings. The creator of an endpoint automatically receives CAN MANAGE.

Unity Gateway tracks requests, token usage, and latency through system tables, giving you a queryable record of consumption for monitoring and cost analysis. It also lets you attribute Databricks cost to services, target models, principals, and tags. Platform teams can use this to see how LLM usage is distributed.

Yes. Unity Gateway can log requests and responses to Unity Catalog Delta tables for auditing, monitoring, and debugging. The documentation frames this under auditing requests and responses. Because the logs land in Unity Catalog, they follow the same governance as your other data.

Unity Gateway can distribute requests across multiple model destinations and add failover to increase availability. This is how the gateway keeps AI applications available when one destination is not serving requests. Traffic distribution and failover are configured as part of the gateway's central routing.

Service policies are the mechanisms Unity Gateway uses to implement guardrails for AI services. They control how each request and response proceeds based on content and the caller, and built-in, custom, and external policies can be attached to model and MCP services.

MLflow tracing records the inputs, outputs, latency, token usage, and cost of every intermediate step in an agent or app, and the trace maps the full execution flow so you can see how each step behaves.

Call mlflow.langchain.autolog() before running your app, and autologging captures a trace with a span for each step. LangGraph uses LangChain's autolog, so the same call covers LangGraph steps. For supported frameworks, you get this instrumentation without manual tracing code.

Add custom spans with manual and custom tracing for the parts of your agent that autologging does not cover. Custom spans let you instrument your own functions and steps alongside the automatically traced framework calls. This gives you a complete trace of the agent even where autologging stops.

Run mlflow.genai.evaluate() on an evaluation dataset to score a whole app version at once and compare versions. As your app matures, you can move the scorers you use in the UI into this repeatable evaluate() loop. Results are recorded as traces with scorer feedback in your experiment.

MLflow scores traces with scorers, which are LLM judges and code-based checks that attach a quality assessment to each trace. You can select built-in LLM judges such as Correctness, Completeness, and Fluency, reuse a registered scorer, or create a new scorer. Each score is saved as feedback on the trace.

MLflow evaluation uses built-in and custom scorers to measure agent quality, such as built-in LLM judges like Correctness and RetrievalGroundedness. You iterate on your prompt and agent with those scorers to fix what you find. For production, the same scorers can run automatically on a sample of live traffic to catch regressions.

The MLflow Prompt Registry is a centralized repository for managing prompt templates across their lifecycle. Prompts are named entities in Unity Catalog, so they are governed like other data assets. Each prompt has versions and can carry version-specific tags. The registry is in Beta.

Prompt versions are immutable snapshots with auto-incrementing numbers, which gives you a stable record of every change. Aliases are mutable pointers, so an alias such as production can be reassigned to a different version without altering stored versions. The registry follows a Git-like model with commit messages and rollback, and tags are version-specific key-value pairs.

Databricks' agent docs recommend the MLflow ResponsesAgent interface for building agents, which lets you build agents with any third-party framework and integrate them with Databricks logging, tracing, evaluation, deployment, and monitoring. Wrapping an agent in this interface gives out-of-the-box compatibility with AI Playground, Agent Evaluation, and Agent Monitoring. The page labels this approach legacy and recommends the Agent Bricks CLI for new agents.

The deploy() function from the Agent Framework Python API deploys an agent on Model Serving. Deployment creates a scalable REST API serving endpoint with automatic load balancing for user-facing applications. It also provides a Review App where stakeholders can interact with the agent and give feedback.

For new agents, Databricks recommends the Agent Bricks CLI, which deploys agents on the Agent Bricks agent server and Agent Runtime. The Agent Framework docs still describe agents.deploy() for Model Serving, and they give a migration path for moving existing agents from Model Serving to Databricks Apps.

Use Unity Catalog functions when your agent needs to run predefined logic, such as a SQL query with agent-supplied parameters. Create and test the function with DatabricksFunctionClient, then set function_name to the function's full name in your framework example. Local mode retrieves the function from Unity Catalog and runs it in a local subprocess, which is useful during development.

Yes. Databricks Function Calling is OpenAI-compatible and is available during model serving, so it follows the OpenAI API format. Foundation Model APIs broadly support the function definitions accepted by OpenAI. Check the documented limitations for your model before relying on specific calling patterns.

Parallel function calling is not supported. During Public Preview, function calling on Databricks is optimized for single-turn calls, and the docs recommend supported Claude models for multi-turn calls. For provisioned throughput, function calling is supported on new endpoints, so plan your agent design around single-turn calls.

Specify your structured output with the response_format field in your chat request, and it works with supported chat models to generate data in a defined JSON format. The docs recommend a simpler JSON schema because it results in higher-quality JSON generation. For Claude Opus 5.5, Sonnet 5.5, and Haiku 5.5, use the format field of output_config with the Anthropic Messages API instead; response_format on those endpoints through Chat Completions returns HTTP 400.

Databricks provides managed MCP servers for Genie One, Databricks SQL, GitHub, Slack, and other tools, so you can use them without hosting a server. For business questions in natural language, the docs recommend the Genie One MCP server, which grounds answers in Genie Ontology. Databricks-provided MCP servers use Unity Catalog permissions.

Add a self-hosted or third-party MCP server through the MCPs UI, then share it with your team. Registering the server in Databricks lets teammates use it without wiring it into each agent separately. Unity Gateway controls access, manages credentials, and monitors usage for registered MCP servers.

You can build custom tools or turn an HTTP REST API into MCP tools with Databricks Apps templates. This lets agents query data, search documents, and call your services through a standard MCP interface. The custom server is built from the Databricks Apps templates.

Agent Bricks is the Databricks agent developer platform for building, deploying, and debugging agents. Agent Runtime hosts agents built with any framework on Databricks-hosted compute, with durable execution so runs survive restarts and crashes. It also connects agents to memory and context from Genie One and managed AI Search indexes, and MLflow Tracing records each step for debugging and evaluation.

Knowledge Assistant lets you create a chatbot that answers questions about your documents, and it creates an agent endpoint that your applications call to get answers. Databricks labels Knowledge Assistant a legacy offering. For new low-code agents, the docs recommend Genie Agents, and for code-based agents they recommend the Agent Bricks CLI.

A Knowledge Assistant can use three kinds of data sources: files in a Unity Catalog volume or volume directory, a Unity Catalog table with a file content column, and an AI Search index that uses one of the supported embedding models. You can provide up to 10 knowledge sources per assistant. Tables must be streaming tables or have Change Data Feed enabled, and only the selected content column is ingested.

The Supervisor Agent coordinates Genie Agents, agent endpoints, Unity Catalog functions, MCP servers, and custom agents to complete complex tasks across specialized domains. The page labels it a legacy offering: for new low-code agents, use Genie Agents, and to coordinate multiple agents in code, use the Agent Bricks CLI. Subagent access is limited to 50 agents and tools per supervisor.

Information extraction transforms a large volume of unlabeled text documents into a structured table with extracted information for each document. Source files must be in a Unity Catalog volume or table, and a selected folder must contain documents in a supported format. The feature is in Beta, and this page covers the older edition, so check the current guidance before starting a new project.

Agent Bricks Custom LLM, a legacy Beta offering, optimizes prompts on your behalf and automatically infers evaluation criteria from your instructions and examples. To start, provide at least 3 example inputs and expected outputs, and provide at least 100 inputs to optimize the agent. If you use a Unity Catalog table as input, its name cannot contain special characters such as hyphens.

Custom LLM supports string, int, and double data types for its inputs, so columns in the input table should use one of these types. You can provide a labeled dataset, an unlabeled dataset, or a few examples. For a labeled dataset, you choose the column that holds the expected output.

ai_query is the general-purpose AI Function for task and model flexibility, where you supply a prompt and choose a supported model. ai_classify classifies input text according to labels you provide, and ai_extract pulls structured fields from documents or text using a schema you define. AI Functions can be run from Databricks SQL, notebooks, Lakeflow pipelines, and Workflows, but they are not available on Classic SQL warehouses.

Databricks states that the compute that runs your query is always billed, for example a SQL warehouse or job cluster. Model inference can add a separate charge, and depending on the function it may run on separate Databricks-managed infrastructure. Check the pricing for the specific function you use.

Databricks advises submitting the full dataset in a single query rather than manually splitting it into small batches, because AI Functions automatically manage parallelization, retries, and scaling. The docs note that performance may not scale linearly from very small workloads to large-scale ones, so benchmark with realistic volumes. For ai_query, the page recommends Databricks-hosted foundation models (prefixed with databricks-) rather than provisioned throughput endpoints.

Pass the responseFormat named argument to ai_query. It accepts a DDL-style schema string such as STRUCT<name:STRING, age:INT>, or a JSON string with type text, json_object, or json_schema. It requires Databricks Runtime 15.4 LTS or above and applies only to chat foundation models. Adding failOnError => false returns a STRUCT with the parsed response in a response field and the error in an errorMessage field.

Pay-per-token is the easiest way to start accessing foundation models on Databricks and charges for tokens consumed. Provisioned throughput allocates dedicated capacity and Databricks recommends it for production workloads, especially those requiring high throughput, as well as fine-tuned models. Provisioned throughput is the better fit when you need dedicated capacity, while pay-per-token suits experiments and variable traffic.

Priority mode admits a request ahead of standard best-effort pay-per-token traffic, and it is billed at a higher per-token rate. Databricks recommends it for latency-sensitive workloads that need more consistent performance under load. It is still a pay-per-token option, so it does not give you the dedicated throughput of provisioned throughput.

On-demand provisioned throughput has no term commitment, and you can change or remove the allocation at any time, paying for the capacity you allocate. Reserved capacity covers a fixed 1- or 3-month term at a lower per-unit rate. Because you pay for allocated capacity, utilization determines whether provisioned throughput costs less than paying per token.

Databricks describes Agent Bricks as the Databricks agent developer platform. Its building blocks include Agent Runtime, which hosts agents from any framework on Databricks-hosted compute, and Agent Server, which handles synchronous, streaming, and background runs with crash recovery. It also offers connections to MCP servers, including a Genie One MCP server, and retrieval from AI Search indexes.

Yes. The Agent Bricks CLI scaffolds, runs, and deploys agent code, including from coding agents. Databricks Sandbox lets agents run the code they generate, with a home directory that persists across sessions. These capabilities sit alongside Agent Runtime for hosting agents on Databricks-managed compute.

Databricks describes Unity Gateway as providing one API for frontier and open models, so you can switch models without changing agent code. Unity Gateway applies guardrails, rate limits, and usage tracking to models, MCP servers, and skills. Observability uses MLflow Tracing to record agent steps, and evaluation uses LLM judges, custom scorers, and expert feedback.

Only with Continuous sync. Continuous sync automatically updates the index when the source table or view changes, with seconds of latency, but it runs a compute cluster for the streaming pipeline and costs more. Triggered sync does not update on its own: you start each sync from the UI, the Python SDK with index.sync(), or the REST API.

A Delta Sync index is built from a Delta table or materialized view and can be structured to sync automatically when the underlying source table is updated. A Direct Vector Access index requires manual updates through the REST API when the embeddings table changes. Choose Delta Sync when your embeddings already live in a Delta table that you want kept in step automatically.

Only Triggered sync. Databricks states that for storage-optimized endpoints only Triggered sync mode is supported, and continuous sync is not supported on them. The embedding dimension must be evenly divisible by 16 on these endpoints, and you start each sync explicitly.

Each endpoint supports up to 50 indexes, and each index supports up to 50 columns. The maximum embedding dimension per index is 4096, and Delta Sync row size is capped at 100 KB. Query results are capped at 10,000 for approximate nearest neighbor and full-text search and at 200 for hybrid search.

The column name _id is reserved in AI Search, so you must rename it in the source table before creating the index. Index names also cannot contain spaces, periods, forward slashes, or control characters. Checking these naming rules before creating the index avoids a failed build.

ai_summarize(content, max_words) returns a summary of a string using a Databricks Foundation Model API chat endpoint. The max_words value is a best-effort target and defaults to 50; setting it to 0 removes the word limit. It returns NULL for NULL input. The function is in Public Preview, requires Databricks Runtime 15.4 LTS or later (18.2 recommended), and is not available on Databricks SQL Classic.

Use ai_mask(content, labels). The labels argument is an ARRAY<STRING> literal naming the entity types to hide, such as array('person', 'email'), and matching information is replaced with [MASKED]. The documentation example turns "John Doe lives in New York. His email is john.doe@example.com." into "[MASKED] lives in New York. His email is [MASKED]." It returns NULL for NULL input, is in Public Preview, and requires Runtime 15.4 LTS or later.

Call ai_analyze_sentiment(content) on a STRING column. It returns one of four labels: positive, negative, neutral, or mixed, and NULL when sentiment cannot be detected. The function is in Public Preview, requires Databricks Runtime 15.4 LTS or later (18.2 recommended), and is tuned for English although the model handles several languages. It is not available on Databricks SQL Classic, and pricing is listed on the SQL pricing page.

ai_similarity(expr1, expr2) returns a FLOAT semantic similarity score between two STRING expressions, where 1 means the texts are equal. The score is relative and should be used only for ranking. It is in Public Preview, requires Databricks Runtime 15.4 LTS or later (18.2 recommended), and is tuned for English. It invokes a Foundation Model API model, and customers must ensure compliance with applicable model licenses.

ai_prep_search is a Beta Databricks SQL function that turns parsed documents or plain text into search-ready chunks. It accepts ai_parse_document output or a STRING and splits each document into semantic chunks, enriching each with context such as the title, section headers, and an LLM-generated context sentence. The output is a VARIANT holding chunk_to_retrieve and chunk_to_embed fields plus typed metadata. It requires Databricks Runtime 18.2 or later, and on serverless compute it requires serverless environment version 3 or later.

ai_search is a Beta Databricks SQL function that retrieves from one or more knowledge sources, including AI Search indexes and Lakebase synced tables with Lakebase Search enabled. It rewrites the query, deduplicates and reranks results across sources, and by default returns a grounded answer along with the ranked documents. Set generate_answer to 'false' for documents only, or enable generate_citations for source references. It requires Databricks Runtime 18.2 or later and at least one knowledge source.

ai_enrich is a Beta SQL function that adds derived columns to each row by having a model fill in fields from a schema you define, using the same grammar as ai_extract. Typical uses include tagging, categorizing, and filling missing metadata. The optional knowledge_sources argument grounds values in AI Search indexes or a live web search. Each field returns a rationale and a value by default, and setting enableRationale to 'false' returns only the value. It requires Databricks Runtime 18.2 or later.

Databricks recommends MLflow ResponsesAgent for building agents. It lets you build agents with any third-party framework and then integrate them with Databricks AI features. It supports custom_inputs and custom_outputs, and streaming emits output_text.delta events followed by a final response.output_item.done event. MLflow infers a signature that is compatible with Databricks product features automatically.

The Databricks agent docs describe two ways to deploy a ResponsesAgent. The Agent Bricks CLI deploys to Agent Runtime on Databricks Apps using DurableAgentServer and is recommended for new agents. The legacy MLflow AgentServer template deploys to Databricks Apps with databricks bundle deploy and databricks bundle run. The author-agent page does not cover Model Serving as a deployment target.

Use make_judge from mlflow.genai.judges with a name, natural-language instructions, and a feedback_value_type such as a Literal of allowed labels, plus an optional model. Instructions can use only the template variables {{ inputs }}, {{ outputs }}, {{ expectations }} and {{ trace }}, as other variables such as {{ question }} raise validation errors. A judge that analyzes {{ trace }} must specify the model argument.

Databricks recommends review queues (Beta) for new human-review workflows. A review queue combines the structured questions you define with reviewer assignment, and it works on both traces and evaluation dataset records. Reviewers can give Feedback on trace outputs or record ground-truth Expectations. A workspace admin enables it under Manage previews, MLflow Review Queues, and the Python SDK in mlflow.genai.databricks.review_queues requires MLflow 3.17.1 or later.

A labeling session is a special MLflow run that holds a set of traces for domain experts to review in the Review App. It collects Feedback or Expectation labels on existing traces to support systematic agent evaluation. Reviewers are assigned by user email, and the session appears in the Evaluations tab. The questions reviewers see come from a labeling schema, which can be a built-in schema such as EXPECTED_FACTS, EXPECTED_RESPONSE, or GUIDELINES, or a custom one. Databricks recommends review queues for new workflows.

AI Playground is a chat environment under AI/ML, then Playground, for interactively testing and comparing LLMs and prototyping tool-calling agents and question-answering bots. Select the plus icon to add endpoints and see several model responses side by side. The documentation recommends the Agent Bricks CLI for building and testing agents in code, so use the playground to compare and choose a model before you write the agent.

Parent-child retrieval, also called small-to-big retrieval, stores small child chunks for precise matching and links each one to a larger parent chunk. The search runs over the children, and the parent text is returned to the model for more context. The AI Search retrieval quality guide describes this pattern and suggests testing chunk sizes of 256, 512, and 1024, since smaller chunks pinpoint facts while larger chunks preserve surrounding context.

Yes, the AI Search retrieval quality guide recommends adding document summaries and section headers to chunks. You can prepend them to the chunk text, which gives the embedding model more semantic signal, or store them as separate metadata columns, which rerankers and hybrid search can use if you configure them to read those columns. The guide focuses on metadata extraction, high-quality parsing, and measuring relative improvements across strategies rather than tuning chunk size alone.

The Databricks RAG pipeline guide says chunking should weigh chunk strategy, chunk size, overlap, semantic coherence, and metadata. Smaller chunks capture specific details but can lose surrounding context, while larger chunks keep context but add noise and cost. Overlap helps preserve continuity between neighboring chunks, and chunks should follow document structure such as paragraphs or sections. The guide states there is no one-size-fits-all approach, so the right method is iterative and depends on the use case.

AI Search endpoints come in Standard and Storage Optimized types. You can create them in the UI under Compute, then AI Search, with the Python SDK, through the REST API, or as a bundle resource, and optionally assign a usage policy for cost tracking. If Databricks computes embeddings, you can use a pre-configured Foundation Model APIs endpoint or a new serving endpoint, and Databricks recommends turning off the scale-to-zero default. Storage Optimized endpoints support only Triggered sync mode, while Standard endpoints support both Triggered and Continuous.

ai_translate(content, to_lang) translates a STRING into a target language inside a SQL query. The to_lang argument accepts IETF BCP 47 codes, full language names, or descriptive phrases. The documented supported languages are English, French, German, Hindi, Italian, Portuguese, Spanish, and Thai. For example, SELECT ai_translate('Hello, how are you?', 'es') returns a Spanish greeting. The function is in Public Preview and requires Databricks Runtime 15.4 LTS or later.

Use ai_transcribe, a Beta SQL function that converts audio into time-stamped segments with automatic speaker labels, returned as a VARIANT that other AI functions can consume. The input can be a FILE expression or BINARY bytes, such as the content column from read_files with the binaryFile format on a Unity Catalog volume. It supports MP3, WAV, FLAC, Opus, and M4A, but not video, and English and Spanish are the supported languages. An admin must enable AI Transcribe on the Previews page.

ai_fix_grammar(content) takes a STRING and returns the corrected text, returning NULL when the input is NULL. It is a task-specific AI function that calls a Databricks Foundation Model API chat endpoint, so you apply it directly to a text column in SQL. The function is in Public Preview and requires Databricks Runtime 15.4 LTS or later (18.2 recommended). It is tuned for English, is not available on Databricks SQL Classic, and pricing is listed on the SQL pricing page.

ai_query is the general-purpose function for custom prompts: you provide any prompt and choose any supported Foundation Model API model. ai_gen also accepts a free-form prompt, but the overview groups it under content generation. The other functions are task-specific and each does one job, such as ai_extract, ai_classify, ai_summarize, ai_mask, ai_translate, ai_analyze_sentiment, ai_forecast, and ai_parse_document. The overview groups functions by task so you can pick a narrow function for a known task and use ai_query for custom prompts.

The Databricks reranker re-scores retrieved results against the query to improve ordering. num_results sets how many results are returned and does not change how many results are used for reranking. Only the first 2,000 characters across the listed reranking columns are considered, so column order matters. If the reranker is temporarily unavailable, results return unreranked and the debug output flags it.

Vector Search returns up to 10,000 results for approximate nearest neighbor and full-text queries, and up to 200 for hybrid queries. Use the higher ceiling when your application needs a deep candidate set, and size hybrid requests within their 200-result cap.

Each AI Search endpoint supports up to 50 indexes, and each index supports up to 50 columns. A workspace can have up to 500 AI Search endpoints according to the AI Search resource limits. A standard endpoint holds about 320 million vectors at 768 dimensions for Delta Sync indexes, and a storage-optimized endpoint holds over one billion at 768 dimensions.

Vector Search filters can use only indexed columns, and you define them at query time. The index itself does not support row- or column-level permissions, so the documented pattern is to build application-level ACLs with the filter API so each user's query returns only what they are allowed to see.

Yes. MLflow Tracing auto-traces OpenAI, LangChain, LlamaIndex, DSPy, and Pydantic AI. Enabling a library takes one line, such as mlflow.openai.autolog(), and you can enable several integrations at once to get a single trace that combines steps from each. On serverless compute, call the autolog function explicitly.

Databricks encrypts and securely stores the credentials you provide for each external model provider. You can reference a Databricks secret for each key instead of pasting it in as plaintext. Credentials are deleted when their serving endpoint is deleted, so cleanup removes them along with the endpoint.

A memory pipeline connects one session store to one memory store, and it is in Beta. For each actor, it distills durable facts from new conversations with an LLM, merges and deduplicates them into topic-based entries in the memory store, and consolidates overlapping entries over time. Each actor has a checkpoint, so later runs pick up where the last one stopped.

Yes, in Beta. Managed agent memory provides long-term memory that persists across conversations, with search that ranks entries by full-text (BM25) and semantic matching. Databricks stores the memory in Lakebase, so you do not build that storage layer yourself. For short-term history within a single interaction, use agent sessions.

Yes. The Agent Bricks CLI, in Beta, scaffolds a custom agent project, runs it locally with agentbricks dev, and deploys it with agentbricks deploy. Tool bindings are declared in agent.toml, and MLflow tracing is on by default. No workspace setting is required to enable it.

Yes, in Beta, and it requires the Enhanced Unity Gateway beta to be enabled by an account admin. The unified trace table captures requests and responses across all Unity Gateway services into one Unity Catalog table in OpenTelemetry format. Trace delivery is best-effort, so it complements rather than replaces Databricks audit logs.

Yes. The built-in Jailbreak guardrail can be selected when you create a service policy, and built-in guardrails take no policy-specific configuration. On MCP services it can screen inputs, outputs, or both, and adding an input guardrail lets you check a tool call before it runs. On model services it runs on input only. Service policies are in Beta and require the Unity Gateway Service Policies preview.

Unity Gateway sends content to your guardrail vendor's endpoint through a Unity Catalog HTTP connection that uses OAuth M2M. The vendor returns ALLOW or DENY, and in Enforce mode a DENY blocks the call. Log mode records the verdict without blocking. Policies fail closed, so timeouts, errors, or unrecognized verdicts result in a denial. The feature is in Beta and needs the Unity Gateway Service Policies preview.

You can start from agent traces rather than writing a dataset from scratch. The MLflow 3 tutorial simulates traffic to create traces, saves them to an evaluation dataset, runs mlflow.genai.evaluate() with built-in LLM judges, and has you investigate the test cases where the app performed poorly. You then create an improved version and compare the results against the previous one. The tutorial does not give a minimum dataset size, so in practice start from real traces and let the dataset grow as you collect more.

Pre-launch test sets only cover the cases someone thought to write, so evaluation works best when it is repeated on traces from real use. MLflow 3's evaluation workflow saves traces to a dataset, scores them, and surfaces the test cases where the app performed poorly. Use those failing cases to drive the next iteration of the agent and the evaluation dataset. Offline evaluation is a snapshot of quality, so pair it with ongoing review of production answers.

Databricks' documentation treats LLM judges as one kind of scorer among several rather than the only check. MLflow 3 scorers include built-in and custom LLM judges, natural-language guidelines for custom requirements, and deterministic code-based scorers for checks such as exact matching or format validation. Judges default to a Databricks-hosted LLM, and you can change the judge model with the model argument. Checks that can be written as code, such as output format, can run as code-based scorers instead of being judged by an LLM.

Yes, as a Beta feature: AI Gateway service policies include a built-in system.ai.block_hallucination policy that runs on the output phase, after the model responds and before the response returns to the caller. It denies responses containing hallucinated content, and workspace admins opt in from the Previews page. Because the evaluator is itself a model, its verdicts are non-deterministic. Use it as one layer alongside evaluation rather than as proof that every answer is factually correct.

The AI Gateway tutorial walks through attaching built-in Unsafe Content guardrails to input and output (Before the Model and After the Model), and it does not cover the hallucination guardrail. Hallucination detection is a built-in service policy, system.ai.block_hallucination, that runs on the output phase and denies hallucinated responses. It is in Beta and requires the Unity Gateway Service Policies preview, which admins enable on the Previews page. Keep MLflow scorers running on the same agent as well.