Azure Databricks documentation archive

In this archive, you can find earlier versions of documentation for Azure Databricks products, features, APIs, and workflows.

Administration

• Workspace-level SCIM provisioning

Compute

• Create cluster UI (legacy)
• Cluster UI preview
• Install a library with an init script (legacy)
• Cluster-named init scripts (legacy)
• Global init scripts (legacy)

Dev tools

• Legacy UniForm IcebergCompatV1
• Transactional writes to cloud storage with DBIO
• Hive table (legacy)
• Skew join optimization using skew hints
• Koalas
• Manage libraries with %conda commands (legacy)
• Workspace libraries (legacy)
• Explore and create tables in DBFS
• FileStore
• Browse files in DBFS
• Databricks CLI (legacy)
• What is dbx by Databricks Labs?
• dbutils.library
• Migrate to Spark 3.x
• VSCode with Git folders
• VSCode workspace directory
• Pulumi Databricks resource provider

Governance

• External metastores (legacy)
• Create Unity Catalog managed storage using a service principal (legacy)
• Credential passthrough (legacy)

Machine learning and AI

• Optimized LLM serving
• Migrate optimized LLM serving endpoints to provisioned throughput
• Model serving (legacy)
• Share feature tables across workspaces (legacy)
• MLeap ML model export
• Distributed training with TensorFlow 2
• Horovod
• Model inference using Hugging Face Transformers for NLP
• Train a PySpark model and save in MLeap format
• Set up and considerations for ai_generate_text()
• Analyze customer reviews with ai_generate_text() and OpenAI
• Apache Spark MLlib and automated MLflow tracking
• Load data using Petastorm

MLflow

• Run MLflow Projects on Azure Databricks
• Quickstart R
• Quickstart Java and Scala

Notebooks

• No-code EDA with bamboolib

Release notes

• Databricks light
• Databricks runtime release notes (end-of-support)
• Unity Catalog GA release note

Repos and Git source control

• Git notebook versioning (legacy)

Resources

• Office hours

Security

• Audit log schemas for security monitoring
• Compute policies best practices

Storage

• Connecting Azure Databricks and Azure Synapse with PolyBase (legacy)
• Azure Blob storage file source with Azure Queue Storage (legacy)
• Azure Cosmos DB
• Structured Streaming writes to Azure Synapse
• Neo4j
• Read and write XML data using the spark-xml library
• Configure Delta storage credentials
• Connect to Azure Blob Storage with WASB (legacy)