Deploying Bundles: bundle deploy, Deployment State, Development vs Production Modes and Bind

1.bundle deploy, bundle identity and deployment state

databricks bundle deploy pushes a bundle to a remote workspace. Pass -t (or --target) to choose a target. If you pass no target, the CLI uses the bundle's default target. The same bundle can go to development, staging and production workspaces, and each of those deployments has its own identity. That identity comes from root_path, which defaults to ~/.bundle/${bundle.name}/${bundle.target}. So by default a deployment is identified by who deployed it, the bundle name and the target name. If two bundles share all three, their deployments will interfere with each other.

A deployment stores state in the workspace file system that records the ID of each resource it created. On every deploy, a resource that is in the configuration but not in the workspace is created. A resource that is in both is updated. A resource that was deployed before but has since been removed from the configuration is deleted from the target workspace. The only way to make a deployment forget a resource is to change the bundle name, the target, or the workspace. databricks bundle validate prints a summary of those values.

Sources1

2.Deploy flags and when to use each

--force and --force-lock are easy to mix up. --force gets past validations, such as deploying from a branch other than the target's configured git.branch. --force-lock is about the deployment lock, which stops concurrent deployments from interfering with each other. --select takes resource keys, optionally qualified by type, and you can repeat it or comma-separate the keys:

databricks bundle deploy --select my_job
databricks bundle deploy --select jobs.my_job --select pipelines.my_pipeline
databricks bundle deploy --select my_job,my_pipeline

Sources1

3.Development and production deployment modes

A target can set mode, which applies a group of default behaviours that match a stage of CI/CD. Modes are optional, and you can deploy without one. The two modes behave very differently:

targets:
  dev:
    mode:  ? 

Databricks recommends service principals for production deployments, enforced by setting run_as to a service principal. Without one, production mode adds extra checks. artifact_path, file_path, root_path and state_path must not be overridden to a specific user, and run_as and permissions must be specified. Deploying to an immutable, read-only folder means non-admin users can't change deployed assets and running jobs aren't affected by a deployment in progress.

Presets let you change individual mode behaviours. Unless a preset documents an exception, presets override the mode, and a resource's own settings override the presets:

targets:
  dev:
    presets:
      name_prefix: 'testing_' # prefix all resource names with testing_
      pipelines_development: true # set development to true for pipelines
      trigger_pause_status: PAUSED # set pause_status to PAUSED for all triggers and schedules
      jobs_max_concurrent_runs: 10 # set max_concurrent runs to 10 for all jobs
      tags:
        department: finance

Sources2

4.Bringing existing resources under a bundle with bind

Jobs and pipelines created in the UI or through the API can be adopted rather than recreated. databricks bundle generate writes YAML for an existing resource, and databricks bundle deployment bind KEY RESOURCE_ID links a bundle resource key to the existing object. After the next bundle deploy, the workspace resource is updated to match the bundle's configuration. Bind does not recreate data. For example, a bound pipeline keeps its data and its materialized views don't have to be recomputed. Use bind with --target. It prints a diff and asks you to confirm unless you pass --auto-approve. databricks bundle deployment unbind KEY removes the link.

databricks bundle deployment bind hello_job 6565621249

Sources1

Exam traps

Each one states something that sounds right. Open it to see what is actually true.

1. 1.Renaming a job's name field in the YAML detaches it from the deployed job, so the next deploy creates a fresh job.Why is that wrong?

   The deployment tracks resources by ID in its state, not by name. A deployment forgets a resource only if the bundle name, target or workspace changes.

   Covered in bundle deploy, bundle identity and deployment state

2. 2.--select is a safe way to push one hotfixed job to production without redeploying the whole bundle.Why is that wrong?

   --select is recommended for development workspaces only. It deploys the selected resources and their upstream dependencies but leaves downstream resources stale, which can cause dependency errors.

   Covered in Deploy flags and when to use each

3. 3.Production mode, like development mode, lets you point every job at a different cluster with --cluster-id at deploy time.Why is that wrong?

   Cluster overrides are a development-mode behaviour. Production mode does not allow overriding the cluster definitions in the configuration.

   Covered in Development and production deployment modes

Practise it for real

Create a bundle from a template, deploy it to a development target, and run its job.

1. 1.Run databricks auth login --host <workspace-url> and accept the suggested profile name.

   Why: The CLI needs an authenticated profile for the target workspace; U2M is fine for interactive work.

   You should see: A browser login, after which a configuration profile is saved.

2. 2.Run databricks bundle init and choose the default Python template.

   Why: Templates are the quickest way to get a valid databricks.yml, a notebook and a job definition.

   You should see: A new project folder containing databricks.yml and resource files.

3. 3.From the project folder, run databricks bundle validate.

   Why: Validation catches configuration errors and warns about unknown resource properties before anything is deployed.

   You should see: A summary of the bundle name, target and workspace, plus any warnings.

4. 4.Run databricks bundle deploy -t dev.

   Why: This deploys to the development target; if it uses mode: development, resources get the [dev <your name>] prefix and schedules are paused.

   You should see: The job appears in the workspace with the dev prefix.

5. 5.Run databricks bundle run -t dev with the job's resource key.

   Why: bundle run starts the deployed workflow from the command line.

   You should see: A job run starts and its progress is reported.