Canary vs Blue-Green Rollouts on Databricks Model Serving

1.First choose the inference mode, then the rollout pattern

A rollout strategy only matters if something is serving live requests. The MLOps guidance separates two cases. Batch and streaming pipelines load whichever model version holds the Champion alias in Unity Catalog. They pick up a new version the next time they run, so moving the alias is the deployment, and no traffic needs shifting. Real-time use cases are different: they need a REST endpoint, and Databricks builds that with Model Serving. Cost is the main factor in this choice. Batch or streaming is the cheaper option when you need high throughput and can accept higher latency. When predictions must come back quickly but can be computed ahead of time, the guidance suggests publishing batch predictions to an online key-value store rather than serving them live.

Putting the strategies side by side shows where each one fits a high-traffic application. Batch and streaming deployments have no live endpoint to protect: the deployment is an alias move, and the cost advantage grows with throughput. They suit high volume only when callers can wait. When callers cannot wait, a Model Serving endpoint is the route. It is described as highly available and low latency, and it can support over 25K queries per second with an overhead latency of less than 50 ms. Two rollout patterns then apply on that endpoint. A canary shifts a small percentage of traffic to a new served entity through the endpoint's traffic split, which limits how many requests a bad model can reach. A blue-green cutover builds the new configuration while the old one keeps serving, then moves all traffic in one update. For a high-traffic endpoint, the canary is the safer choice when you are unsure of the new model, because the blast radius is a fraction of requests. Blue-green is simpler and fits a version you have already validated offline. The next sections cover each pattern in detail.

Sources12

2.Canary: send a small share of traffic to the new version

The Databricks documentation does not use the words "canary" or "blue-green". This lesson uses them for two patterns the documentation does describe. A canary rollout serves the new version next to the current one on the same endpoint, sends it a small share of requests, and raises that share as the metrics hold up. In the documented example, an endpoint called multi-model has a served entity named current (model-A version 1) getting 90% of traffic and one named challenger (model-B version 1) getting 10%. Each served entity has its own entity_name, entity_version, workload_size and scale_to_zero_enabled. Because the challenger is sized separately, you can provision it before it takes much traffic.

      "traffic_config":
      {
         "routes":
         [
            {
               "served_model_name":"current",
               "traffic_percentage":"90"
            },
            {
               "served_model_name":"challenger",
               "traffic_percentage":"10"
            }
         ]
      }

The MLOps workflow treats this as an online comparison between the Champion and a Challenger: an A/B test or a gradual rollout that may run for a long time. One limit is easy to miss. Every served entity on an endpoint must be the same kind of model. You cannot run a canary that pairs a custom model with an external model on one endpoint.

Sources31

3.Blue-green: build the new configuration, then switch all traffic

The blue-green pattern works the same way. The new version is built completely while the old one keeps serving, and then 100% of traffic moves at once. On Model Serving you get this from an ordinary config update that names the new entity_version with a single route at 100%. The platform does not switch traffic until the new configuration is ready. If the update fails, nothing breaks: the current configuration stays live as though the update never happened. The cost is that once the cutover succeeds, every request goes to the new version. If it misbehaves, rolling back takes another update.

Sources14

4.Which pattern fits a high-traffic endpoint

Two documented facts shape this choice. A configuration update can take 10–25 minutes, and only one update can run at a time. A canary that goes from 1% to 10% to 100% is therefore three update cycles, and rolling back is a fourth. Because changes happen in sequence, the canary's real advantage is limiting how many requests a bad model can reach. A blue-green cutover finishes in one update but puts all traffic on the new version immediately. On a high-traffic endpoint, a bad version then affects every request until a rollback update completes.

Whichever pattern you use, high traffic also depends on how each served entity is configured. Route optimization is recommended for endpoints with high QPS. Size Compute Scale-out to the number of requests the entity must handle at once, roughly QPS × model run time: Small covers 0–4 concurrent requests, Medium 8–16 and Large 16–64. If you set min_provisioned_concurrency and max_provisioned_concurrency yourself, the values must be multiples of 4. Scale to zero is not recommended for production endpoints, because capacity is not guaranteed and a cold start adds latency when the endpoint scales back up. That applies to a challenger as much as to the current version.

Sources45

Exam traps

Each one states something that sounds right. Open it to see what is actually true.

1. 1.Changing traffic_percentage takes effect immediately, so a canary can be rolled back in seconds.Why is that wrong?

   A traffic change is a config update. Updates can take 10–25 minutes and cannot overlap, so each step and each rollback is a full update cycle.

   Covered in Which pattern fits a high-traffic endpoint

2. 2.A blue-green cutover on Model Serving causes an outage while the new version loads.Why is that wrong?

   Model Serving keeps the old configuration serving until the new one is ready, so the update has no downtime.

   Covered in Blue-green: build the new configuration, then switch all traffic

3. 3.Turning on scale to zero for a challenger in production saves money without risk because it only gets 10% of traffic.Why is that wrong?

   Databricks does not recommend scale to zero for production endpoints: capacity is not guaranteed and scaling back up adds cold-start latency.

   Covered in Which pattern fits a high-traffic endpoint