What you will be able to do
- Read and write CREATE POLICY statements for row filter and column mask policies
- Predict which rows a row filter UDF returns and which values a column mask UDF produces
- Trace how Unity Catalog evaluates policies at query time
- Explain fail-closed behaviour, and what breaks when a tag or UDF that a policy depends on is removed
1.The CREATE POLICY statement and its prerequisites
Before you can create a row filter or column mask policy, three things must be in place:
- Databricks Runtime 16.4 or above, or serverless compute. - A UDF in Unity Catalog that you have EXECUTE on, or an inline SQL function, to carry the filtering or masking logic. - Governed tags applied to the target objects.
You also need MANAGE on the object the policy attaches to, or ownership of it. Attaching at the metastore level (Beta) requires a metastore admin. The general shape of the statement is:
CREATE [OR REPLACE] POLICY policy_name
ON { METASTORE | CATALOG catalog_name | SCHEMA schema_name | TABLE table_name }
[COMMENT description]
{ row_filter_body | column_mask_body }| Clause | Meaning |
|---|---|
| TO principal [, ...] | Users, groups or service principals the policy applies to |
| EXCEPT principal [, ...] | Principals exempt from filtering or masking |
| FOR TABLES | Targets tables, including views (policies on views are Beta) |
| WHEN condition | Tag expression choosing which tables apply; defaults to TRUE if omitted |
| MATCH COLUMNS condition AS alias | Up to 3 column conditions, all of which must match |
| ON COLUMN alias | Column mask only: which matched column to mask |
| USING COLUMNS (function_arg [, ...]) | UDF arguments: MATCH COLUMNS aliases, constant literals, or tag introspection functions |
Policy names must be unique among the policies on the same securable. In a condition you can combine has_tag and has_tag_value with AND, OR and NOT. The two clauses read tags differently: in a WHEN condition they check tags set directly on the table or inherited from a parent catalog or schema, while in MATCH COLUMNS they check tags set directly on the column only, so a table tag never matches a column condition.
USING COLUMNS accepts aliases from MATCH COLUMNS and constant values (quoted strings, numeric literals, TRUE/FALSE, or NULL), in the order the function expects them. It also accepts tag introspection functions, which extract a tag's value at query time and pass it to the UDF.
The next policy combines both kinds of condition. It masks name and address columns only on tables where the classification : unverified tag is no longer present:
CREATE FUNCTION catalog.schema.mask_pii(val STRING)
RETURNS STRING
RETURN '***';
CREATE POLICY mask_reviewed_pii
ON CATALOG my_catalog
COLUMN MASK catalog.schema.mask_pii
TO `account users`
EXCEPT `data_admins`
FOR TABLES
WHEN NOT has_tag_value('classification', 'unverified')
MATCH COLUMNS (has_tag_value('pii', 'name') OR has_tag_value('pii', 'address')) AS m
ON COLUMN m;Checkpoint 1 of 5· Check yourself
A row filter policy is attached ON SCHEMA prod.customers and has no WHEN clause. Which tables does it apply to?
An omitted WHEN defaults to TRUE, so the policy covers every table in its scope.
“If omitted, defaults to TRUE (applies to all tables in scope).”Source: docs.databricks.com
2.Row filter policies: a boolean decides each row
A row filter policy references a UDF that evaluates each row and returns a boolean. Rows where it returns TRUE are kept, and rows where it returns FALSE are dropped. Use one when access depends on the values in each row, such as a region column. Column values reach the UDF through USING COLUMNS. In the example below, every table in the sales catalog that has a column tagged region is filtered for the EMEA team so they see only EMEA rows. No table is named anywhere.
CREATE FUNCTION filter_by_region(region STRING, allowed STRING) RETURNS BOOLEAN
RETURN region = allowed;
CREATE POLICY regional_access_emea
ON CATALOG sales
ROW FILTER filter_by_region
TO `emea team`
FOR TABLES
MATCH COLUMNS has_tag('region') AS rgn
USING COLUMNS (rgn, 'EMEA');A row filter can also block a whole table. In one common pattern, new objects are tagged classification : unverified, and a row filter whose UDF hides every row applies to any table with that tag. When a data steward finishes classifying the data and changes the tag, the blocking policy no longer matches.
Checkpoint 2 of 5· Fill the gap
Which value makes this policy hide every row of unverified tables from non-admin users?
-- Block access to unverified tables for all non-admin users
CREATE FUNCTION catalog.schema.block_all() RETURNS BOOLEAN
RETURN ? ;
CREATE POLICY block_unverified
ON CATALOG my_catalog
ROW FILTER catalog.schema.block_all
TO `account users` EXCEPT `data_admins`
FOR TABLES
WHEN has_tag_value('classification', 'unverified');Rows for which the filter returns FALSE are excluded, so a function that always returns FALSE hides every row from the principals in TO who are not in EXCEPT. A row filter UDF returns a boolean, so a string such as 'blocked' is not a valid result.
Source: docs.databricks.comCheckpoint 3 of 5· Exam question
A healthcare organization tags any column that stores a social security number with the governed tag `pii = ssn`, across dozens of tables in a catalog, and expects new tables added later to be protected automatically. Which Unity Catalog capability meets this requirement without per-table maintenance?
Correct answer: A — An ABAC column mask policy at the catalog level with `MATCH COLUMNS has_tag_value('pii','ssn')`, applied automatically to any current or future matching column.
- A. This is correct: a catalog-level ABAC column mask policy that matches on the `pii = ssn` tag is evaluated dynamically, so any column carrying that tag today or added to a future table is masked without further changes.
- B. This is incorrect. There is no Delta table constraint that redacts columns by matching their name; masking in Unity Catalog is implemented through column mask policies and functions, not table constraints.
- C. This is incorrect. A per-table mask statement only affects the table it was run against; Databricks does not automatically propagate it to tables created afterward.
- D. This is incorrect. A view's masking logic is scoped to that specific view definition and is not inherited by unrelated tables created later, so each new table would still need its own view.
3.Column mask policies: return the original or a redacted value
A column mask policy leaves every row in place and changes what a column shows. Its UDF takes the column value and returns either the original value or a masked one. Use it to redact SSNs, phone numbers or email addresses while still letting the principal query the table. The column to mask is the MATCH COLUMNS alias named in ON COLUMN. Its value is bound automatically as the UDF's first argument, and USING COLUMNS supplies any extra arguments.
CREATE FUNCTION mask_ssn(ssn STRING, show_last INT) RETURNS STRING
RETURN CONCAT('***-**-', RIGHT(ssn, show_last));
CREATE POLICY mask_ssn_columns
ON CATALOG hr_catalog
COLUMN MASK mask_ssn
TO `account users` EXCEPT `compliance team`
FOR TABLES
MATCH COLUMNS has_tag_value('pii', 'ssn') AS ssn_col
ON COLUMN ssn_col
USING COLUMNS (4);The mask's return type must match the column's data type or be castable to it, so check that every branch of the function returns a compatible value. The Databricks docs give a salary example: a function that returns DOUBLE in every branch (with 0.0 as the fallback) works for a DOUBLE column, while a function that returns the string 'CONFIDENTIAL' fails because that value cannot be cast to a DOUBLE column type. Databricks recommends SQL UDFs because the optimizer can inline them. Python UDFs registered in Unity Catalog are supported but can't be optimized the same way.
A mask can use more than one column condition. One documented policy masks a pii : email column unless a column tagged consent_to_contact is true. It passes the consent column to the UDF through USING COLUMNS (c). Because every MATCH COLUMNS condition must match, tables that lack either tagged column are not covered by the policy.
Checkpoint 4 of 5· Check yourself
A column mask policy uses two MATCH COLUMNS conditions: has_tag_value('pii', 'email') and has_tag('consent_to_contact'). A table in scope has a pii : email column but no consent_to_contact column. What do users in TO see?
All MATCH COLUMNS conditions must match. If one doesn't, the policy simply doesn't apply, so the data is returned unmasked.
“If a table does not have columns matching both conditions, the policy does not apply and the data is returned unmasked.”Source: docs.databricks.com
4.How policies are evaluated at query time, and why they fail closed
Evaluation happens in two stages. Unity Catalog evaluates the policies, and the Databricks Runtime enforces them. For each table in a query, Unity Catalog uses the table's tags and the user's identity and group memberships, so two users running the same query can see different results. A change to group membership or to a tag changes the effective policies at the next query.
Checkpoint 5 of 5· Put it in order
Put Unity Catalog's per-table evaluation steps in order
- 1.Identify all policies whose scope covers the queried table
- 2.Evaluate whether the table matches each policy's conditions
- 3.Check that the querying user is in the TO list and not in the EXCEPT list
- 4.Send the effective row filter or column mask to the Databricks Runtime with the table metadata
The documented order is: scope, then principals, then conditions, then the effective filter or mask is handed to the runtime. The runtime enforces it as a secure view over the table scan.
“Identifies all policies whose scope covers the queried table.”Source: docs.databricks.com
When a user queries a table through a view, two identities are involved. Access to the underlying tables is checked against the view owner, but the policies are evaluated against the querying user. To see which policies apply to a given table and user, run SHOW EFFECTIVE POLICIES.
ABAC is fail-closed: Databricks allows access only when it can enforce every applicable policy. Some operations are incompatible with filters and masks, including access from runtimes below 16.4, deep and shallow cloning, OpenSharing (the share owner must be exempt) and AI Search index syncing. These operations fail instead of bypassing the policy. To run one, the principal must be in the EXCEPT clause of every ABAC policy on the table.
Policies also depend on their tags and UDFs. If a governed tag that a policy references is deleted, every query against the object the policy is attached to, and against its children, fails with UC_ABAC_UNKNOWN_TAG_POLICY. The UI and API don't stop you deleting such a tag. If a referenced UDF is deleted, queries fail with UC_DEPENDENCY_DOES_NOT_EXIST. You fix either error by restoring the dependency or by updating the policy.
Databricks won't drop a column that has a governed tag applied. A user with ASSIGN on the tag and APPLY TAG on the object must remove the tag first, then the pipeline can run. Afterwards the tag must be reapplied to the relevant columns. If it isn't, queries fail because the policy is still in scope but the tag it expects is missing.
Exam traps
Each one states something that sounds right. Open it to see what is actually true.
1.Deleting a governed tag that a policy references only affects the tables that actually carried that tag.Why is that wrong?
Every query against the object the policy is attached to, and against its child objects, fails with UC_ABAC_UNKNOWN_TAG_POLICY, including tables that never had the tag.
Covered in How policies are evaluated at query time, and why they fail closed
2.Operations such as deep cloning or OpenSharing simply skip ABAC policies on a protected table.Why is that wrong?
These operations fail. They can run only when the executing principal is exempt from every policy on the table.
Covered in How policies are evaluated at query time, and why they fail closed
Sources
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.
“Databricks Runtime 16.4 or above, or serverless compute.”
↩︎ The CREATE POLICY statement and its prerequisites“A policy can include up to 3 MATCH COLUMNS expressions, and all must match for the policy to apply.”
↩︎ The CREATE POLICY statement and its prerequisites“optionally combined with AND, OR, and NOT”
↩︎ The CREATE POLICY statement and its prerequisites“The UDF evaluates each row and returns a boolean. Rows where the UDF returns FALSE are excluded from query results.”
↩︎ Row filter policies: a boolean decides each row“If omitted, defaults to TRUE (applies to all tables in scope).”
↩︎ Checkpoint“Rows where the UDF returns FALSE are excluded from query results.”
↩︎ Prediction - 2.
“In column conditions (MATCH COLUMNS), checks tags set directly on the column only”
↩︎ The CREATE POLICY statement and its prerequisites“It also accepts tag introspection functions, which extract a tag's value at query time and pass it to the UDF.”
↩︎ The CREATE POLICY statement and its prerequisites“The masked column value is bound automatically as the first argument from the ON COLUMN clause”
↩︎ Column mask policies: return the original or a redacted value“The return type must match or be castable to the column's data type.”
↩︎ Column mask policies: return the original or a redacted value“SQL UDFs are recommended for better performance.”
↩︎ Column mask policies: return the original or a redacted value“If a table does not have columns matching both conditions, the policy does not apply and the data is returned unmasked.”
↩︎ Checkpoint - 3.
“Create a row filter policy that blocks access to tables tagged classification : unverified.”
↩︎ Row filter policies: a boolean decides each row“Fails: 'CONFIDENTIAL' cannot be cast to a DOUBLE column type”
↩︎ Column mask policies: return the original or a redacted value“Succeeds: Masks a DOUBLE column, returns DOUBLE in every branch”
↩︎ Column mask policies: return the original or a redacted value - 4.https://docs.databricks.com/aws/en/data-governance/unity-catalog/abac/policy-evaluationOfficial docs
“Different users may see different results from the same query because policy evaluation depends on the user's identity, group memberships, and the tags”
↩︎ How policies are evaluated at query time, and why they fail closed“Access to the underlying tables is checked against the view owner's identity, while policies are evaluated against the querying user's identity”
↩︎ How policies are evaluated at query time, and why they fail closed“ABAC follows a fail-closed model, where Databricks defaults to denying access if it cannot verify security.”
↩︎ How policies are evaluated at query time, and why they fail closed“This occurs even if the tag was not applied to the queried tables.”
↩︎ Exam trap 1“the principal must be listed in the EXCEPT clause of every ABAC policy that applies to the table”
↩︎ Exam trap 2“Identifies all policies whose scope covers the queried table.”
↩︎ Checkpoint“the query fails closed (access is denied) to prevent unprotected data exposure”
↩︎ Prediction - 5.
“SHOW EFFECTIVE POLICIES to see which policies apply to a given table and user.”
↩︎ How policies are evaluated at query time, and why they fail closed