Skip to content

You are viewing documentation for Immuta version 2022.5.

For the latest version, view our documentation for Immuta SaaS or the latest self-hosted version.

Write a Local Policy

Audience: Data Owners and Governors

Content Summary: This page outlines step-by-step instructions for building Local Policies in Immuta.

For instructions on writing Global Subscription Policies and Global Data Policies, see Chapter 3.

Additional Tutorials Contents:

  • Certify Global Policies
  • Copy Policies from Other Data Sources
  • Custom Where Clause Functions
  • Policy Diffs
  • Policy Exemptions
  • Restricted Global Policies

Use Case

Now that the project manager has added members to the data source, she can create restrictive Data and Subscription Policies and apply them directly to her data sources.

1 - Write a Subscription Policy

  1. Select a data source and click the Policies tab.
  2. In the Subscription Policy section, click Edit Subscription Policy.
  3. Select an access restriction. Click the tabs below to view specific access restrictions and their instructions.

    Allow Anyone

    1. Opt to check the Require users to take action to subscribe checkbox to turn off automatic subscription. Enabling this feature will require users to manually subscribe to the data source if they meet the policy.

    2. Opt to complete the Enter Rationale for Policy field.

    Allow Anyone Who Asks (and Is Approved)

    1. Click anyone or an individual selected by user from the first dropdown menu in the Subscription Policy Builder.

      Subscription Policy Builder

      Note: If you choose an individual selected by user, when users request access to a data source they will be prompted to identify an approver with the permission specified in the policy and how they plan to use the data.

      Request Access

    2. Select the Owner (of the data source), User_Admin, Governance, or Audit permission from the subsequent dropdown menu.

      Note: You can add more than one approving party by selecting + Add and repeating the previous steps.

    Users with Specific Groups/Authorizations

    1. Choose the condition that will drive the policy: when user is a member of a group or possesses attribute.

    2. Use the subsequent dropdown to choose the group or attribute key / value pair for your condition.

    3. If you would like to make your data source visible in the list of all data sources in the UI to all users, click the Allow Discovery checkbox. Otherwise, this data source will not be discoverable by users who do not meet the criteria established in the policy.

    4. Opt to check the Require users to take action to subscribe checkbox to turn off automatic subscription. Enabling this feature will require users to manually subscribe to the data source if they meet the policy.

      Note: You can add more than one condition by selecting + ADD. The dropdown menu in the far right of the Subscription Policy Builder contains conjunctions for your policy. If you select or, only one of your conditions must apply to a user for them to see the data. If you select and, all of the conditions must apply.

    Individual Users You Select

    If you select this option, Data Owners will have to manually add users to the data source on the Members tab for users to get access to the data source.

  4. Click Save to finish your policy.

2 - Write a Data Policy

Masking

If a column contains sensitive data (i.e. credit card or social security numbers), Data Owners can apply a masking policy to conceal the data in that column to users unless the user possesses an attribute, belongs to a group, or is acting under a purpose defined by the Data Owner.

Differences Between Spark and the Query Engine

Spark policies are applied at the lowest possible level in the Spark plan for security reasons, which may lead to different results when applying policies. For instance, in the Query Engine a user may be able to compute a column and then generate a masking policy on that computed column. In Spark, however, this is not possible, so the query may be blocked outright.

Masking Policy Builder Example

  1. Select a data source and click the Policies tab.
  2. In the Data Policies section, click New Policy.
  3. Select mask in the first dropdown.
  4. Select a custom masking type in the next dropdown menu: using hashing, with reversibility , by making null, using a constant, using a regex , by rounding, with format preserving masking, with K-Anonymization , using randomized response , or using the custom function.

    If you select using a constant as your masking type, enter a constant in the field that appears next to the masking type dropdown:

    Masking Policy Enter Constant

    If you choose using a regex as your masking type, enter a regular expression and replacement value in the fields that appear next to the masking type dropdown. Another dropdown will appear with possible modifiers for your regular expression. Make your regex case insensitive and/or global:

    Masking Policy Regex

    If you choose by rounding to mask a column with a numerical value, select the number to the nearest in the resulting dropdown. A field will appear for you to enter a bucket size or use the suggested bucket size, which is generated by the data fingerprint.

    The example below would round the column value to the nearest 50:

    Masking Policy Round Number

    If you choose by rounding to mask time-based values, the time to the nearest will autogenerate to the suggested time bucket, which is determined by the data fingerprint, in the resulting dropdown. If you click this dropdown menu, other time bucket options will appear:

    Masking Policy Round Time

    If you select with K-Anonymization (which will be available after the fingerprint service has been run on the data source) to mask columns, you can choose using fingerprint or requiring a group size of at least to auto-populate or manually select the group size, respectively.

    K-Anon Policy

    Note: Selecting many columns to mask with K-Anonymization increases the processing that must occur to calculate the policy, so saving the policy may take time.

    If you select using the custom function, enter the custom function native to the underlying database.

    Note: The function must be valid for the data type of the column. If it is not, Immuta will error and send a message that the function is not valid.

  5. Select your column in the next dropdown.

  6. Choose the condition that will drive the policy: for or when, which allows conditional masking driven by data in the row.

    If you choose for,

    1. Use the next dropdown to continue the condition: everyone, everyone except, or everyone who.
    2. Use the subsequent dropdown to choose the group, purpose, or attribute key / value pair for your condition.

    If you choose when,

    1. Enter your custom SQL clause in the next field. When you place your cursor in this field, a tool-tip appears that details valid input and the column names of your data source.
    2. Choose the condition that will drive the policy: for everyone, for everyone except, or for everyone who.
    3. Use the next field to choose the group, purpose, or attribute key / value pair for your condition.

    Notes:

    • If you choose for everyone who as a condition, complete the Otherwise clause before continuing to the next step.
    • You can add more than one condition by selecting + ADD. The dropdown menu in the far right of the Policy Builder contains conjunctions for your policy. If you select or, only one of your conditions must apply to a user for them to see the data. If you select and, all of the conditions must apply.
    • When building conditional masking policies or row-level policies with custom SQL statements, avoid using a column that is masked using randomized response in the SQL statement, as this can lead to different behavior depending on whether you’re using the Query Engine, Spark, or Snowflake and may produce results that are unexpected.
  7. Click Create to finish your policy.

    Masking Policy 1

  8. Click Save All to apply the policy to your data source.

    Masking Policy 2

SQL Support Matrix

The SQL Support Matrix button in the Data Policies section allows users to view all masking policy types and details what is supported for each integration.

SQL Support Matrix

Row Redaction

For query-backed data sources, Data Owners can restrict which rows in the data source tables are visible to which users. This redaction is done by matching values in a specific column against a user's groups, attributes, or purposes.

For similar policy mechanics in object-backed data sources, see Object-level Security.

Note: A data source cannot have more than one row redaction policy applied.

Row Redaction Policy Builder Example

To create a row redaction policy,

  1. Select a data source and click the Policies tab.
  2. In the Data Policies menu, click New Policy.
  3. Select the Only show rows action in the first dropdown.
  4. Choose where user in the next dropdown. Note that you can also redact rows based on column values or by using a custom WHERE clause. (See the Filtering Data with a Custom WHERE Clause tab.)
  5. Choose the condition that will drive the policy in the next dropdown: is a member of a group or possesses an attribute.
  6. Use the next field to choose the attribute, group, or purpose that you will match values against.
  7. Use the next dropdown menu to choose the column that will drive this policy.

    Note: You can add more than one condition by selecting + ADD. The dropdown menu in the far right of the Policy Builder contains conjunctions for your policy. If you select or, only one of your conditions must apply to a user for them to see the data. If you select and, all of the conditions must apply.

  8. Choose the condition that will drive the policy: for everyone, for everyone except, or for everyone who.

  9. Use the subsequent dropdown to choose the group, purpose, or attribute key / value pair for your condition.

    Note: If you choose for everyone who as a condition, complete the Otherwise clause by before continuing to the next step.

  10. Click Create to finish your policy.

    Redaction Policy 1

  11. Click Save All to apply the policy to your data source.

    Redaction Policy 2

Minimization

Data sources may contain minimization policies that hide a specified percentage of query results from a user, based on a column with high cardinality (e.g. an employee ID number or other unique identifier), is auto-selected based on the statistics of the data fingerprint.

Minimization Policy Builder Example

To create a minimization policy,

  1. Select a data source and click the Policies tab.
  2. In the Data Policies menu, click New Policy.
  3. Select the Minimize Data Source action in the first dropdown.
  4. In the next field, type the percentage of the data that you want to limit the data source to.
  5. Choose the condition that will drive the policy: for everyone, for everyone except, or for everyone who.
  6. Use the next field to choose the attribute, group, or purpose that you will match values against.

    Notes:

    • If you choose for everyone who as a condition, complete the Otherwise clause before continuing to the next step.
    • You can add more than one condition by selecting + ADD. The dropdown menu in the far right of the Policy Builder contains conjunctions for your policy. If you select or, only one of your conditions must apply to a user for them to see the data. If you select and, all of the conditions must apply.
  7. Click Create to finish your policy.

    Minimization Policy 1

  8. Click Save All to apply the policy to your data source.

Object-level Security

Data Owners can use the Policy Builder to restrict access to objects (blobs) in object-backed data sources. This is done by matching values in a specific blob metadata attribute against a user's groups, attributes, or purposes.

For similar policy mechanics in query-backed data sources, see the Row Redaction tab or the Filtering Data with a Custom WHERE Clause tab.

Only Show Objects Policy Builder Example

To create an only show objects policy,

  1. Select a data source and click the Policies tab.
  2. In the Data Policies menu, click New Policy.
  3. Select the Only show objects action in the first dropdown.
  4. Choose the condition that will drive the policy in the next dropdown.
  5. Use the next field to choose the attribute, group, or purpose that you will match values against.
  6. Use the next dropdown menu to choose the blob metadata attribute that will drive this policy.

    Note: You can add more than one condition by selecting + ADD. The dropdown menu in the far right of the Policy Builder contains conjunctions for your policy. If you select or, only one of your conditions must apply to a user for them to see the data. If you select and, all of the conditions must apply.

  7. Choose the condition that will drive the policy: for everyone, for everyone except, or for everyone who.

  8. Use the next field to choose the attribute, group, or purpose that you will match values against.

    Note: If you choose for everyone who as a condition, you will need to complete the Otherwise clause before continuing to the next step.

  9. Click Create to finish your policy.

    Object Policy 1

  10. Click Save All to apply the policy to your data source.

    Object Policy 2

Time-Based Restrictions

These policies restrict access to rows/objects/files that fall within the time restrictions set in the policy. If a data source has time-based restriction policies, queries run against the data source by a user will only return rows/blobs with a date in its event-time column/attribute from within a certain range.

This type of policy can be used for both object-backed and query-backed data sources.

Only Show Data by Time Policy Builder Example

To create a time-based policy,

  1. Select a data source and click the Policies tab.
  2. In the Data Policies section, click Add Policy.
  3. Select the Only show data by time action in the first dropdown menu.
  4. In the subsequent dropdown menu, select more recent than or older than, and then complete the enter number of field.

    Only Show Data by Time

  5. Select MINUTES, HOURS, DAYS, or YEARS from the next dropdown menu.

  6. Then, choose the condition that will drive the policy: for everyone, for everyone except, or for everyone who.
  7. Use the subsequent dropdown to choose is a member of group, is acting under purpose, or possesses attribute key / value pair for your condition.

    Note: If you choose for everyone who as a condition, complete the Otherwise clause by before continuing to the next step.

  8. Opt to complete the Enter Rationale for Policy (Optional) field.

  9. Click Create, and then click Save All.

Purpose-Based Restrictions

Data Owners in Immuta can restrict usage of any data source to one or more purposes. If a user wishes to run SQL queries against a purpose-restricted data source, they must use the SQL credentials provided by a project containing that purpose.

Purpose-Based Restrictions Policy Builder Example

To create a purpose-based restrictions policy,

  1. Select a data source and click the Policies tab.
  2. In the Data Policies menu, click Add Policy.
  3. Select Limit usage to purpose(s) in the first dropdown.
  4. In the next field, select ANY PURPOSE or the specific purpose that you would like to restrict usage of this data source to.

    Note: You can add more than one condition by selecting + ADD. The dropdown menu in the far right of the Policy Builder contains conjunctions for your policy. If you select or, only one of your conditions must apply to a user for them to see the data. If you select and, all of the conditions must apply.

  5. In the next dropdown, select for everyone or for everyone except. If you select for everyone except, you must select conditions that will drive the policy.

  6. Click Create to finish your policy.

    Purpose Policy 1

  7. Click Save All to apply the policy to your data source.

    Purpose Policy 2

If a description has been added to the purpose, the description will be visible when you hover over the purpose name in the policy or on the project page.

Purpose Description

Filtering Data with a Custom WHERE Clause

Data Owners can apply the most fine-grained control of their data by creating a custom WHERE clause policy. Using the Policy Builder, you can type your desired SQL clause directly into the policy statement. Unlike row redaction, the policy conditions that you select need to match exactly with cells in your data source. As demonstrated in the example below, Data Owners can pair a custom WHERE clause with any condition(s) that they desire in the policy statement.

Custom WHERE Clause Policy Builder Example

  • If you choose for everyone who as a condition, you will need to complete the Otherwise clause before continuing to the next step.
  • You can add more than one condition by selecting + ADD. The dropdown menu in the far right of the Policy Builder contains conjunctions for your policy. If you select or, only one of your conditions must apply to a user for them to see the data. If you select and, all of the conditions must apply.
  • When building conditional masking policies or row-level policies with custom SQL statements, avoid using a column that is masked using randomized response in the SQL statement, as this can lead to different behavior depending on whether you’re using the Query Engine, Spark, or Snowflake and may produce results that are unexpected.

To create a custom where clause policy,

  1. Select a data source and click the Policies tab.
  2. In the Data Policies menu, click Add Policy.
  3. Select Only show rows in the first dropdown.
  4. Select where in the next dropdown.
  5. The next field allows you to enter your custom SQL clause. When you place your cursor in this field, a tool-tip should appear that details valid input and the column names of your data source. For example, loan_amnt >= 10000.

    WHERE Clause Policy 1

  6. Choose the condition that will drive the policy: for everyone, for everyone except, or for everyone who.

  7. Use the next field to choose the attribute, group, or purpose that you will match values against.

    Notes:

    • If you choose for everyone who as a condition, you will need to complete the Otherwise clause before continuing to the next step.
    • You can add more than one condition by selecting + ADD. The dropdown menu in the far right of the Policy Builder contains conjunctions for your policy. If you select or, only one of your conditions must apply to a user for them to see the data. If you select and, all of the conditions must apply.
    • When building conditional masking policies or row-level policies with custom SQL statements, avoid using a column that is masked using randomized response in the SQL statement, as this can lead to different behavior depending on whether you’re using the Query Engine, Spark, or Snowflake and may produce results that are unexpected.
  8. Click Create to finish your policy.

    WHERE Clause Policy 2

  9. Click Save All to apply the policy to your data source.

Results

Now we have created a local Data policy that masks using the constant "Redacted" in the columns ssn for everyone except when the user possesses the attribute Environment.prod or Environment.test.

Users with the attribute Environment.dev will see redacted data and users with the attributes Environment.test or Environment.prod will see all the data:

Dev User

Dev Results

Test User

Test Results

Prod User

Prod Results

Additional Tutorials

Click on the tabs below to view other Local policy tutorials.

Create Subscription Policies with Advanced Rules DSL Builder

You can use the Advanced Rules DSL Builder to create more complex policies for Users with Specific Groups/Attributes than the Subscription Policy Builder allows. To begin,

  1. Select Users with Specific Groups/Attributes, and then click the Advanced Rules DSL bubble.
  2. Complete the Enter Rules field with the available functions and variables: @iam, @isInGroups, and @hasAttribute. When you place your cursor in this field, a tooltip appears with the functions and values you can use to build your policy.

    Advanced Rules DSL

  3. If you would like to make your data source visible in the list of all data sources in the UI to all users, click the Allow Discovery checkbox. Otherwise, this data source will not be discoverable by users who do not meet the criteria established in the policy.

  4. Check the Require users to take action to subscribe checkbox to turn off automatic subscription. Enabling this feature will require users to manually subscribe to the data source if they meet the policy.

  5. Click Save to finish your policy.

Certify Global Policies

After the HIPAA De-identification Policy or CCPA policy is applied to your data source, you will receive a notification indicating that you need to certify the policy.

To certify the policy,

  1. Navigate to the Policies tab of the affected data source, and review the policy in the Data Policies section.
  2. Click the Certify Policy icon in the top right corner of the policy.

    Certify Policy Button

  3. In the Policy Certification modal, click Sign and Certify.

    Policy Certification

Copy Policies from Other Data Sources

If you have created a data source that is similar to an existing data source and you would like to apply the same policies, you can quickly copy those policies from the Policy Builder.

  1. Select a data source and click the Policies tab.
  2. In the upper right corner of the Policies page, click on the dropdown menu that says Edit Subscription Policy and then select Apply Existing Policies from Data Source.
  3. Search for and select the data source that you would like to copy policies from. If your data source is capable of supporting these policies, they will appear in the Policy Builder. If not, you will receive an error. Be sure that the data source that you are copying policies from follows a similar structure to your current data source so that the policies remain relevant.

    Copy Policies

Custom Where Clause Functions

Immuta offers several custom PostgreSQL functions for advanced data source policy logic. You can learn about these functions here.

View Policy Diffs

Once you have a Data Policy in effect, you can view the changes in your policies by clicking the Policy Diff button in the top right of the Data Policies section on a data source's Policies tab.

Policy Diff

The Policy Diff button displays previous policies and the current policy applied to the data source. In the example below, the previous policy masked social security numbers for everyone except users who possessed the attribute environment prod. The current policy has been changed to mask social security numbers for everyone except users who possess the attribute environment prod and environment test.

Policy Diff Example

Add Policy Exemptions

Once this setting is enabled on the App Settings page, you can exempt users from policies on a per-data-source basis. Note: By default, policy exemptions are disabled in Immuta.

  1. Select a data source and click the Policies tab.
  2. In the Data Policies menu, click Add Exemptions. This button will only be visible if policy exemptions have been enabled in your Immuta instance.

    Add Exemptions

  3. Type the names of the users or groups that you wish to exempt from your policies in the corresponding fields.

  4. Click Create to finish your exemption policy.

    Create Policy Exemption

  5. Click Save All to apply the policy to your data source.

Write Restricted Global Policies

Data Owners who are not Governors can write two types of Restricted Global Policies on the Policies page: Subscription and Data Policies. With this feature, Data Owners have higher-level policy controls and can write and enforce policies on multiple data sources simultaneously, eliminating the need to write redundant Local Policies on data sources.

Unlike Global Policies, the application of these policies is restricted to the data sources owned by the users or groups specified in the policy and will change as users' ownerships change.

Restricted Global Subscription Policies

To write a Restricted Global Subscription Policy,

  1. Click the Policies icon in the left sidebar and navigate to the Subscription Policies tab.
  2. Click Add Policy, complete the Enter Name field, and then select the level of access restriction you would like to apply to your data source. Click the tab below for further instructions.

    Allow Anyone

    1. Check the Require users to take action to subscribe checkbox to turn off automatic subscription. Enabling this feature will require users to manually subscribe to the data source if they meet the policy.

    2. Click the dropdown menu beneath Where should this policy be applied, and select On all data sources or On data sources. If you selected On data sources, finish the condition in one of the following ways:

    3. tagged: Select this option and then search for tags in the subsequent dropdown menu.

    4. with columns tagged: Select this option and then search for tags in the subsequent dropdown menu.
    5. with column names spelled like: Select this option, and then enter a regex and choose a modifier in the subsequent fields.
    6. in server: Select this option and then choose a server from the subsequent dropdown menu to apply the policy to data sources that share this connection string.
    7. created between: Select this option and then choose a start date and an end date in the subsequent dropdown menus.

    Allow Anyone Who Asks (and Is Approved)

    1. Click anyone or an individual selected by user from the first dropdown menu in the Subscription Policy Builder.

      Subscription Policy Builder

      Note: If you choose an individual selected by user, when users request access to a data source they will be prompted to identify an approver with the permission specified in the policy and how they plan to use the data.

      Request Access

    2. Select the Owner (of the data source), User_Admin, Governance, or Audit permission from the subsequent dropdown menu.

      Restricted Request Approval

      Note: You can add more than one approving party by selecting + Add.

    3. From the Where should this policy be applied dropdown menu, select When selected by data owners, On all data sources, or On data sources. If you selected On data sources, finish the condition in one of the following ways:

      • tagged: Select this option and then search for tags in the subsequent dropdown menu.

      • with columns tagged: Select this option and then search for tags in the subsequent dropdown menu.

      • with column names spelled like: Select this option, and then enter a regex and choose a modifier in the subsequent fields.

      • in server: Select this option and then choose a server from the subsequent dropdown menu to apply the policy to data sources that share this connection string.

      • created between: Select this option and then choose a start date and an end date in the subsequent dropdown menus.

      Restricted Global Policy Application

    Allow Users with Specific Group/Attributes

    1. Choose the condition that will drive the policy: when user is a member of a group or possesses attribute.

    2. Use the subsequent dropdown to choose the group or attribute key / value pair for your condition.

      Specific Groups or Attributes

      Note: You can add more than one condition by selecting + ADD. The dropdown menu in the far right of the Subscription Policy Builder contains conjunctions for your policy. If you select or, only one of your conditions must apply to a user for them to see the data. If you select and, all of the conditions must apply.

    3. If you would like to make your data source visible in the list of all data sources in the UI to all users, click the Allow Discovery checkbox. Otherwise, this data source will not be discoverable by users who do not meet the criteria established in the policy.

    4. Check the Require users to take action to subscribe checkbox to turn off automatic subscription. Enabling this feature will require users to manually subscribe to the data source if they meet the policy.

    5. Select When selected by data owners or On data sources from the Where should this policy be applied? dropdown menu. If you selected On data sources, finish the condition in one of the following ways:

      • tagged: Select this option and then search for tags in the subsequent dropdown menu.

      • with columns tagged: Select this option and then search for tags in the subsequent dropdown menu.

      • with column names spelled like: Select this option, and then enter a regex and choose a modifier in the subsequent fields.

      • in server: Select this option and then choose a server from the subsequent dropdown menu to apply the policy to data sources that share this connection string.

      • created between: Select this option and then choose a start date and an end date in the subsequent dropdown menus.

    Advanced Rules DSL

    1. Click Advanced Rules DSL in the top right corner of the policy builder.

    2. Complete the Enter Rules field with the available functions and variables: @iam, @isInGroups, and @hasAttribute. When you place your cursor in this field, a tooltip appears with the functions and values you can use to build your policy.

      Advanced Rules

    3. Select When selected by data owners or On data sources from the Where should this policy be applied? dropdown menu. If you selected On data sources, finish the condition in one of the following ways:

      • tagged: Select this option and then search for tags in the subsequent dropdown menu.

      • with columns tagged: Select this option and then search for tags in the subsequent dropdown menu.

      • with column names spelled like: Select this option, and then enter a regex and choose a modifier in the subsequent fields.

      • in server: Select this option and then choose a server from the subsequent dropdown menu to apply the policy to data sources that share this connection string.

      • created between: Select this option and then choose a start date and an end date in the subsequent dropdown menus.

    Allow Individually Selected Users

    1. Click the dropdown menu beneath Where should this policy be applied, and select On all data sources or On data sources. If you selected On data sources, finish the condition in one of the following ways:

      • tagged: Select this option and then search for tags in the subsequent dropdown menu.

      • with columns tagged: Select this option and then search for tags in the subsequent dropdown menu.

      • with column names spelled like: Select this option, and then enter a regex and choose a modifier in the subsequent fields.

      • in server: Select this option and then choose a server from the subsequent dropdown menu to apply the policy to data sources that share this connection string.

      • created between: Select this option and then choose a start date and an end date in the subsequent dropdown menus.

    2. Beneath Whose Data Sources should this policy be restricted to?, add users or groups to the policy restriction by typing in the text fields and selecting from the dropdown menus that appear.

      Policy Restrictions

    3. Opt to complete the Enter Rationale for Policy (Optional) field, and then click Create to save the policy.

Restricted Global Data Policies

To write a Restricted Global Data Policy,

  1. Click the Policies icon in the left sidebar and navigate to the Data Policies tab.
  2. Click Add Policy, complete the Enter Name field, and then build the data policy following these instructions above.
  3. Opt to complete the Enter Rationale for Policy (Optional) field and click Add.
  4. Where should this policy be applied, choose When selected by data owners, On all data sources, or On data sources and complete the condition using the subsequent dropdown menus (when applicable).
  5. Beneath Whose Data Sources should this policy be restricted to, add users or groups to the policy restriction by typing in the text fields and selecting from the dropdown menus that appear.

    Policy Restrictions

  6. Click Create to save the policy.