Setting role-based field filters with the Splunk platform

Preview features are provided by Splunk to you "as is" without any warranties, maintenance and support, or service level commitments. Splunk makes this preview feature available in its sole discretion and may discontinue it at any time. Use of preview features is subject to the Splunk General Terms.

Role-based field filtering controls visibility within events by redacting or obfuscating confidential information such as personal identifiable information (PII) and protected health information (PHI) data when specific users run searches. This type of filtering can remove fields from events returned by search results, or replace values of fields when those fields appear in the results. If you don't want certain unauthorized people to know that a field even exists, you can redact the field by removing it with a null value. Alternatively, you can redact the field value by replacing it with a custom string, or you can replace the field value with a SHA-256 or SHA-512 (SHA-2 family) hash value that obfuscates the sensitive data. After you configure field filtering on a role, the settings then affect the results of searches run by people who hold that role by filtering out specific indexed or default fields.

Develop a plan for your field filters before you configure them. See Planning for role-based field filtering in your organization.

The elements of a field filter

A field filter includes the following elements:

• A field filter assignment using fieldFilter-<fieldname> = <option> that you configure on a specific role. See Configuring role-based field filters.
• A field filter limit using fieldFilterLimit = [sourcetype::<sourcetype>|host::<host>|source::<source>] that you set to limit the field filter to a specific host, source, or source type in order to optimize search performance. See Limiting role-based field filters to specific hosts, sources, indexes, and source types.

Configuring role-based field filters

To configure field filters on specific roles, you must be able to update the settings in a role using one of the following methods:

• Update the authorize.conf file by adding fieldFilter-<fieldname> = <option> to the role. The instructions in this topic tell you how to do this.
• Use the Splunk platform REST API authorization/roles/{name} endpoint to update settings for the role. You must hold a role with the edit_field_filter capability, such as the predefined admin role, to use the endpoint to configure role-based field filtering. See authorization/roles/{name} in Splunk Cloud Platform REST API Reference Manual.

The following sections describe how to configure a field for filtering by assigning a value to a field.

Splunk Cloud Platform

To add a field filter to a role, request help from Splunk Support. If you have a support contract, file a new case using the Splunk Support Portal at Support and Services. Otherwise, contact Splunk Customer Support.

Splunk Enterprise

To add fieldFilter-<fieldname> = <option> to a role in the authorize.conf file, follow these steps.

Prerequisites

• Have the permissions to edit configuration files. Only users with file system access, such as system administrators, can edit configuration files.
• Know how to edit configuration files. Review the steps in How to edit a configuration file in the Splunk Enterprise Admin Manual.
• Decide which directory to store configuration file changes in. There can be configuration files with the same name in your default, local, and app directories. See Where you can place (or find) your modified configuration files in the Splunk Enterprise Admin Manual.

Never change or copy the configuration files in the default directory. The files in the default directory must remain intact and in their original location. Make changes to the files in the local directory.

Steps

1. Open or create a local authorize.conf file at $SPLUNK_HOME/etc/system/local.
2. In the stanza for the role that you want to restrict, add the line fieldFilter-<fieldname> = <option>.
   See the following sections for descriptions of the <fieldname> and <option> arguments.

The fieldname argument

The <fieldname> argument in the fieldFilter-<fieldname> = <option> setting in the authorize.conf file is the name of the field that you want to restrict with the field filter. For example, if you want to set a field filter on a sensitive field called account_number, use account_number as the <fieldname>.

You cannot use wildcards to specify multiple fields for <fieldname>.

The option argument

The <option> argument in the fieldFilter-<fieldname> = <option> setting in the authorize.conf file is the value you want displayed when the field filter is applied to a field. The values available for <option> depend on whether the value of <fieldname> is _raw or any other field name.

The values for <option> are described in more detail in the following sections.

The value of option when the fieldname is _raw

When the value of the <fieldname> argument is _raw, <option> is a sed expression. The sed expression acts on searches to which this filter is applied. The sed expression replaces strings in search results that are matched by a regular expression (s) or transliterates characters found in search results with corresponding characters provided by the sed expression (y).

The syntax for using the sed (s) command to replace strings in search results that are matched by a regular expression is s/<regex>/<replacement>/<flags>.

The arguments are described as follows:

Value	Description
<regex>	A PCRE regular expression, which can include capturing groups.
<replacement>	A string that replaces the regular expression match. Use \<n> for back references, where <n> is a single digit.
<flags>	Can be either g, to globally replace all matches, or a number that specifies the position of the matching text to replace in the regular expression. Other sed flags for the (s) command are not supported.

The syntax for using the sed (y) command to transliterate characters that Splunk software finds in search results with corresponding characters that you provide is y/<source_characters>/<destination_characters>/.

The arguments are described as follows:

Value	Description
(y)	The (y) command syntax transliterates the <source_characters> in search results with corresponding <destination_characters> that you provide in the expression. For example, /: y/abc/def/ replaces a with d, b with e, and c with f. This expression changes the string aaabbc to dddeef.
<source_characters>	The source characters. The lists of <source_characters> and <destination_characters> must contain the same number of characters.
<destination_characters>	The destination characters. The lists of <source_characters> and <destination_characters> must contain the same number of characters.

The value of option when the fieldname is not _raw

When the value of the <fieldname> argument is any field name other than _raw, the <option> argument can be one of the following values:

Value	Description
NULL	Removes the value of the field from results of searches to which this filter is applied. Use this option if your sensitive data needs to be redacted with the highest form of privacy for legal or regulatory reasons.
<string>	Replaces the value of the field with the specified <string> wherever the field appears in results of searches to which this filter is applied. Use this option if your sensitive data needs to be redacted with a high level of privacy while still indicating the existence of the field.
SHA256	Computes and returns the secure hash of the value of the field based on the FIPS-compliant SHA-256 (SHA-2 family) hash function. This hash is then used to replace the value of the field wherever it appears in results of searches to which this filter is applied. See Cryptographic functions in the Splunk Cloud Platform Search Reference.
SHA512	Computes and returns the secure hash of the value of the field based on the FIPS-compliant SHA-512 (SHA-2 family) hash function. This hash is then used to replace the value of the field wherever it appears in results of searches to which this filter is applied. See Cryptographic functions in the Splunk Cloud Platform Search Reference.

Examples

The following examples show how you can use the fieldFilter-<fieldname> = <option> setting to perform operations on fields in searches run by users with a specific role.

1. Redacting a field

At your organization, the indexed field patient_name is sensitive for security reasons. You have a role named staff, and you want people who belong to the staff role to be unable to access the patient_name field in their search results. Meanwhile, it's okay if users with other roles can see the patient_name field as usual.

If you want to redact the field from the results of searches run by people holding the staff role, apply the following configuration to the staff role. This configuration sets the patient_name field to NULL and removes it from the results of searches by people with the staff role, so they won't even know the field exists:

[role_staff]
fieldFilter-patient_name = NULL

Removing a field with a null value is the most secure method of protection because someone with malicious intentions can't deduce anything about the data if there is no trace of it in the search results.

2. Redacting a field value

If you want to hide your field values, but not remove them from results entirely, you can set your role-based field filter to replace the field value with a custom string. For example, if you want users with the staff role to see search results such as patient_name = XXXX, set a field filter to a custom string on the staff role in your authorize.conf file:

[role_staff]
fieldFilter-patient_name = XXXX

Now when someone who holds the staff role searches on events that include the patient_name field, the string XXX will appear in the results in place of the actual confidential information about the patient.

3. Obfuscating a field value

You can also set your role-based field filter to replace the field value with a hash value. For example, if you want users with the staff role to see a hash value instead of a patient's account number in search results, you would set a field filter like this in the staff role:

[role_staff]
fieldFilter-patient_account = SHA256

Now when someone who holds the staff role searches on events that include the patient_account field, a hash value appears in the results in place of the actual confidential information about the patient.

4. Setting multiple field filters on a single role

You can set multiple field filters on a single role. For example, say you add the following field filters to a role called admin2, which masks the host= and user= fields:

[role_admin2]
fieldFilter-host = unknown host
fieldFilter-_raw = s/user=[^ ]+/REMOVED-USER/

Then, when the person who holds the role called admin2 runs the following search, the host name and username are redacted:

index="main"

Their search results might look something like this:

Time	Event
2022-01-31 15:01:58.679	[timestamp=01-31-2022 15:01:58.679, REMOVED-USER action=search, info=granted REST: /search/ast] host = unknown host
2022-01-31 15:01:58.678	[timestamp=01-31-2022 15:01:58.678, REMOVED-USER action=search, info=granted REST: /search/timeparser] host = unknown host
2022-01-31 15:01:58.677	[timestamp=01-31-2022 15:01:58.677, REMOVED-USER action=edit_win_eventlogs, info=denied ] host = unknown host
2022-01-31 15:01:58.676	[timestamp=01-31-2022 15:01:58.676, REMOVED-USER action=edit_win_regmon, info=denied ] host = unknown host

5. Using field filtering with imported roles

When you import roles that are configured with role-based field filtering and field names are the same, the field value of the role that is doing the importing overwrites the field value of the role that is being imported. For example, say you configure field filtering on two roles, with one role importing the other like this:

[role_userA]
fieldFilter-host=YYY

[role_userB]
fieldFilter-_raw =s/user=[^ ]+/REMOVED-USER/
fieldFilter-host=unknown host
importRoles=userA

Since userB role imports userA role, the Splunk platform processes the field filter defined for the userA role first, and then it processes the field filter defined for the userB role. As a result, when a person who holds the role called userB runs a search, host = unknown host appears in their results instead of fieldFilter-host = YYY, which looks something like this:

Time	Event
2022-01-31 15:01:58.679	[timestamp=01-31-2022 15:01:58.679, REMOVED-USER action=search, info=granted REST: /search/ast] host = unknown host
2022-01-31 15:01:58.678	[timestamp=01-31-2022 15:01:58.678, REMOVED-USER action=search, info=granted REST: /search/timeparser] host = unknown host
2022-01-31 15:01:58.677	[timestamp=01-31-2022 15:01:58.677, REMOVED-USER action=edit_win_eventlogs, info=denied ] host = unknown host
2022-01-31 15:01:58.676	[timestamp=01-31-2022 15:01:58.676, REMOVED-USER action=edit_win_regmon, info=denied ] host = unknown host

6. Hashing a field value and getting the count

An important consideration to performing hash functions on field values using SHA encryption is that field filtering preserves the uniqueness of the field. That means that even if you encrypt identifying information, such as user ID numbers, you can still get a count of the number of user IDs without exposing the value. For example, say you add the following SHA-256 field filter for the host field to a role called admin2:

[role_admin2]
fieldFilter-host=SHA256

Then, you run the following search:

index=main | stats count by host

Your search results might look something like this:

host	count
C1fe560f903b48c9681269132a783a9333414bbf7ac26bf41cebbaf79b6d	8223

7. Redacting PII and PHI data in search results

You can use role-based field filtering to redact sensitive fields such as Personal identifiable information (PII) and Protected Health Information (PHI) from searches of all of your event data. For example, perhaps your search results include the name and phone number for a patient named Alex Martin, which looks something like this:

Time

Event

2/17/22 3:57:04.597 PM

[timestamp=02-17-2022 15:57:04.597, user=test_api, action=search, info=granted , search_id='1645142224.7', search='search patient_name=AlexMartin phone=+1-555-555-0199', autojoin='1', buckets=300, ttl=600, max_count=500000, maxtime=8640000, enable_lookups='1', extra_fields=, apiStartTime='Thu Feb 17 15:42:04 2022', apiEndTime='Thu Feb 17 15:57:04 2022', apiIndexStartTime='ZERO_TIME', apiIndexEndTime='ZERO_TIME', savedsearch_name="", is_proxied=false, app="search", provenance="UI:Search", mode="historical"]

if you want to replace patient_name=AlexMartin and phone=+1-555-555-0199 with custom strings, you can add the following field filter to a role called staff that doesn't have authorization to see that sensitive information:

[role_staff]
fieldFilter-_raw = s/patient_name=[^ ']+/REMOVED-NAME/ s/phone=[^ ']+/REMOVED-PHONE/

The person who holds the staff role runs the following search:

index=main

The person now sees only search results for the patient with REMOVED_NAME REMOVED_PHONE, instead of patient_name=AlexMartin phone=+1-555-555-0199, which looks something like this:

Time

Event

2/17/22 3:57:04.597 PM

[timestamp=02-17-2022 15:57:04.597, user=test_api, action=search, info=granted , search_id='1645142224.7', search='search REMOVED-NAME REMOVED-PHONE', autojoin='1', buckets=300, ttl=600, max_count=500000, maxtime=8640000, enable_lookups='1', extra_fields=, apiStartTime='Thu Feb 17 15:42:04 2022', apiEndTime='Thu Feb 17 15:57:04 2022', apiIndexStartTime='ZERO_TIME', apiIndexEndTime='ZERO_TIME', savedsearch_name="", is_proxied=false, app="search", provenance="UI:Search", mode="historical"]

The confidential data is no longer visible to all people who hold the role configured with the field filter that replaces the patient name and phone number.

8. Customizing field filters with imported roles

You can use imported roles to fine-tune your field filtering to allow each of your users to access some fields, but not others. For example, say you have three roles that are filtering field1, field2, and field3, like this:

[role_testers1]
fieldFilter-field1 = NULL

[role_testers2]
fieldFilter-field2 = NULL

[role_testers3]
fieldFilter-field3 = NULL

Then, you decide that you want your users to have different access to field1, field2, and field3 when they run their searches. For example, you want Ali to be able to see only field1, but not field2 and field3. You want Raj to be able to see only field2, but not field1 and field3. And you want Maria to be able to see only field3, but not field1 and field2.

To do this, you need to strategically set up your roles and imported roles. For example, you would import the testers2 and testers3 roles for Ali, then import the testers1 and testers3 roles for Raj, and import the testers1 and testers2 roles for Maria, like this:

[role_testers1]
fieldFilter-field1 = NULL

[role_testers2]
fieldFilter-field2 = NULL

[role_testers3]
fieldFilter-field3 = NULL

[role_Ali]
importRoles = testers2,testers3

[role_Raj]
importRoles = testers1,testers3

[role_Maria]
importRoles = testers1,testers2

Now when Ali runs his searches, he seesfield1 and its data, but not field2 and field3 because they are filtered.

When Raj runs his searches, he sees field2 and its data, but not field1 and field3 because they are filtered.

And, when Maria runs her searches, she sees field3 and its data, but not field1 and field2 because they are filtered.

See also

Protecting PII and PHI data with role-based field filtering

Turning on Splunk platform role-based field filtering