Planning for role-based field filtering in your organization

Preview features are provided by Splunk to you "as is" without any warranties, maintenance and support, or service level commitments. Splunk makes this preview feature available in its sole discretion and may discontinue it at any time. Use of preview features is subject to the Splunk General Terms.

Before you configure role-based field filtering in your organization, carefully plan how you want to implement field filters in the Splunk platform. Start at the highest level and decide which index you want to include, then work your way down to the hosts, sources, or source types. Then, consider the fields you want to restrict, the field values you want to assign to those fields, and so on.

In general, it doesn't matter in which order you plan each area. For example, you can start by choosing existing roles to filter and then add field filtering to the roles for each of the fields you identify in your planning. Alternatively, you can start by identifying the fields you want to protect and then create the roles to filter them.

Consider how role-based filtering affects the following areas as you plan your strategy.

• Indexes
• Hosts, sources, or source types
• Fields and field values
• Roles and imported roles
• Optimizing performance
• Downstream operations
• Planning search heads and indexers
• Limitations on using field filtering in your environment

Indexes

Which index contains the data you want to protect? For best performance and enhanced security, specify the searchable indexes for a role before you configure field filtering on the role. For more information about managing indexes and roles, see Create and manage roles with Splunk Web.

If you are using Splunk Enterprise, you can use either the srchIndexesAllowed setting or srchIndexesDisallowed setting in the authorize.conf file to include or exclude searchable indexes for a role. See Manage an existing role.

Hosts, sources, or source types

Which specific hosts, sources, or source types apply to the fields that you want to protect with role-based field filtering? See Limiting role-based field filters to specific hosts, sources, indexes, and source types.

Fields and field values

Which sensitive data in your events do you need to protect? Are there certain indexed fields that contain personal identifiable information (PII), protected health information (PHI), or other confidential data that you need to keep private?

The sensitive fields you want to filter must be indexed fields. This is important because the Splunk platform processes field filter configurations at search time ahead of all other search-time operations that add fields to events, including field extractions. As a result, fields that are extracted or added at search time do not exist when field filter configurations are processed. See The sequence of search-time operations ​in the Splunk Cloud Platform Knowledge Manager Manual.

As you plan your field filtering strategy, identify the specific fields that need to be removed with a null value, or replaced with a custom string or hash. Your answer depends on the level of protection your data needs.

• If your data needs the highest form of privacy for legal or regulatory reasons, your plan might need to include redacting field values with NULL or a custom string. The null option removes the field and the custom string option replaces the field with a string value.
• If you plan to obfuscate your field values with SHA-256 or SHA-512 (SHA-2 family) hash functions, make sure that replacing field values with a hash offers the security your data requires.
• If you want to preserve the uniqueness of a field for statistical use cases, for example, when using the stats or timechart commands in searches, plan to replace the field value with a hash.

Once you consider which fields you want to filter with role-based field filtering, decide how many field filters you need on each role. If your organization has multiple fields per source type, you might need to configure multiple field filters for each source type. See Setting role-based field filters with the Splunk platform and Limiting role-based field filters to specific hosts, sources, indexes, and source types.

Roles and imported roles

Who does your sensitive data need to be protected from? Which users in your organization are authorized or not authorized to see your sensitive data? Identify which privileged roles need to be able to continue accessing sensitive data, and which roles should be restricted. You can control who sees what in your organization using role-based field filtering, so plan ahead for which field filters will be on which roles. See Create and manage roles with Splunk Web.

Filtering fields for imported roles

Do you need to plan your field filtering around roles that import other roles? You might need to use imported roles if you want to use a role to filter fields from different hosts, sources, or source types. If you need to filter events from different target hosts, sources or source types on a single role, design the field filters for each role to optimize performance for one host, source or source type using the fieldFilterLimit setting. Then, combine the roles, as needed, using imported roles. For examples of using field filtering with imported roles, see Setting role-based field filters with the Splunk platform and Limiting role-based field filters to specific hosts, sources, indexes, and source types.

When you specify field filter configurations for a role that is importing other roles that also have field filter configurations, Splunk platform processes field filter configurations for the imported roles before it processes field filter configurations for roles that are importing other roles. As a result, when field names are the same, the field value of the role that is doing the importing overwrites the field value of the role that is imported. For example, say role A has fieldFilter-patient_name = YYY and role B has fieldFilter-patient_name = XXX. If role B imports role A, the Splunk platform processes the field filter defined for role A first, and then it processes the field filter defined for role B. This means that users with role B always see patient_name = XXX in their results because the field filter configuration for role B is processed last.

Splunk platform runs each role in an import hierarchy only once. If multiple roles in an import hierarchy apply a field filter configuration to a field, Splunk platform runs them in the order of imported roles to roles that are importing other roles in the import hierarchy, from left to right as listed in the importRoles setting in the authorize.conf file.

Optimizing performance

Are there performance impacts of role-based field filtering on your searches that you need to plan around? All role-based field filters affect the performance of searches to some extent. As you plan how you will deploy role-based field filtering in your organization, keep the following performance considerations in mind.

Customize field filters

The performance impact of role-based field filtering depends on how each role-based field filter is defined and how many there are. Search performance degrades as you increase the number and complexity of role-based field filters. A few simple field filters that don't use sed expressions or SHA hash functions have minimal performance impact on searches, similar to that of filtering with the eval command.

Use field limits

For optimal performance, include a field limit for field filters on each role. Limiting a field filter to a specific target host, source, or source type with the field filter limit setting avoids performance impacts on non-related search results. For example, if a limit is not specified, the role-based field filter will be applied to all search results, which will result in inefficient searches. See Limiting role-based field filters to specific hosts, sources, indexes, and source types.

Downstream operations

As the first operation in the search-time operation sequence, role-based field filtering is processed before all other operations. This means that operations that come later in the sequence that depend on certain field values might break when expected field values are filtered with field filtering. You might need to configure field filtering using hash functions that preserve the statistical uniqueness of field values for later operations, or reevaluate search operations that are used together with role-based field filtering. See The sequence of search-time operations in the Splunk Cloud Platform Knowledge Manager Manual.

Planning search heads and indexers

If you plan to run role-based field filters on a search head, the versions of the search head and indexers must be compatible. For example, to configure field filters on a search head running Splunk Enterprise version 9.0.0, the indexers must also run Splunk Enterprise version 9.0.0 or higher. Your plans for rolling out field filters may need to include upgrading your search heads and indexers, so that all of your versions are in sync.

Limitations on using field filtering in your environment

Consider the following issues when setting up your environment to take advantage of role-based field filtering.

• Generating commands
• Adding new fields
• Restricted commands
• Data model acceleration
• Summary indexes

Generating commands

When you set up searches with field filters, keep in mind that role-based field filters do not support searches that use generating commands other than the search command. This means that the first command in the search string must not be another command besides search. For example, in the following search, the field filter is not applied because the generating command is inputcsv, not search:

| inputcsv ... | search ...

However, the field filter is applied in the following search. Technically, the search command comes first in the pipeline, but doesn't actually need to be specified because it is implied.

index=main | ....

See Generating commands in the Splunk Cloud Platform Search Reference.

Adding new fields

Don't use role-based field filters to add new fields at search time. Use calculated fields for this purpose. See About calculated fields in the Splunk Cloud Platform Knowledge Manager Manual.

Restricted commands

Does anyone in your organization need to use the following restricted commands with role-based field filtering?

• mpreview
• mstats
• tstats
• typeahead
• walklex

These commands can return sensitive data that a role with field filters might not be allowed to access. They might pose a potential security risk for your organization if someone with malicious intentions tries to use them to circumvent role-based field filtering. As a result, the Splunk platform restricts these commands when used by people with roles that are configured with field filtering.

If you want certain highly trusted roles that are configured with field filtering to be able to use these restricted commands, you must configure the roles to have one of the following capabilities:

• The run_commands_ignoring_field_filter capability. Users with this capability can run commands that return index information even when a field filter is configured for that user's role.
• The admin_all_objects capability. This capability is very powerful. Users with this capability have access to all objects in the system. Use this capability with caution and only with the most trusted roles in your organization.

To add capabilities to a role, see Define roles on the Splunk platform with capabilities.

Data model acceleration

Role-based field filtering doesn't support data model acceleration because the tstats command is a restricted command and can return sensitive data that a role with field filters might not be allowed to access. To allow certain highly trusted roles that are configured with field filtering to continue to use accelerated data models, you must configure the role to have one of the following capabilities:

• The run_commands_ignoring_field_filter capability. Users with this capability can run commands that return index information even when a field filter is configured for that user's role.
• The admin_all_objects capability. This capability is very powerful. Users with this capability have access to all objects in the system. Use this capability with caution and only with the most trusted roles in your organization.

Roles configured with field filtering and one of these capabilities can view data acceleration as usual, but without field filtering.

See Accelerate data models in the Splunk Cloud Platform Knowledge Manager Manual.

Summary indexes

Summary indexes generated by scheduled reports can expose sensitive index information that you might not want unprivileged roles to be able to access. If you collect sensitive summary index information that you want to filter with role-based field filtering, carefully consider which highly trusted roles can create those summary indexes.

By default, all reports run with the permissions of the report "owner", which is the person who created the report. To ensure that unprivileged users don't have access to sensitive data, set up your summary indexes using the same role that you are configuring with role-based field filtering. For example, if you're configuring role-based field filtering on the support role, you must also use the support role when you set up your summary indexes.

See Determine whether to run reports as the report owner or report user in the Splunk Cloud Platform Reporting Manual and Create a summary index in Splunk Web in the Splunk Cloud Platform Knowledge Manager Manual.

Example

If you don't use the same role to set up your scheduled searches as the role that you configure with role-based field filtering, you might expose PII or PHI data to people who aren't allowed to access it. Consider what might happen if you use the admin role to set up a scheduled search, and then set the following field filter on a role called test to obfuscate the host name:

[role_test]

fieldFilter-host = SHA256

importRoles = user

When the person who holds the test role runs a search on the index, this is what they might see:

Time	Event
2/28/22 10:15:00.000 AM	02/28/2022 10:15:00 -0800, search_name=TestSummary, search_now=1646075700.000, info_min_time=1646072100.000, info_max_time=1646075700.000, info_search_time=1646075700.607, orig_host="test-mbp-571a7", action=upload_mmdb_files, psrsvd_v=1, psrsvd_gc=9, testField="testValue" host = c1fe560f903b48c96812691329a783a9333414bbf7ac26c26bf41cebbaf79b6d

Notice that the host name shows up in two places in the summary index. The value for the first instance of the host field is test-mbp-571a7, and the value for the second instance is a hash. In this case, using one role, the admin role, to set up the scheduled search and then running the search using a different role, the test role, that is configured with field filtering resulted in exposing the data that was supposed to be obfuscated with the hash value.

If the same test role that is configured with field filtering is also used to set up the scheduled search, this is what the user who holds the test role sees.

Time	Event
2/28/22 10:15:00.000 AM	02/28/2022 10:15:00 -0800, search_name=TestSummary, search_now=1646075700.000, info_min_time=1646072100.000, info_max_time=1646075700.000, info_search_time=1646075700.607, orig_host="c1fe560f903b48c96812691329a783a9333414bbf7ac26c26bf41cebbaf79b6d", action=upload_mmdb_files, psrsvd_v=1, psrsvd_gc=9, testField="testValue" host = c1fe560f903b48c96812691329a783a9333414bbf7ac26c26bf41cebbaf79b6d

Now both instances of the host field in the summary index are obfuscated with the hash value. As a result, sensitive data about the true value of the host name is no longer exposed to unprivileged users.

See also

Protecting PII and PHI data with role-based field filtering

Turning on Splunk platform role-based field filtering