Optimize field filter performance using configuration files

Preview features are provided by Splunk to you "as is" without any warranties, maintenance and support, or service level commitments. Splunk makes this preview feature available in its sole discretion and may discontinue it at any time. Use of preview features is subject to the Splunk General Terms.

READ THIS FIRST: Should you deploy field filters in your organization?

Field filters are a powerful tool that can help many organizations protect their sensitive fields from prying eyes, but it might not be a good fit for everyone. If your organization runs Splunk Enterprise Security or if your users rely heavily on commands that field filters restricts by default (mpreview, mstats, tstats, typeahead, and walklex), do not use field filters in production until you have thoroughly planned how you will work around these restricted commands. See READ THIS: Restricted commands do not work in searches on indexes that have field filters.

How to optimize field filter performance using configuration files

By default, a field filter configuration filters all events on the Splunk platform from all available hosts, sources, and source types in the specified target indexes, which can impact performance on unrelated search results. This section describes how you can fine-tune the hosts, sources, and source types that a field filter draws from. Limiting field filters to hosts, sources, and source types is not required to set up field filters, but doing so can significantly improve search performance. See Plan for field filters in your organization.

You can only use configuration files to create and manage field filters with Splunk Enterprise.

Prerequisites

• Have the necessary permissions to edit configuration files. Only users with file system access, such as system administrators, can create and manage field filters using configuration files.
• Know how to edit configuration files. Review the steps in How to edit a configuration file in the Splunk Enterprise Admin Manual.
• Decide which directory to store configuration file changes in. There can be configuration files with the same name in your default, local, and app directories. See Where you can place (or find) your modified configuration files in the Splunk Enterprise Admin Manual.

Never change or copy the configuration files in the default directory. The files in the default directory must remain intact and in their original location. Make changes to the files in the local directory.

Restrict field filters to specific hosts, sources, or source types

For more efficient searches and better performance, restrict field filters to one or more hosts, sources, or source types. For example, your searches will be faster if you configure your field filter to filter events from a specific host called hostname1. Instead of filtering all events, only those events from the host hostname1 are filtered.

Keep the following considerations in mind when you plan to restrict your field filter to specific indexes, hosts, sources, or source types:

• You can specify only one value for each field filter limit type, but you can specify multiple hosts, sources, or source types limits.
• The field filter limit setting doesn't support statements that include wildcards or the following operators: AND, OR.

Steps

To restrict a field filter from specific hosts, sources, or source types, follow these steps.

1. Open or create a local field_filters.conf file at $SPLUNK_HOME/etc/system/local.
2. In the [<fieldFilterName>] stanza for the field filter you want to modify, add a new line using Backus–Naur form (BNF) for the limit setting using the following BNF for <actions> syntax:
   
   

   limit = <limit_type>::host name | source name | source type name
   

   The limit_type is host, source, or source type. Only one value can be specified for the field filter limit type.

   One or more comma-separated values can be specified for the name of the host, source, or source type.

Examples

1. Set the source type for a field

Say you have a field filter that redacts values of the IP_addr field in searches with the string xxxx. The IP_addr field filter applies to all events with the IP_addr field.

To apply the IP_addr field filter only to events from the accounts index that have the IP_addr field and the cisco_syslog source type, add the following lines to your field_filters.conf file:

[filter_IP_addr_indexed] 
index = accounts
limit = sourcetype::cisco_syslog
action = "IP_addr" = xxxx 
description = Replace the IP_addr field with xxx.

See also

Protect PII, PHI, and other sensitive data with field filters

Create field filters using Splunk Web

Create field filters using configuration files