Optimize field filter performance using configuration files
Preview features are provided by Splunk to you "as is" without any warranties, maintenance and support, or service level commitments. Splunk makes this preview feature available in its sole discretion and may discontinue it at any time. Use of preview features is subject to the Splunk General Terms.
READ THIS FIRST: Should you deploy field filters in your organization?
Field filters are a powerful tool that can help many organizations protect their sensitive fields from prying eyes, but it might not be a good fit for everyone. If your organization runs Splunk Enterprise Security or if your users rely heavily on commands that field filters restricts by default (mpreview
, mstats
, tstats
, typeahead
, and walklex
), do not use field filters in production until you have thoroughly planned how you will work around these restricted commands. See READ THIS: Restricted commands do not work in searches on indexes that have field filters.
How to optimize field filter performance using configuration files
By default, a field filter configuration filters all events on the Splunk platform from all available hosts, sources, and source types in the specified target indexes, which can impact performance on unrelated search results. This section describes how you can fine-tune the hosts, sources, and source types that a field filter draws from. Limiting field filters to hosts, sources, and source types is not required to set up field filters, but doing so can significantly improve search performance. See Plan for field filters in your organization.
You can only use configuration files to create and manage field filters with Splunk Enterprise.
Prerequisites
- Have the necessary permissions to edit configuration files. Only users with file system access, such as system administrators, can create and manage field filters using configuration files.
- Know how to edit configuration files. Review the steps in How to edit a configuration file in the Splunk Enterprise Admin Manual.
- Decide which directory to store configuration file changes in. There can be configuration files with the same name in your default, local, and app directories. See Where you can place (or find) your modified configuration files in the Splunk Enterprise Admin Manual.
Never change or copy the configuration files in the default directory. The files in the default directory must remain intact and in their original location. Make changes to the files in the local directory.
Restrict field filters to specific hosts, sources, or source types
For more efficient searches and better performance, restrict field filters to one or more hosts, sources, or source types. For example, your searches will be faster if you configure your field filter to filter events from a specific host called hostname1
. Instead of filtering all events, only those events from the host hostname1
are filtered.
Keep the following considerations in mind when you plan to restrict your field filter to specific indexes, hosts, sources, or source types:
- You can specify only one value for each field filter limit type, but you can specify multiple hosts, sources, or source types limits.
- The field filter limit setting doesn't support statements that include wildcards or the following operators: AND, OR.
Steps
To restrict a field filter from specific hosts, sources, or source types, follow these steps.
- Open or create a local field_filters.conf file at $SPLUNK_HOME/etc/system/local.
- In the [<fieldFilterName>] stanza for the field filter you want to modify, add a new line using Backus–Naur form (BNF) for the
limit
setting using the following BNF for <actions> syntax:
limit = <limit_type>::host name | source name | source type name
The
limit_type
is host, source, or source type. Only one value can be specified for the field filter limit type.One or more comma-separated values can be specified for the name of the host, source, or source type.
Examples
1. Set the source type for a field
Say you have a field filter that redacts values of the IP_addr
field in searches with the string xxxx
. The IP_addr field filter applies to all events with the IP_addr
field.
To apply the IP_addr
field filter only to events from the accounts
index that have the IP_addr
field and the cisco_syslog
source type, add the following lines to your field_filters.conf file:
[filter_IP_addr_indexed] index = accounts limit = sourcetype::cisco_syslog action = "IP_addr" = xxxx description = Replace the IP_addr field with xxx.
See also
Create field filters using configuration files | Turn off Splunk platform field filters |
This documentation applies to the following versions of Splunk® Enterprise: 9.3.0, 9.3.1, 9.3.2
Feedback submitted, thanks!