Data preview and distributed Splunk
Data preview and distributed Splunk
You can use data preview to create new source types, which you can then assign to inputs from specific files/directories or from tcp/udp. Data preview saves any new source type to a
props.conf configuration file on the Splunk instance you're running it on. If you want to use the source type on other Splunk instances, you can distribute the file as needed.
There are two steps to using a new source type in a distributed environment, where you have forwarders consuming data and then forwarding the data to indexers:
1. Distribute the
props.conf file containing the source type definition to any indexers that will be indexing data with the source type.
2. You can then use the new source type when you define an input on forwarders sending data to those indexers.
When a forwarder sends data tagged with the new source type to an indexer, the indexer will be able to correctly process it into events.
This topic first describes the configuration file that data preview creates. It then explains how to distribute the file to Splunk indexers in your deployment. Finally, it tells you how to specify the new source type when defining an input on a forwarder.
For detailed information on distributed Splunk, read the Distributed Deployment Manual.
The data preview props.conf file
When you create a new source type in data preview, Splunk saves the source type definition as a stanza in a props.conf file in the data preview apps directory:
The first time you use data preview to create a source type, Splunk generates a new
props.conf file in
$SPLUNK_HOME/etc/apps/splunk_datapreview/local/. If you later create additional source types, Splunk saves the additional source types to the same
Note: A Splunk instance might have multiple versions of some configuration files, spread across several directories. At run-time, Splunk combines the contents of configuration files according to a set of rules. For background on how configuration files work, read "About configuration files" and "Configuration file precedence".
Distribute props.conf to other indexers
After you create new source types, you can distribute the data preview
props.conf file to another Splunk instance. That instance will then be able to index any incoming data that's been tagged with the new source type(s).
Generally, you will want to put the configuration file in its own app directory on the target Splunk instance; for example,
To distribute configuration files to other Splunk instances, you can use Splunk's deployment server or another distribution tool of your choice. To learn how to use the deployment server, read "About deployment server" and the topics that follow it.
Note: Splunk uses the source type definitions in
props.conf to parse incoming data into events. For this reason, you can only distribute the file to a Splunk instance that performs parsing; that is, either an indexer or a heavy forwarder.
Specify the new source type in forwarder inputs
Since forwarders (with the exception of the heavy forwarder) do not contain Splunk Web, you usually configure their inputs through the
inputs.conf configuration file. When you specify an input in that file, you can also specify the input's source type. For detailed information on
inputs.conf, read the section on
inputs.conf in the Configuration file reference.
To tag a forwarder input with a new source type, you just add the source type to the input stanza in
inputs.conf. For example:
[tcp://:9995] sourcetype = new_network_type
You must make sure that all of the forwarder's receiving indexers have copies of the data preview
props.conf file containing the source type definition for "new_network_type". When the forwarder sends data to the indexers, they will then be able to identify the new source type and correctly format the data. The procedure for distributing
props.conf is described earlier in this topic, in the section "Distribute props.conf to other indexers".
Data preview and search head pooling
If you are using the search head pooling feature of distributed search, you need to follow some guidelines to ensure that data preview appears in Splunk Web. This is because data preview is implemented in Splunk as a built-in app. For more information, read "Troubleshoot distributed search" in the Distributed Deployment Manual.