Splunk® Enterprise

Getting Data In

Download manual as PDF

Splunk version 4.x reached its End of Life on October 1, 2013. Please see the migration information.
This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Download topic as PDF

Data preview and distributed Splunk

You can use data preview to create new source types, which you can then assign to inputs from specific files/directories or from tcp/udp. Data preview saves any new source type to a props.conf configuration file on the Splunk instance you're running it on. If you want to use the source type on other Splunk instances, you can distribute the file as needed.

There are two steps to using a new source type in a distributed environment, where you have forwarders consuming data and then forwarding the data to indexers:

1. Distribute the props.conf file containing the source type definition to any indexers that will be indexing data with the source type.

2. You can then use the new source type when you define an input on forwarders sending data to those indexers.

When a forwarder sends data tagged with the new source type to an indexer, the indexer will be able to correctly process it into events.

This topic first describes the configuration file that data preview creates. It then explains how to distribute the file to Splunk indexers in your deployment. Finally, it tells you how to specify the new source type when defining an input on a forwarder.

For detailed information on distributed Splunk, read the Distributed Deployment Manual.

The data preview props.conf file

When you create a new source type in data preview, Splunk saves the source type definition as a stanza in a props.conf file in the data preview apps directory: $SPLUNK_HOME/etc/apps/splunk_datapreview/local/.

The first time you use data preview to create a source type, Splunk generates a new props.conf file in $SPLUNK_HOME/etc/apps/splunk_datapreview/local/. If you later create additional source types, Splunk saves the additional source types to the same props.conf file.

Note: A Splunk instance might have multiple versions of some configuration files, spread across several directories. At run-time, Splunk combines the contents of configuration files according to a set of rules. For background on how configuration files work, read "About configuration files" and "Configuration file precedence".

Distribute props.conf to other indexers

After you create new source types, you can distribute the data preview props.conf file to another Splunk instance. That instance will then be able to index any incoming data that's been tagged with the new source type(s).

Generally, you will want to put the configuration file in its own app directory on the target Splunk instance; for example, $SPLUNK_HOME/etc/apps/splunk_datapreview/local/.

To distribute configuration files to other Splunk instances, you can use Splunk's deployment server or another distribution tool of your choice. To learn how to use the deployment server, read "About deployment server" and the topics that follow it.

Note: Splunk uses the source type definitions in props.conf to parse incoming data into events. For this reason, you can only distribute the file to a Splunk instance that performs parsing; that is, either an indexer or a heavy forwarder.

Specify the new source type in forwarder inputs

Since forwarders (with the exception of the heavy forwarder) do not contain Splunk Web, you usually configure their inputs through the inputs.conf configuration file. When you specify an input in that file, you can also specify the input's source type. For detailed information on inputs.conf, read the section on inputs.conf in the Configuration file reference.

To tag a forwarder input with a new source type, you just add the source type to the input stanza in inputs.conf. For example:

[tcp://:9995]
sourcetype = new_network_type

You must make sure that all of the forwarder's receiving indexers have copies of the data preview props.conf file containing the source type definition for "new_network_type". When the forwarder sends data to the indexers, they will then be able to identify the new source type and correctly format the data. The procedure for distributing props.conf is described earlier in this topic, in the section "Distribute props.conf to other indexers".

Data preview and search head pooling

If you are using the search head pooling feature of distributed search, you need to follow some guidelines to ensure that data preview appears in Splunk Web. This is because data preview is implemented in Splunk as a built-in app. For more information, read "Troubleshoot distributed search" in the Distributed Deployment Manual.

PREVIOUS
Modify event processing
  NEXT
Use a test index to test your inputs

This documentation applies to the following versions of Splunk® Enterprise: 4.3, 4.3.1, 4.3.2, 4.3.3, 4.3.4, 4.3.5, 4.3.6, 4.3.7, 5.0, 5.0.1, 5.0.2, 5.0.3, 5.0.4, 5.0.5, 5.0.6, 5.0.7, 5.0.8, 5.0.9, 5.0.10, 5.0.11, 5.0.12, 5.0.13, 5.0.14, 5.0.15, 5.0.16, 5.0.17, 5.0.18


Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters