Getting Data In

Override automatic source type assignment

Splunk software attempts to assign a source type to your data automatically. You can specify what source type to assign. You can also configure Splunk software so that it assigns a source type based on either the data input or the data source.

For details on the precedence rules that Splunk software uses to assign source types to data, read How Splunk software assigns source types.

Overrides only work on file and directory monitoring inputs or files you have uploaded. You cannot override the source type on network inputs. Additionally, overrides only affect new data that arrives after you set up the override. To correct the source types of events that have already been indexed, create a tag for the source type instead.

This topic describes how to specify a source type based for data based on its input and source.

Specify source type for an input

You can assign the source type for data coming from a specific input, such as /var/log/. If you have Splunk Enterprise, you do this in Splunk Web or by editing the inputs.conf configuration file. If you have Splunk Cloud, use Splunk Web to define source types.

Note: While assigning source type by input seems like a simple way to handle things, it is not very granular--when you use it, Splunk software assigns the same source type to all data from an input, even if some of the data comes from different sources or hosts. To bypass automatic source type assignment in a more targeted manner, you can assign source types based on the source of the data, as described later in this topic.

Use Splunk Web

When you define a data input, you can set a source type value to be applied to all incoming data from that input. You can pick a source type from a list or enter your own source type value.

To select a source type for an input, change the source type settings for the data input type you want to add. For example, for file inputs:

1. Click Settings in the upper right-hand corner of Splunk Web.

2. In the Data section of the Settings pop-up, click Data Inputs.

3. Click Files & Directories.

4. Click the New button to add an input.

5. In the "Add Data" page, browse or enter the name of the file you want to monitor, then click "Next".

6. In the "Set Sourcetype" page, click the "Sourcetype" drop-down and choose from the list of pretrained source types. Splunk Web updates the page to show how the data looks when it receives the new source type.

7. If you want to make changes to the source type, use the "Event Breaks", "Timestamp", and "Advanced" tabs to modify settings and refresh the data preview. See The Set Sourcetype page in this manual.

8. If you want to save the source type as a different name, click Save As… to open a dialog box to save the new source type. Otherwise, proceed to Step 10.

9. If you chose to save the source type, Splunk Web displays the "Save Sourcetype" dialog. Enter the name, description, category, and app that the source type should apply to. See Save modifications as a new source type.

10. Click "Next" to set the source type for the data and proceed to the Input settings" page.

Splunk software now assigns your selected source type to all events it indexes for that input.

Use the inputs.conf configuration file

When you configure an input in inputs.conf, you can specify a source type for the input. Edit inputs.conf in $SPLUNK_HOME/etc/system/local/ or in your own custom application directory in $SPLUNK_HOME/etc/apps/. For information on configuration files in general, see About configuration files in the Admin manual.

To specify a source type, include a sourcetype attribute within the stanza for the input. For example:

[tcp://:9995]
connection_host=dns
sourcetype=log4j
source=tcp:9995

This example sets the source type to "log4j" for any events coming from your TCP input on port 9995.

Caution: Do not put quotes around the attribute value: sourcetype=log4j, not sourcetype="log4j".

Specify source type for a source

Use props.conf to override automated source type matching and explicitly assign a single source type to all data coming from a specific source.

Edit props.conf in $SPLUNK_HOME/etc/system/local/ or in your own custom application directory in $SPLUNK_HOME/etc/apps/. For information on configuration files in general, see About configuration files.

If you want to override a source type, you must configure the setting in props.conf on the forwarder where the input is configured.

To override source type assignment, add a stanza for your source to props.conf. In the stanza, identify the source path, using regular expression (regex) syntax for flexibility if necessary. Then specify the source type by including a sourcetype attribute. For example:

[source::.../var/log/anaconda.log(.\d+)?]
sourcetype=anaconda 

This example sets the source type to "anaconda" for events from any sources containing the string /var/log/anaconda.log followed by any number of numeric characters.

Your stanza source path regular expressions (such as [source::.../web/....log]) should be as specific as possible. Avoid using a regex that ends in "...". For example, do not do this:

[source::/home/fflanda/...]
sourcetype=mytype

This is dangerous. It tells Splunk software to process any gzip files in /home/fflanda as "mytype" files rather than gzip files.

Instead, write:

[source::/home/fflanda/....log(.\d+)?]
sourcetype=mytype