Splunk® Enterprise

Getting Data In

Download manual as PDF

This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Download topic as PDF

Override automatic source type assignment

Splunk Enterprise attempts to assign a source type to your data automatically. You can specify what source type to assign. You can also configure Splunk Enterprise so that it assigns a source type based on either the data input or the data source.

For details on the precedence rules that Splunk Enterprise uses to assign source types to data, read How Splunk assigns source types.

Important: Overrides only work on file and directory monitoring inputs or files you have uploaded. You cannot override the source type on network inputs. Additionally, overrides only affect new data that arrives after you set up the override. To correct the source types of events that have already been indexed, create a tag for the source type instead.

This topic describes how to specify a source type based for data based on its:

Specify source type for an input

You can assign the source type for data coming from a specific input, such as /var/log/. You do this in either Splunk Web or the inputs.conf configuration file.

Note: While assigning source type by input seems like a simple way to handle things, it isn't very granular--when you use it, Splunk Enterprise assigns the same source type to all data from an input, even if some of the data comes from different sources or hosts. To bypass automatic source type assignment in a more targeted manner, you can arrange for Splunk to assign source types based on the source of the data, as described later in this topic.

Use Splunk Web

When you define a data input, you can set a source type value that Splunk Enterprise applies to all incoming data from that input. Splunk Enterprise gives you the option of picking a source type from a list or entering a unique source type value of your own.

To select a source type for an input, change the source type settings for the data input type you want to add. For example, for file inputs:

1. Click Settings in the upper right-hand corner of Splunk Web.

2. In the Data section of the Settings pop-up, click Data Inputs.

3. Click Files & Directories.

4. Click the New button to add an input.

5. In the "Add Data" page, browse or enter the name of the file you want to monitor, then click "Next".

6. In the "Set Sourcetype" page, click the "Sourcetype" drop-down and choose from the list of pretrained source types. Splunk Enterprise updates the page to show how the data looks when it receives the new source type.

7. If you want to make changes to the source type, use the "Event Breaks", "Timestamp", and "Advanced" tabs to modify settings and refresh the data preview. See "The Set Sourcetype page" in this manual.

8. If you want to save the source type as a different name, click Save As… to open a dialog box to save the new source type. Otherwise, proceed to Step 10.

9. If you chose to save the source type, Splunk Enterprise displays the "Save Sourcetype" dialog. Enter the name, description, category, and app that the source type should apply to. See "Save modifications as a new source type."

10. Click "Next" to set the source type for the data and proceed to the Input settings" page.

Splunk Enterprise now assigns your selected source type to all events it indexes for that input.

Use the inputs.conf configuration file

When you configure an input in inputs.conf, you can specify a source type for the input. Edit inputs.conf in $SPLUNK_HOME/etc/system/local/ or in your own custom application directory in $SPLUNK_HOME/etc/apps/. For information on configuration files in general, see "About configuration files" in the Admin manual.

To specify a source type, include a sourcetype attribute within the stanza for the input. For example:

[tcp://:9995]
connection_host=dns
sourcetype=log4j
source=tcp:9995

This example sets the source type to "log4j" for any events coming from your TCP input on port 9995.

Warning: Do not put quotes around the attribute value: sourcetype=log4j, not sourcetype="log4j".

Specify source type for a source

Use props.conf to override automated source type matching and explicitly assign a single source type to all data coming from a specific source.

Edit props.conf in $SPLUNK_HOME/etc/system/local/ or in your own custom application directory in $SPLUNK_HOME/etc/apps/. For information on configuration files in general, see "About configuration files".

Important: If you are forwarding data, and you want to assign a source type for a source, you must do this in props.conf on the forwarder. If you do it in props.conf on the receiver, the override does not take effect.

To override source type assignment, add a stanza for your source to props.conf. In the stanza, identify the source path, using regular expression (regex) syntax for flexibility if necessary. Then specify the source type by including a sourcetype attribute. For example:

[source::.../var/log/anaconda.log(.\d+)?]
sourcetype=anaconda 

This example sets the source type to "anaconda" for events from any sources containing the string /var/log/anaconda.log followed by any number of numeric characters.

Important: Your stanza source path regexes (such as [source::.../web/....log]) should be as specific as possible. Avoid using a regex that ends in "...". For example, don't do this:

[source::/home/fflanda/...]
sourcetype=mytype

This is dangerous. It tells Splunk to process any gzip files in /home/fflanda as "mytype" files rather than gzip files.

It would be much better to write:

[source::/home/fflanda/....log(.\d+)?]
sourcetype=mytype

Note: For a primer on regular expression syntax and usage, see Regular-Expressions.info. You can test regexes by using them in searches with the rex search command. The Splunk community wiki also has a list of useful third-party tools for writing and testing regular expressions.

PREVIOUS
Why source types matter
  NEXT
Configure rule-based source type recognition

This documentation applies to the following versions of Splunk® Enterprise: 6.2.0, 6.2.1, 6.2.2, 6.2.3, 6.2.4, 6.2.5, 6.2.6, 6.2.7, 6.2.8, 6.2.9, 6.2.10, 6.2.11, 6.2.12, 6.2.13, 6.2.14, 6.2.15


Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters