Splunk Cloud Platform

Getting Data In

Override automatic source type assignment

attempts to assign a source type to your data automatically. You can specify what source type to assign. You can also configure the Splunk platform so that it assigns a source type based on either the data input or the data source.

For details on the precedence rules that the Splunk platform uses to assign source types to data, read How the Splunk platform assigns source types in Why source types matter.

Overrides work only on file and directory monitoring inputs or files you upload. You can't override the source type on network inputs. Additionally, overrides affect only new data that arrives after you set up the override. To correct the source types of events that are already indexed, create a tag for the source type instead. See Tag field-value pairs in Search in the Knowledge Manager Manual.

You can specify a source type for data based on its input and source.

Specify source type for an input

You can assign the source type for data coming from a specific input, such as /var/log/. If you use Splunk Cloud Platform, use Splunk Web to define source types. If you use Splunk Enterprise, define source types in Splunk Web or by editing the inputs.conf configuration file.

Be aware that assigning source type by input is not very granular. When you specify source type by input, the Splunk platform assigns the same source type to all data from an input, even if some of the data comes from different sources or hosts. To bypass automatic source type assignment in a more targeted manner, you can assign source types based on the source of the data, as described in the Specify source type for a source section.

Use Splunk Web

When you define a data input, you can set a source type value to be applied to all incoming data from that input. You can pick a source type from a list or enter your own source type value.

To select a source type for an input, change the source type settings for the data input type you want to add. For example, for file inputs, complete the following steps:

  1. Click Settings in the upper right-hand corner of Splunk Web.
  2. In the Data section of the Settings drop-down list, click Data Inputs.
  3. Click Files & Directories.
  4. Click New to add an input.
  5. In the Add Data page, browse or enter the name of the file you want to monitor, then click Next.
  6. In the Set Source Type page, click the Sourcetype drop-down list and choose from the list of pretrained source types. See List of pretrained source types.
    Splunk Web updates the page to show how the data looks when it receives the new source type.
  7. If you want to make changes to the source type, use the Event Breaks, Timestamp, and Advanced tabs to modify settings and refresh the data preview. See Assign the correct source types to your data in this manual.
  8. If you want to save the source type as a different name, click Save As… to open the Save Sourcetype dialog box to save the new source type. Otherwise, proceed to Step 10.
  9. If you choose to save the source type, enter the name, description, category, and app that the source type will apply to. See Save modifications as a new source type.
  10. Click Next to set the source type for the data and proceed to the Input Settings page. See Modify input settings.

now assigns your selected source type to all events it indexes for that input.

Use the inputs.conf configuration file

When you configure an input in the inputs.conf configuration file on a Splunk Enterprise instance, you can specify a source type for the input. On a Splunk Cloud Platform instance, you can configure a universal forwarder on the machine that has the data you want to collect and forward it to the Splunk Cloud Platform instance.

Edit the inputs.conf file in $SPLUNK_HOME/etc/system/local/ or in your own custom application directory in $SPLUNK_HOME/etc/apps/. For information on configuration files in general, see About configuration files in the Admin Manual.

To specify a source type, include a sourcetype attribute within the stanza for the input. For example:

[tcp://:9995]
connection_host=dns
sourcetype=log4j
source=tcp:9995

This example sets the source type to log4j for any events coming from your TCP input on port 9995.

Do not put quotes around the attribute value. The correct format, for example, is sourcetype=log4j, not sourcetype="log4j".

Specify source type for a source

Use the props.conf file to override automated source type matching and explicitly assign a single source type to all data coming from a specific source.

Edit props.conf in $SPLUNK_HOME/etc/system/local/ or in your own custom application directory in $SPLUNK_HOME/etc/apps/. For information on configuration files in general, see About configuration files in the Admin Manual.

If you want to override a source type, you must configure the setting in props.conf on the forwarder where the input is configured.

To override source type assignment, add a stanza for your source to props.conf. In the stanza, identify the source path, using regular expression (regex) syntax for flexibility if necessary. Then specify the source type by including a sourcetype attribute. For example:

[source::.../var/log/anaconda.log(.\d+)?]
sourcetype=anaconda 

This example sets the source type to anaconda for events from any sources containing the string /var/log/anaconda.log followed by any number of numeric characters.

Your stanza source path regular expressions, such as [source::.../web/....log], must be as specific as possible. Avoid using a regex that ends in .... For example, do not do this:

[source::/home/fflanda/...]
sourcetype=mytype

This formatting is dangerous. It tells the Splunk platform to process any GZIP files in /home/fflanda as mytype files rather than GZIP files.

Instead, write using the following format:

[source::/home/fflanda/....log(.\d+)?]
sourcetype=mytype
Last modified on 27 October, 2021
Why source types matter   Configure rule-based source type recognition

This documentation applies to the following versions of Splunk Cloud Platform: 8.2.2112, 8.2.2201, 8.2.2202, 8.2.2203, 9.0.2205, 9.0.2208, 9.0.2209, 9.0.2303, 9.0.2305, 9.1.2308, 9.1.2312, 9.2.2403, 9.2.2406 (latest FedRAMP release), 9.3.2408


Was this topic useful?







You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters