Splunk® Enterprise

Getting Data In

Acrobat logo Download manual as PDF


Splunk Enterprise version 8.1 will no longer be supported as of April 19, 2023. See the Splunk Software Support Policy for details. For information about upgrading to a supported version, see How to upgrade Splunk Enterprise.
Acrobat logo Download topic as PDF

Manage source types

Create, edit, and delete source types on the Source Types page. To get to the Source Types page in Splunk Web, go to Settings > Source types. While this page and the Set Source Type page have similar names, the pages offer different functions.

The Source Types page displays all source types that have been configured on a instance. It shows the default source types provided by your deployment and any source types that you added.

Sort source types

By default, the Source Types page sorts source types alphabetically. You can change how the page sorts by clicking the header bar for the Name, Category, and App columns.

Each header bar, except for Actions, acts as a toggle. Click once to sort in ascending order and click again to sort in descending order.

Filter source types

You can filter the number of source types you see on the Source Type page.

Filter by commonality

To see only the most common source types, click the Show only popular checkbox along the top of the page. Popular source types are the most common source types. They have a pulldown_type source type field value of 1. When the Show only popular checkbox is not selected, the page shows all source types that are defined on the instance.

Filter by category

To see only source types that belong to a certain category, click the Category drop-down list and select the category you want. Only source types that belong to that category will display. To see all source types again, select All from the Category drop-down list.

Filter by application context

To see only source types that belong in a certain application context, click the App drop-down list and select the application context that the source type applies to. Only source types that apply to that application context will display. To see all source types again, select All from the App drop-down list.

Filter by string

To see only source types whose names contain a certain string, type that string in the Filter text box next to the App drop-down list, then press Enter. Only source types whose names or descriptions match what you have typed in the Filter box display. To see all source types again, click the x button on the right side of the Filter text box.

By default, the Source Types management page shows up to 20 source types on a page. If you want to see more or less, click 20 per page on the right side of the page and select the number of source types you want to see. Choices are 10, 20, 50, or 100 listings on a page.

Modify source types

To modify a source type, click its name in the list, or click Edit in the Actions column. The Edit Source type page appears.

The Edit Source Type dialog box lets you change the configuration of a source type. You can change the following fields:

Field Description
Description The description of the source type.
Destination app The application context that the source type applies to.

You cannot change the app destination for source types that come with Splunk software.

Category The category that the source type is a member of. Click the button to select from the list of categories and choose the one you want. When you save, the source type appears in the category you selected.

The Log to Metrics source type category is used specifically for converting log events to metric data points. See About the Log to Metrics source type category later in this topic.

Indexed Extractions A format for extracting fields at index time from files with structured data. Select the type of indexed extraction that best represents the contents of the file:
  • none: The file does not contain structured data.
  • json: The file is in JavaScript Object Notation (JSON) format.
  • csv: The file has comma-separated values.
  • tsv: The file has tab-separated values.
  • psv: The file has pipe (|) separated values.
  • w3c: The file conforms to the World Wide Web Consortium (W3C) logging format.
  • field_extraction: The file contains unstructured events with fields in <field>=<value> format.
Timestamp How timestamps are determined for events from the source file. See the Timestamp section later in this topic.
Advanced Shows all of the configurations for the source type in key/value format. See the Advanced section later in this topic.


Timestamp

The Timestamp section of the dialog box controls how timestamps are determined for events from the source file. You can choose the following options:

  • Auto: Use default logic to extract timestamp from event.
  • Current Time: Use current system time.
  • Advanced: Use advanced internal logic to extract timestamp from event.

The following advanced configurations are available when you select Advanced in the Timestamp Extraction section:

  • Time zone: Specify the time zone to be assigned to the timestamps.
  • Timestamp format: Specify the format of the timestamp in the source event. The available formats come from the properties of the strptime() programming function.

For example, suppose the source file contains logs with timestamps in this format:

6 Jun 2015 18:35:05

Then, you might specify the following in the Timestamp format field:

%d %b %Y %H:%M:%S

Consider the following example:

Tue Jun 4 2:55:18 PM 2015

If the source file contains logs with timestamps in that format, specify the following in the Timestamp format field:

%a %b %d %I:%M:%S %p %Y

For a list of the strings that you can use to define the timestamp format, see strptime(3) at https://linux.die.net/man/3/strptime on the die.net Linux site.

Timestamp prefix: A regular expression that represents the characters that come before a timestamp. When sees this set of characters in an event, it expects a timestamp to occur after that.

Lookahead: Specifies the maximum number of characters to scan into an event for the timestamp. If you specify a regular expression in the Timestamp prefix field, it looks no more than the number of characters specified past the string that the regular expression represents for the timestamp.

Advanced

The Advanced section shows you all of the configurations for the source type in key/value format. This represents what is in the props.conf configuration file that defines the source type. You can edit each setting directly, or add and delete settings.

Use the Advanced section with care. Adding or changing values here can cause data to be incorrectly indexed.

To delete settings, click the x on the right side of each setting. To add an entry, click New setting at the bottom of the dialog box to expose a key/value pair of fields. Enter the key name in the Name field and its value in the Value field.

The Source Types page works differently for Splunk Victoria. You cannot choose which app to put configurations into, as all configurations go into the 000-self-service app. Additionally, you cannot set search-time props because 000-self-service does not export props globally.

About the Log to Metrics source type category

The Log to Metrics source type category is used for the ingest-time conversion of logs to metric data points. If you select it, a Metrics tab appears. For more information about this source type and the fields in the Metrics tab, see Convert event logs to metric data points in Metrics.

Add a source type

To create a new source type:

  1. Click New Source Type. The Create Source Type dialog box opens.
  2. Specify a name for the new source type. Source type names do not support the following characters: <, >, ?, #, or &.
  3. Follow the same steps used to modify an existing source type to configure your new source type. See Modify source types earlier in this topic.
  4. Click Save.

Delete a source type

To delete a source type, click the Delete link in the Actions column for the source type that you want to delete. You can only delete a source type in Cloud if you are a Victoria Experience customer.

If you have a Splunk Cloud Platform instance and are unable to delete one of your user or app created source types, please contact Splunk Support for assistance. You cannot delete the source types built-in to Splunk Enterprise.

When you delete a source type, the following dialog appears:

71 Sourcetype delete.png

Deleting a source type has significant consequences, especially if the source type is already in use:

  • Data can be indexed incorrectly after you delete the source type. Making the data searchable in the way you want later can take a lot of effort. Many apps and add-ons use source types to look for data, and data indexed under a missing source type is data those apps and add-ons do not see.
  • Any configurations that the source type uses, such as field extractions, index time filtering, and timestamp formats, are irretrievably lost.
  • You cannot undo a source type deletion. The only options available in this case are to restore the props.conf file that defines the source type from a backup, or recreate the source type manually.

If you are sure you want to delete the source type, click Delete.

The dialog box closes, and Splunk Web returns you to the Source Types management page.

Last modified on 25 July, 2023
PREVIOUS
Create source types
  NEXT
Rename source types at search time

This documentation applies to the following versions of Splunk® Enterprise: 7.2.0, 7.2.1, 7.2.2, 7.2.3, 7.2.4, 7.2.5, 7.2.6, 7.2.7, 7.2.8, 7.2.9, 7.2.10, 7.3.0, 7.3.1, 7.3.2, 7.3.3, 7.3.4, 7.3.5, 7.3.6, 7.3.7, 7.3.8, 7.3.9, 8.0.0, 8.0.1, 8.0.2, 8.0.3, 8.0.4, 8.0.5, 8.0.6, 8.0.7, 8.0.8, 8.0.9, 8.0.10, 8.1.0, 8.1.1, 8.1.2, 8.1.3, 8.1.4, 8.1.5, 8.1.6, 8.1.7, 8.1.8, 8.1.9, 8.1.10, 8.1.11, 8.1.12, 8.1.13, 8.1.14, 8.2.0, 8.2.1, 8.2.2, 8.2.3, 8.2.4, 8.2.5, 8.2.6, 8.2.7, 8.2.8, 8.2.9, 8.2.10, 8.2.11, 8.2.12, 9.0.0, 9.0.1, 9.0.2, 9.0.3, 9.0.4, 9.0.5, 9.0.6, 9.0.7, 9.0.8, 9.1.0, 9.1.1, 9.1.2, 9.1.3, 9.2.0


Was this documentation topic helpful?


You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters