About default fields (host, source, sourcetype, and more)

About default fields (host, source, sourcetype, and more)

When Splunk indexes data, it tags each event with a number of fields. These fields become part of the index's event data. The fields that Splunk adds automatically are known as default fields.

Default fields serve a number of purposes. For example, the default field index identifies the index in which the event is located. The default field linecount describes the number of lines the event contains, and timestamp specifies the time at which the event occurred. Splunk uses the values in some of the fields, particularly sourcetype, when indexing the data, in order to create events properly. Once the data has been indexed, you can use the default fields in your searches.

Here's the complete list of default fields:

Type of field	List of fields	Description
Internal fields	_raw, _time, _indextime	These fields contain information that Splunk uses for its internal processes.
Basic default fields	host, index, linecount, punct, source, sourcetype, splunk_server, timestamp	These fields provide basic information about an event, such as where it originated, what kind of data it contains,what index it's located in, how many lines it contains, and when it occurred.
Default datetime fields	date_hour, date_mday, date_minute, date_month, date_second, date_wday, date_year, date_zone	These fields provide additional searchable granularity to event timestamps. Note: Only events that have timestamp information in them as generated by their respective systems will have date_* fields. If an event has a date_* field, it represents the value of time/date directly from the event itself. If you have specified any timezone conversions or changed the value of the time/date at indexing or input time (for example, by setting the timestamp to be the time at index or input time), these fields will not represent that.

For information about default fields from the search perspective, see "Use default fields" in the User manual.

Note: You can also specify additional, custom fields for Splunk to include in the index. See "Create custom fields at index-time" in this chapter.

This topic focuses on three key default fields:

• host
• source
• sourcetype

Defining host, source, and sourcetype

The host, source, and sourcetype fields are defined as follows:

• host - An event's host value is typically the hostname, IP address, or fully qualified domain name of the network host from which the event originated. The host value enables you to easily locate data originating from a specific device. For more information on hosts, see "About hosts".
• source - The source of an event is the name of the file, stream, or other input from which the event originates. For data monitored from files and directories, the value of source is the full path, such as /archive/server1/var/log/messages.0 or /var/log/. The value of source for network-based data sources is the protocol and port, such as UDP:514.
• sourcetype - The source type of an event is the format of the data input from which it originates, such as access_combined or cisco_syslog. The source type determines how Splunk formats your data. For more information on source types, see "Why source types matter".

Source vs sourcetype

Don't confuse source and sourcetype! They're both default fields, but they're entirely different otherwise:

• The source is the name of the file, stream, or other input from which a particular event originates.

• The sourcetype field specifies the format for the event. Splunk uses this field to determine how to format the incoming data stream into individual events.

Events with the same source type can come from different sources. For example, say you're monitoring source=/var/log/messages and receiving direct syslog input from udp:514. If you search sourcetype=linux_syslog, Splunk will return events from both of those sources.

Under what conditions should you override host and sourcetype assignment?

Much of the time, Splunk can automatically identify host and sourcetype values that are both correct and useful. But situations do come up that require you to intervene in this process and provide override values.

Override host assignment

You might want to change your default host assignment when:

• You are bulk-loading archive data that was originally generated from a different host and you want those events to have that host value.
• Your data is being forwarded from a different host. (The forwarder will be the host unless you specify otherwise.)
• You are working with a centralized log server environment, which means that all of the data received from that server will have the same host, even if it originated elsewhere.

For detailed information about hosts, see the chapter "Configure host values".

Override sourcetype assignment

You might want to change your default sourcetype assignment when:

• Splunk is unable to automatically format the data properly, resulting in problems such as wrong timestamping or event linebreaking.
• You want to apply source types to specific events coming through a particular input, such as events that originate from a discrete group of hosts, or even events that are associated with a particular IP address or userid.

There are also steps you can take to expand the range of source types that Splunk automatically recognizes, or to simply rename source types.

For detailed information about source types, see the chapter "Configure source types".

• Was this topic useful?
• Post a comment