Splunk® Enterprise

Getting Data In

Download manual as PDF

Download topic as PDF

Monitor First In, First Out (FIFO) queues

This topic describes how to configure a First In, First Out (FIFO) input by editing the inputs.conf file on a Splunk Enterprise instance (Splunk Web does not currently support the definition of FIFO inputs.) If you have Splunk Cloud, use a heavyweight forwarder to read FIFO queues.

Note: Data that you send over FIFO queues does not remain in computer memory and can be an unreliable method for data sources. To ensure data integrity, use the monitor input instead.

Add a FIFO input to inputs.conf

To add a FIFO input, add a stanza for it to inputs.conf in $SPLUNK_HOME/etc/system/local/ or your own custom application directory in $SPLUNK_HOME/etc/apps/.

If you have not worked with configuration files before, read About Configuration Files in the Admin manual before you begin.

This input stanza configures Splunk Enterprise to read from a FIFO queue at the specified path.

[fifo://<path>]
<attribute1> = <val1>
<attribute2> = <val2>
...

You can use the following attributes with FIFO stanzas:

Attribute Description Default
host = <string> The host key/field to a static value for this stanza. The <string> is prepended with 'host::'.

Sets the host key's initial value. This key is used during parsing and indexing to set the host field. It also uses the host field at search time.

The IP address or fully qualified domain name of the host where the data originated
index = <string> The index where events from this input will be stored. The <string> is prepended with 'index::'. main, or whatever you have set as your default index.
sourcetype = <string> The sourcetype key/field for events from this input. Explicitly declares the source type for this data, as opposed to letting it be determined automatically. This is important both for searchability and for applying the relevant formatting for this type of data during parsing and indexing.

Sets the sourcetype key's initial value. This value is used during parsing and indexing to set the source type field. It is also the source type field used at search time.

  • The <string> is prepended with 'sourcetype::'.
  • For more information about source types, see Why source types matter in this manual.
Splunk software picks a source type based on various aspects of the data. There is no hard-coded default.
source = <string> Sets the source key/field for events from this input. The <string> is prepended with 'source::'.

Do not override the source field unless absolutely necessary. The input layer provides a more accurate string to aid in problem analysis and investigation, accurately recording the file from which the data was retreived. Consider use of source types, tagging, and search wildcards before overriding this value.

The input file path.
queue = [parsingQueue|indexQueue] Where the input processor should deposit the events that it reads.

Set to "parsingQueue" to apply props.conf and other parsing rules to your data. Set to "indexQueue" to send your data directly into the index.

Defaults to parsingQueue.
PREVIOUS
Troubleshoot HTTP Event Collector
  NEXT
Monitor changes to your file system

This documentation applies to the following versions of Splunk® Enterprise: 6.0, 6.0.1, 6.0.2, 6.0.3, 6.0.4, 6.0.5, 6.0.6, 6.0.7, 6.0.8, 6.0.9, 6.0.10, 6.0.11, 6.0.12, 6.0.13, 6.0.14, 6.1, 6.1.1, 6.1.2, 6.1.3, 6.1.4, 6.1.5, 6.1.6, 6.1.7, 6.1.8, 6.1.9, 6.1.10, 6.1.11, 6.1.12, 6.1.13, 6.2.0, 6.2.1, 6.2.2, 6.2.3, 6.2.4, 6.2.5, 6.2.6, 6.2.7, 6.2.8, 6.2.9, 6.2.10, 6.2.11, 6.2.12, 6.2.13, 6.2.14, 6.3.0, 6.3.1, 6.3.2, 6.3.3, 6.3.4, 6.3.5, 6.3.6, 6.3.7, 6.3.8, 6.3.9, 6.3.10, 6.3.11, 6.3.12, 6.3.13, 6.4.0, 6.4.1, 6.4.2, 6.4.3, 6.4.4, 6.4.5, 6.4.6, 6.4.7, 6.4.8, 6.4.9, 6.4.10, 6.5.0, 6.5.1, 6.5.1612 (Splunk Cloud only), 6.5.2, 6.5.3, 6.5.4, 6.5.5, 6.5.6, 6.5.7, 6.5.8, 6.5.9, 6.6.0, 6.6.1, 6.6.2, 6.6.3, 6.6.4, 6.6.5, 6.6.6, 6.6.7, 6.6.8, 6.6.9, 6.6.10, 6.6.11, 7.0.0, 7.0.1, 7.0.2, 7.0.3, 7.0.4, 7.0.5, 7.0.6, 7.0.7, 7.1.0, 7.1.1, 7.1.2, 7.1.3, 7.2.0


Comments

Probably should add to this that you need to (most likely) create a default-mode.conf file in the local directory containing the lines below.

[pipeline:fifo]
disabled=false

This feature is disabled by default.

Jonathonbauman
November 29, 2017

Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters