persistent queue

persistent queue

noun

A feature that you configure to take data that is in an input queue and store it to files on disk. Using a persistent queue can prevent data loss if the forwarder or indexer has too much data to process at one time.

By default, forwarders and indexers have an in-memory input queue of 500 KB. Without persistent queues, there is a potential for data loss if the input stream overflows the in-memory queue. With persistent queues, when the in-memory queue is full, the forwarder or indexer writes the input stream to files on disk. The forwarder or indexer then processes data from the queues (in-memory and disk) until it reaches the point when it can again start processing directly from the data stream.

You enable persistent queues in the inputs.conf configuration file on an input-by-input basis. Persistent queues are disabled by default.

For more information

In Getting Data In:

• Use persistent queues to help prevent data loss