About indexed field extraction

When Splunk software indexes data, it parses the data stream into a series of events. As part of this process, the software adds a number of fields to the event data. These fields include default fields that it adds automatically, as well as any custom fields that you specify.

The process of adding fields to events is known as field extraction. There are two types of field extraction:

• Indexed field extraction, which takes place when the fields are stored in the index and become part of the event data. There are two types of indexed fields:
  • Default fields, which Splunk software automatically adds to each event. For more details, see About default fields.
  • Custom fields, which you specify. For more details, see Create custom fields at index time.
• Search-time field extraction, which takes place when you search through data. Splunk software creates those fields when compiling search results and does not store them in the index. For information about this type of field extraction, see About fields in the Knowledge Manager Manual.

When working with fields, consider that most machine data either doesn't have structure or has structure that changes constantly. For unstructured data, use search-time field extraction for maximum flexibility. Search-time field extraction is easy to modify after you define it.

Other types of data might exhibit a more fixed structure, or the structure might already be defined within the data or events in the file. You can configure Splunk software to read the structure of these kinds of files, such as comma-separated value (CSV) files, tab-separated value (TSV) files, pipe-separated value files, and JavaScript Object Notation (JSON) data sources, and map fields at index time. To learn how these processes work, see Extract fields from files with structured data.