Splunk® Enterprise

Getting Data In

Download manual as PDF

This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Download topic as PDF

Modify event processing

If you're not satisfied with how Splunk Enterprise initially processes your data, as described in "View and set source types for event data", you can use data preview to change the event processing settings and save the improved settings as a new source type. Here are the main steps:

1. View the event data, as described in "View and set source types for event data".

2. Modify the event processing settings.

3. Review the effect of your changes and iterate until you are satisfied.

4. Save the modified settings as a new source type.

You can then apply the new source type to any of your inputs.

Modify the event processing settings

Splunk Enterprise is ready to create a new source type by default. The "Sourcetype: System Defaults" drop-down in the "Set sourcetypes" page indicates this. To create the new source type, set the event-breaking and time stamp parameters as shown later in this topic, then save the source type.


62 datapreview adjustevents.png

On the left side of the "Set Sourcetypes" page, there are collapsible tabs and links for the three types of adjustments that you can perform:

  • Event Breaks. Adjust the way that Splunk Enterprise breaks the data into events.
  • Timestamps. Adjust the way Splunk Enterprise determines event timestamps.
  • Advanced mode. Edit props.conf directly.

Event breaks

62 datapreview eventbreaks.png

To modify event break parameters, click on the Event Breaks bar to expand it. The bar opens to display the following buttons:

  • Auto - Splunk Enterprise performs event breaking based on where it finds timestamps in the data.
  • Every line - Splunk Enterprise considers every line a single event.
  • Regex... - Click on this button to specify a regular expression that Splunk Enterprise uses to break data into events.

For detailed information on event linebreaking, see "Configure event linebreaking".

For a primer on regular expression syntax and usage, see Regular-Expressions.info. You can test your regular expression by using it in a search with the rex search command. Splunk Enterprise also maintains a list of useful third-party tools for writing and testing regular expressions.

Timestamps

62 datapreview timestamps.png

To modify time stamp recognition parameters click the Timestamps bar to expand it. The bar opens to reveal these options:

For Extraction, you can choose one of these options:

  • Auto - Splunk Enterprise automatically locates the timestamp.
  • Current Time - Splunk Enterprise uses the current time on the local instance.
  • Advanced - Splunk provides additional advanced parameters to adjust.

The "Advanced" parameters are:

  • Timezone - Select the time zone that you want to use for the events.
  • Timestamp format - Type in a string that represents the time stamp format you expect Splunk Enterprise to use when searching for time stamps within the data.
  • Timestamp prefix - Enter a regular expression that represents the characters that appear before a time stamp.
  • Lookahead - Enter the number of characters that Splunk Enterprise should look into the event (or, for the regular expression that you specified in "Timestamp prefix") for the time stamp.

Important: If you specify a timestamp format in the "Timestamp format" field and the timestamp is not located at the very start of each event, you must also specify a prefix in the Timestamp prefix field. Otherwise, Splunk Enterprise will not be able to process the formatting instructions, and every event will contain a warning about the inability to use strptime. (It's possible that you will still end up with a valid timestamp, based on how Splunk attempts to recover from the problem.)

For detailed information on configuring timestamps, see the topics in the chapter "Configure timestamps".

Advanced

To modify advanced parameters, click the Advanced bar to expand it. The bar opens to reveal options that let you specify source type properties by directly editing the underlying props.conf file.

Here, you can add or change source type properties, by specifying attribute/value pairs. See props.conf for details on how to set these properties.

This box shows the current, complete set of properties for the source type you're editing, including:

  • any settings generated by changes made in the Event Breaks or Timestamps tabs (after you click the Apply button).
  • any pre-existing settings for a source type that was either auto-detected or manually selected when you first fed the file to Data Preview.
  • any settings you apply from the Additional settings text box (after you click the Apply settings button).

62 datapreview advmode.png

For information on how to set source type properties, see props.conf in the Configuration file reference. Also, you can refer to these topics on timestamp configuration and event linebreaking.

How Splunk Enterprise combines settings

The settings you make in Advanced mode always take precedence. For example, if you alter a timestamp setting using the Timestamps tab and also make a conflicting timestamp change in Advanced mode - no matter whether before or after - the Advanced mode change wins.

Starting with highest precedence, here is how Splunk Enterprise combines any adjustments with the underlying default settings:

  • Advanced mode changes
  • Event Breaks/Timestamps changes
  • Settings for the underlying source type, if any
  • Default system settings for all source types

Also, if you return to the Event Breaks or Timestamps tabs after making changes in Advanced mode, the changes will not be visible from those tabs.

Review your changes

When you're ready to view the effect of your changes, select Apply settings. Splunk Web refreshes the screen, so you can review the effect of your changes on the data.

If you want to make further changes, you can now do so, using any of the three adjustment methods available. Once again, select Apply changes to view the effect of the changes on your data.

Save modifications as a new source type

To save the changes as a new source type, click the green "Save As" button next to the "Sourcetype" button. Splunk Web displays a dialog box where you can name your new source type, choose the category in which it should be shown in the "Sourcetype" button dialog, and the application context it should use.

62 datapreview savesourcetype.png

1. Enter the Name of the new source type.

2. Enter the Description of what the source type is.

3. Choose the Category in which the source type should appear when you select the "Sourcetype" button.

4. Choose the App for which the new source type should be used.

5. Click the green Save button to save the source type and return to the "Set Sourcetypes" page.

At this point you can:

  • Click the green Next button to apply the source type to your data and proceed to the Input settings page.
  • Click the white "<" button to go back and choose a new file to upload or monitor.
  • Click the Add data text to return to the beginning of the Add Data wizard.
PREVIOUS
View and set source types for event data
  NEXT
Data preview and distributed Splunk Enterprise

This documentation applies to the following versions of Splunk® Enterprise: 6.2.0, 6.2.1, 6.2.2, 6.2.3, 6.2.4, 6.2.5, 6.2.6, 6.2.7, 6.2.8, 6.2.9, 6.2.10, 6.2.11, 6.2.12, 6.2.13, 6.2.14, 6.2.15


Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters