Modify event processing

Modify event processing

If you're not satisfied with how Splunk initially processes your data, as described in "View event data", you can use data preview to change the event processing settings and save the improved settings as a new source type. Here are the main steps:

1. View the event data, as described in "View event data".

2. Modify the event processing settings.

3. Review the effect of your changes and iterate until you're satisfied.

4. Save the modified settings as a new source type.

You can then apply the new source type to any of your inputs.

Modify the event processing settings

When you select adjust timestamp and event break settings on the initial data preview page, Splunk Web takes you to a page that looks like this:

On the upper part of the page, there are tabs and links for the three types of adjustments you can perform:

• Event Breaks. Adjust the way that Splunk breaks the data into events.
• Timestamps. Adjust the way Splunk determines event timestamps.
• Advanced mode. Edit props.conf directly.

Event breaks

If you select Event Breaks, you have these choices:

• Auto (break on timestamp)
• Every line is one event
• Specify a pattern or regex to break before - specify the regex in the text field

For detailed information on event linebreaking, see "Configure event linebreaking".

For a primer on regular expression syntax and usage, see Regular-Expressions.info. You can test your regex by using it in a search with the rex search command. Splunk also maintains a list of useful third-party tools for writing and testing regular expressions.

Timestamps

If you select Timestamps, you have two sets of choices.

For Location, you can choose one of these options:

• Automatically locate timestamp
• Timestamp is always prefaced by a pattern - you specify the pattern
• Timestamp never extends more than <number> chars into the event

For Format, you can choose any or none of these options:

• Specify timestamp format (strptime)
• Specify timezone

Important: If you specify a timestamp format in the "Format" section and the timestamp is not located at the very start of each event, you must also specify a prefix in the Timestamp is always prefixed by a pattern field in the "Location" section. Otherwise, Splunk will not be able to process the formatting instructions, and every event will contain a warning about the inability to use strptime. (It's possible that you will still end up with a valid timestamp, based on how Splunk attempts to recover from the problem.)

For detailed information on configuring timestamps, see the topics in the chapter "Configure timestamps".

Advanced mode

If you select Advanced mode, Splunk Web takes you to a page where you can specify source type properties by directly editing the underlying props.conf file. Advanced mode presents you with two text boxes:

• Additional settings. Here, you can add or change source type properties, by specifying attribute/value pairs. See props.conf for details on how to set these properties.

• Currently applied settings. This box shows the current, complete set of properties for the source type you're editing, including:
  • any settings generated by changes made in the Event Breaks or Timestamps tabs (after you click the Apply button).
  • any pre-existing settings for a source type that was either auto-detected or manually selected when you first fed the file to Data Preview.
  • any settings you apply from the Additional settings text box (after you click the Apply button).

For information on how to set source type properties, see props.conf in the Configuration file reference. Also, you can refer to these topics on timestamp configuration and event linebreaking.

How Splunk combines settings

The settings you make in Advanced mode always take precedence. For example, if you alter a timestamp setting using the Timestamps tab and also make a conflicting timestamp change in Advanced mode - no matter whether before or after - the Advanced mode change wins.

Starting with highest precedence, here is how Splunk combines any adjustments with the underlying default settings:

• Advanced mode changes
• Event Breaks/Timestamps changes
• Settings for the underlying source type, if any
• Default system settings for all source types

Also, if you return to the Event Breaks or Timestamps tabs after making changes in Advanced mode, the changes will not be visible from those tabs.

Review your changes

When you're ready to view the effect of your changes, select Apply. Splunk Web refreshes the screen, so you can review the effect of your changes on the data.

If you want to make further changes, you can now do so, using any of the three adjustment methods available. Once again, select Apply to view the effect of the changes on your data.

You can select Reset to reset to your previous settings.

When you're satisfied with your settings, select Continue.

Save modifications as a new source type

When you select Continue, Splunk Web takes you to the Review settings dialog box, where you can review the source type settings (including your changes) and name the new source type.

At this point you can:

• Cancel - Don't save the changes as a new source type.
• Quit without indexing - Save the new source type but don't index any new data.
• Add data input - Save the new source type and immediately apply it to a data input. If you select this option, Splunk Web takes you to the Add new page, with the new source type preselected.

• Was this topic useful?
• Post a comment