Splunk® Enterprise

Getting Data In

Download manual as PDF

This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Download topic as PDF

Monitor Windows event log data

Windows generates log data during the course of its operation. The Windows Event Log service handles nearly all of this communication. It gathers log data published by installed applications, services and system processes and places them into event log channels - intermediate locations that eventually get written to an event log file. Programs such as Microsoft's Event Viewer subscribe to these log channels to display events that have occurred on the system.

Splunk Enterprise also supports the monitoring of Windows event log channels. It can monitor event log channels and files stored on the local machine, and it can collect logs from remote machines.

The event log monitor runs as an input processor within the splunkd service. It runs once for every event log input defined in Splunk Enterprise.

Why monitor event logs?

Windows event logs are the core metric of Windows server operations - if there's a problem with your Windows system, the Event Log service likely knows about it. Splunk Enterprise's indexing, searching and reporting capabilities make your logs accessible.

What's required to monitor event logs?

Activity: Required permissions:
Monitor local event logs * Splunk Enterprise must run on Windows
* Splunk Enterprise must run as the Local System user to read all local event logs
Monitor remote event logs * Splunk Enterprise must run on Windows
AND
* Splunk Enterprise must run on a universal forwarder that is installed on the server you wish to collect event logs from
OR
* Splunk Enterprise must run as a domain or remote user with read access to Windows Management Instrumentation (WMI) on the target server
* The user Splunk Enterprise runs as must have read access to the desired event logs

Security and remote access considerations

Splunk Enterprise collects event log data from remote machines using either WMI or a forwarder. Splunk recommends using a universal forwarder to send event log data from remote machines to an indexer. Review "Introducing the universal forwarder" in the Forwarding Data Manual for information about how to install, configure and use the forwarder to collect event log data.

If you choose to install forwarders on your remote machines to collect event log data, then you can install the forwarder as the Local System user on these machines. The Local System user has access to all data on the local machine, but not on remote machines.

If you want Splunk Enterprise to use WMI to get event log data from remote machines, then you must ensure that your network and Splunk instances are properly configured. You cannot install Splunk as the Local System user, and the user you install with determines the event logs Splunk sees. Review "Security and remote access considerations" in the "Monitor WMI-based data" topic in this manual for additional information on the requirements you must satisfy in order for Splunk to collect remote data properly using WMI.

By default, Windows restricts access to some event logs depending on which version of Windows you run. In particular, the Security event logs by default can only be read by members of the local Administrators or global Domain Admins groups.

Collect event logs from a remote Windows machine

If you want Splunk Enterprise to collect event logs from a remote machine, you have two choices:

  • Collect the logs remotely using WMI. You use this option when you select "Remote event log collections" in Splunk Web.
  • Install a universal forwarder on the machine from which you want to collect logs.

If you choose to collect event logs using WMI, you must install Splunk Enterprise with an Active Directory domain user. Refer to "Considerations for deciding how to monitor remote Windows data" for additional information on collecting data from remote Windows machines. If the selected domain user is not a member of the Administrators or Domain Admins groups, then you must configure event log security to give the domain user access to the event logs.

To change event log security for access to the event logs from remote machines, you must:

For instructions on how to configure event log security permissions on Windows XP and Windows Server 2003/2003 R2, review this Microsoft Knowledge Base article. If you're running Windows Vista, Windows 7 or Windows Server 2008/2008 R2, use the wevtutil utility to set event log security.

Anomalous host names visible in event logs on some systems

On Windows Vista and Server 2008 systems, you might see some event logs with randomly-generated host names. This is the result of those systems logging events before the user has named the system, during the OS installation process.

This anomaly only occurs when collecting logs from the above-mentioned versions of Windows remotely over WMI.

Use Splunk Web to configure event log monitoring

Configure local event log monitoring

1. Click Settings in the upper right-hand corner of Splunk Web.

2. Under Data, click Data Inputs.

3. Click Local event log collections.

4. Click Add new to add an input.

5. Select one or more logs from the list of Available Logs and click to add to the list of Selected Logs.

Note: Certain Windows Event Log channels (known as direct channels) do not allow for users to access - or subscribe to - them in order to monitor them. This is because events sent via these log channels are not actually processed by the Windows Event Log framework, and thus can't be forwarded or collected remotely. Often, these direct channels log directly to disk. Attempts to monitor these log channels will generate the error "The caller is trying to subscribe to a direct channel which is not allowed."

6. Click Save.

Splunk Enterprise adds and enables the input.

Configure remote event log monitoring

1. Click Settings in the upper right-hand corner of Splunk Web.

2. Under Data, click Data Inputs.

3. Click Remote event log collections.

4. Click Add new to add an input.

5. Enter a unique name for this collection.

6. Specify a hostname or IP address for the host from which to pull logs, and click Find logs... to get a list of logs from which to choose.

Note: Windows Vista offers many event log channels in addition to the standard set of channels defined in all versions of Windows. Depending on the CPU available to Splunk, selecting all or a large number of them can result in high load.

7. Optionally, provide a comma-separated list of additional servers from which to pull data.

8. Click Save.

Splunk Enterprise adds and enables the input.

Use inputs.conf to configure event log monitoring

You can edit inputs.conf to configure event log monitoring. For more information on configuring data inputs with inputs.conf, read "Configure your inputs" in this manual.

Note: You can always review the defaults for a configuration file by looking at the examples in %SPLUNK_HOME%\etc\system\default or at the spec file in the Admin Manual.

To enable event log inputs by editing inputs.conf:

1. Copy inputs.conf from %SPLUNK_HOME%\etc\system\default to etc\system\local .

2. Use Explorer or the ATTRIB command to remove the file's "Read Only" flag.

3. Open the file and edit it to enable Windows event log inputs.

4. Restart Splunk.

The next section describes the available configuration values for event log monitoring.

Event log monitor configuration values

Windows event log (*.evt) files are in binary format. They can't be monitored like a normal text file. The splunkd service monitors these binary files by using the appropriate APIs to read and index the data within the files.

Splunk uses the following stanzas in inputs.conf to monitor the default Windows event logs:

# Windows platform specific input processor.
[WinEventLog://Application]
disabled = 0 
[WinEventLog://Security]
disabled = 0 
[WinEventLog://System]
disabled = 0 

You can also configure Splunk Enterprise to monitor non-default Windows event logs. Before you can do this, you must import them to the Windows Event Viewer. Once the logs are imported, you can add them to your local copy of inputs.conf, as follows:

[WinEventLog://DNS Server]
disabled = 0
[WinEventLog://Directory Service]
disabled = 0
[WinEventLog://File Replication Service]
disabled = 0

Note: Use the log properties' "Full Name:" to index. For example, to monitor Task Scheduler in Microsoft> Windows > TaskScheduler >Operational, right click on Operational and select properties. Use the "Full Name" to append to WinEventLog:// stanza:

[WinEventLog://Microsoft-Windows-TaskScheduler/Operational]
disabled = 0

To disable indexing for an event log, add disabled = 1 below its listing in the stanza in %SPLUNK_HOME%\etc\system\local\inputs.conf.

Splunk Enterprise uses the following attributes in inputs.conf to monitor Event Log files:

Attribute Description Default
start_from
  • Tells Splunk Enterprise how it should read events chronologically.
  • Acceptable values are oldest (meaning that Splunk should read logs from the oldest to the newest) and newest (meaning that Splunk should read logs from the newest to the oldest.
  • If you set the attribute to newest, Splunk reads logs from the most recent to the oldest, then stops.

Note: You cannot set this attribute to newest while also setting the current_only attribute to 1 as this does not make sense. Splunk ignores this combination.

oldest
current_only
  • Tells Splunk Enterprise how it should index events after it starts.
  • Acceptable values are 1 (where the input only acquires events that arrive after the input starts for the first time, like 'tail -f' on *nix systems) or 0 (where the input first gets all existing events in the log and then continues to monitor incoming events in real time)

Note: You cannot set this attribute to 1 while also setting the start_from attribute to newest as this does not make sense. Splunk ignores this combination.

0
checkpointInterval
  • Tells Splunk Enterprise how frequently, in seconds, that the Windows Event Log input should save a checkpoint.
  • Checkpoints store the eventID of acquired events.
  • This allows Splunk to continue monitoring at the correct event after a shutdown or outage.
5
evt_resolve_ad_obj
  • Tells Splunk Enterprise how it should interact with Active Directory while indexing Windows Event Log events.
  • Valid values are 1 (which tells Splunk to resolve Active Directory objects like Globally Unique IDentifier (GUID) and Security IDentifier (SID) objects to their canonical names for a specific Windows event log channel) and 0 (which tells Splunk not to attempt any resolution.)
  • When you set this value to 1, you can optionally specify the Domain Controller name and/or DNS name of the domain to bind to, which Splunk will then use to resolve the AD objects.
  • If you do not set this value, Splunk attempts to resolve the AD objects.
1
evt_dc_name
  • Tells Splunk Enterprise which Active Directory domain controller it should bind to in order to resolve AD objects.
  • This name can be the NetBIOS name of the domain controller or the fully-qualified DNS name of the domain controller.
  • Either name type can, optionally, be preceded by two backslash characters.
N/A
evt_dns_name
  • Tells Splunk Enterprise the fully-qualified DNS name of the domain it should bind to in order to resolve AD objects.
N/A
suppress_text
  • Tells Splunk whether or not to include the message text that comes with a security event.
  • A value of 1 suppresses the message text, and a value of 0 preserves the text.
0
whitelist
  • Tells Splunk to index events that match the text string specified.
  • This attribute is optional.
  • You can specify one of two formats:
    • One or more Event Log event codes or event IDs.
    • One or more sets of keys and regular expressions. See "Create advanced filters with whitelist and blacklist" later in this topic for details.
  • You cannot mix formats in a single entry.
  • You also cannot mix formats in the same stanza.
  • Splunk Enterprise processes whitelists first, then blacklists.
  • If no whitelist is present, Splunk Enterprise indexes all events.

When using the Event Code/ID format:

  • For multiple codes/IDs, separate the list with commas.
  • For ranges, use hyphens (for example "0-1000,5000-1000").

When using the advanced filtering format:

  • For advanced filtering, use '=' between the key and the regular expression that represents your filter (for example "whitelist = EventCode=%^1([8-9])$%"
  • You can have multiple key/regular expression sets in a single advanced filtering entry. Splunk Enterprise logically conjuncts the sets. This means that the entry is valid only if all of the sets in the entry are true.
  • You can specify up to 10 whitelists per stanza by adding a number to the end of the whitelist attribute, for example whitelist1...whitelist9.
N/A
blacklist
  • Tells Splunk not to index events that match the text string specified.
  • This attribute is optional.
  • You can specify one of two formats:
    • One or more Event Log event codes or event IDs.
    • One or more sets of keys and regular expressions. See "Create advanced filters with whitelist and blacklist" later in this topic for details.
  • You cannot mix formats in a single entry.
  • You also cannot mix formats in the same stanza.
  • Splunk Enterprise processes whitelists first, then processes any blacklists.
  • If no blacklist is present, Splunk Enterprise indexes all events.

When using the Event Log code/ID format:

  • For multiple codes/IDs, separate the list with commas.
  • For ranges, use hyphens (for example "0-1000,5000-1000").

When using the advanced filtering format:

  • For advanced filtering, use '=' between the key and the regular expression that represents your filter (for example "blacklist = EventCode=%^1([8-9])$%"
  • You can have multiple key/regular expression sets in a single advanced filtering entry. Splunk Enterprise logically conjuncts the sets. This means that the entry is valid only if all of the sets in the entry are true.
  • You can specify up to 10 blacklists per stanza by adding a number to the end of the blacklist attribute, for example blacklist1...blacklist9.
index
  • Tells Splunk Enterprise the index that this input should send the data to.
the default index
disabled
  • Tells Splunk Enterprise whether or not the input should run.
  • Valid values are 0 (meaning that the input should run) and 1 (meaning that the input should not run.
0

Use the Security event log to monitor changes to files

You can monitor changes to files on your system by enabling security auditing on a set of files and/or directories and then monitoring the Security event log channel for change events. The event log monitoring input includes three attributes which you can use in inputs.conf. Here's an example:

[WinEventLog://Security]
disabled = 0
start_from = oldest
current_only = 0
evt_resolve_ad_obj = 1
checkpointInterval = 5
# only index events with these event IDs.
whitelist = 0-2000,3001-10000
# exclude these event IDs from being indexed.
blacklist = 2001-3000

To enable security auditing for a set of files or directories, read "Auditing Security Events How To" (http://technet.microsoft.com/en-us/library/cc727935%28v=ws.10%29.aspx) on MS Technet.

You can also use the suppress_text attribute to include or exclude the message text that comes with a security event:

[WinEventLog://Security]
disabled = 0
start_from = oldest
current_only = 0
evt_resolve_ad_obj = 1
checkpointInterval = 5
# suppress message text, we only want the event number.
suppress_text = 1
# only index events with these event IDs.
whitelist = 0-2000,2001-10000
# exclude these event IDs from being indexed.
blacklist = 2001-3000

By default, suppress_text defaults to 0 (false).

Create advanced filters with 'whitelist' and 'blacklist'

You can perform advanced filtering of incoming events with the whitelist and blacklist attributes in addition to filtering based solely on event codes. To do this, specify the key/regular expression format in the attribute:

whitelist = key=<regular expression> [key=<regular expression] ...

In this format, key is a valid entry from the following list:

Key Description
$TimeGenerated The time that the Operating system or application added the event to the Event Log. This is the timestamp that appears in the text of the event, and will be used for _time.
$Timestamp The time that the event was retreived from the Windows Event Log by Splunk. This string does not appear anywhere in the event. It has the same format as $TimeGenerated.
Category The category number for a specific event source.
CategoryString A string translation of the category. The translation depends on the event source.
ComputerName The name of the computer that generated the event.
EventCode The event ID number for an event. Corresponds to "Event ID" in Event Viewer.
EventType A numeric value that represents one of the the five types of events that can be logged ("Error", "Warning", "Information", "Success Audit", and "Failure Audit".) Available only on server machines running Windows Server 2003 and earlier or clients running Windows XP and earlier. See "Win32_NTLogEvent class (Windows)" (http://msdn.microsoft.com/en-us/library/aa394226(v=vs.85).aspx) on MSDN.
Keywords An element used to classify different types of events within an event log channel. The Security Event Log channel has this element, for example.
LogName The name of the Event Log channel that received the event. Corresponds to "Log Name" in Event Viewer.
Message The text of the message in the event.
OpCode The severity level of the event ("OpCode" in Event Viewer.)
RecordNumber The Windows Event Log record number. Each event on a Windows server gets a record number. This number starts at 0 with the first event generated on the system, and increases with each new event generated, until it reaches a maximum of 4294967295. It then rolls back over to 0.
Sid The Security Identifier (SID) of the principal (such as a user, group, computer, or other entity) that was associated with or generated the event. See "Win32_UserAccount class (http://msdn.microsoft.com/en-us/library/windows/desktop/aa394507%28v=vs.85%29.aspx) on MSDN.
SidType A numeric value that represents the type of SID that was associated with the event. See "Win32_UserAccount class" (http://msdn.microsoft.com/en-us/library/windows/desktop/aa394507%28v=vs.85%29.aspx) on MSDN.
SourceName The source of the entity that generated the event ("Source" in Event Viewer)
TaskCategory The task category of the event. Event sources allow you to define categories so that you can filter them with Event Viewer (using the "Task Category" field.) See Event Categories (Windows) (http://msdn.microsoft.com/en-us/library/aa363649%28VS.85%29.aspx) on MSDN.
Type A numeric value that represents one of the the five types of events that can be logged ("Error", "Warning", "Information", "Success Audit", and "Failure Audit".) Only available on server machines that run Windows Server 2008 or later, or clients that run Windows Vista or later. See "Win32_NTLogEvent class (Windows)" (http://msdn.microsoft.com/en-us/library/aa394226(v=vs.85).aspx) on MSDN.
User The user associated with the event. Correlates to "User" in Event Viewer.

and <regular expression> is any valid regular expression that represents the filters that you want to include (when used with the whitelist attribute) or exclude (when used with the blacklist attribute).

To learn more about regular expressions and how to use them, visit the Regularexpressions.info (http://www.regular-expressions.info) website.

You can specify more than one key/regular expression set on a single entry line. When you do this, Splunk Enterprise logically conjuncts the sets. This means that only events which satisfy all of the sets on the line will be valid for inclusion or exclusion. For example, this entry:

whitelist = EventCode="^1([0-5])$" Message="^Error"

tells Splunk Enterprise to include events that have an EventCode ranging from 10 to 15 and contain a Message that begins with the word Error.

You can specify up to 10 separate whitelist or blacklist entries in each stanza. To do so, add a number at the end of the whitelist or blacklist entry on a separate line:

whitelist = key=<regular expression>
whitelist1 = key=<regular expression> key2=<regular expression 2>
whitelist2 = key=<regular expression>

Note: You cannot specify an entry that has more than one key/regular expression set that references the same key. If, for example, you specify:

whitelist = EventCode="^1([0-5])$" EventCode="^2([0-5])$"

Splunk Enterprise ignores the first set and only attempts to include events that match the second set. In this case, only events that contain an EventCode between 20 and 25 match. Events that contain an EventCode between 10 and 15 do not match. Only the last set in the entry ever matches.

To resolve this problem, specify two separate entries in the stanza:

whitelist = EventCode="^1([0-5])$"
whitelist1 = EventCode="^2([0-5])$"

Resolve Active Directory objects in event log files

If you want to specify whether or not Active Directory objects like globally unique identifiers (GUIDs) and security identifiers (SIDs) are resolved for a given Windows event log channel, you can use the evt_resolve_ad_obj attribute (1=enabled, 0=disabled) for that channel's stanza in your local copy of inputs.conf. The evt_resolve_ad_obj attribute is on by default for the Security channel.

For example:

[WinEventLog://Security]
disabled = 0
start_from = oldest
current_only = 0
evt_resolve_ad_obj = 1
checkpointInterval = 5

To specify a domain controller for the domain that Splunk should bind to in order to resolve AD objects, use the evt_dc_name attribute.

The string specified in the evt_dc_name attribute can represent either the domain controller's NetBIOS name, or its fully-qualified domain name (FQDN). Either name type can, optionally, be preceded by two backslash characters.

The following examples are correctly formatted domain controller names:

  • FTW-DC-01
  • \\FTW-DC-01
  • FTW-DC-01.splunk.com
  • \\FTW-DC-01.splunk.com

To specify the FQDN of the domain to bind to, use the evt_dns_name attribute.

For example:

[WinEventLog://Security]
disabled = 0
start_from = oldest
current_only = 0
evt_resolve_ad_obj = 1
evt_dc_name = ftw-dc-01.splunk.com
evt_dns_name = splunk.com
checkpointInterval = 5

Constraints

There are some things you must understand when using the evt_dc_resolve_obj attribute:

  • When you specify this attribute, Splunk first attempts to resolve SIDs and GUIDs using the domain controller (DC) specified in the evt_dc_name attribute first. If it cannot resolve SIDs using this DC, it attempts to bind to the default DC to perform the translation.
  • If Splunk cannot contact a DC to translate SIDs, it then attempts to use the local machine for translation.
  • If none of these methods works, then Splunk prints the SID as it was captured in the event.
  • Splunk cannot translate SIDs that are not in the format S-1-N-NN-NNNNNNNNNN-NNNNNNNNNN-NNNNNNNNNN-NNNN.
  • If you discover that Splunk is not translating SIDs properly, review splunkd.log for clues on what the problem might be.

Specify whether to index starting at earliest or most recent event

Use the start_from attribute to specify whether Splunk Enterprise indexes events starting at the earliest event or the most recent. By default, Splunk starts with the oldest data and indexes forward. You can change this by setting this attribute to newest, telling Splunk to start with the newest data, and index backward. We don't recommend changing this setting, as Splunk stops indexing after it has indexed the backlog using this method.

Use the current_only attribute to specify whether or not you want Splunk to index all preexisting events in a given log channel. When set to 1, Splunk indexes only new events that appear from the moment Splunk was started. When set to 0, Splunk indexes all events.

For example:

[WinEventLog://Application]
disabled = 0
start_from = oldest
current_only = 1

Index exported event log (.evt or .evtx) files

To index exported Windows event log files, use the instructions for monitoring files and directories to monitor the directory that contains the exported files.

Constraints

  • As a result of API and log channel processing constraints on Windows XP and Server 2003 systems, imported .evt files from those systems do not contain the "Message" field. This means that the contents of the "Message" field do not appear in your Splunk index.
  • Splunk running on Windows XP and Windows Server 2003/2003 R2 cannot index .evtx files exported from systems running Windows Vista and later or Windows Server 2008/2008 R2 and later.
  • Splunk running on Windows Vista and later and Server 2008/2008 R2 and later can index both .evt and .evtx files.
  • If your .evt or .evtx file is not from a standard event log channel, you must make sure that any dynamic link library (DLL) files required by that channel are present on the computer on which you are indexing.
  • Splunk indexes an .evt or .evtx file in the primary locale/language of the computer that collects the file.

Caution: Do not attempt to monitor a .evt or .evtx file that is currently being written to; Windows does not allow read access to these files. Use the event log monitoring feature instead.

Note: When producing .evt or .evtx files on one system, and monitoring them on another, it's possible that not all of the fields in each event expand as they would on the system producing the events. This is caused by variations in DLL versions, availability and APIs. Differences in OS version, language, Service Pack level and installed third party DLLs, etc. can also have this effect.

Answers

Have questions? Visit Splunk Answers and see what questions and answers the Splunk community has around Windows event logs.

PREVIOUS
Monitor Active Directory
  NEXT
Monitor file system changes on Windows

This documentation applies to the following versions of Splunk® Enterprise: 6.1, 6.1.1, 6.1.2, 6.1.3, 6.1.4, 6.1.5, 6.1.6, 6.1.7, 6.1.8, 6.1.9, 6.1.10, 6.1.11, 6.1.12, 6.1.13, 6.1.14


Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters