Getting Data In

 


Monitor Windows event log data

NOTE - Splunk version 4.x reached its End of Life on October 1, 2013. Please see the migration information.

This documentation does not apply to the most recent version of Splunk. Click here for the latest version.

Monitor Windows event log data

Windows generates log data during the course of its operation. The Windows Event Log service handles nearly all of this communication. It gathers log data published by installed applications, services and system processes and places them into event log channels - intermediate locations that eventually get written to an event log file. Programs such as Microsoft's Event Viewer subscribe to these log channels to display events that have occurred on the system.

Splunk also supports the monitoring of Windows event log channels. It can monitor event log channels and files stored on the local machine, and it can collect logs from remote machines.

Splunk's event log monitor runs as an input processor within the splunkd service. It runs once for every event log input defined in Splunk.

Why monitor event logs?

Windows event logs are the core metric of Windows server operations - if there's a problem with your Windows system, the Event Log service likely knows about it. Splunk's indexing, searching and reporting capabilities make your logs accessible - sometimes more accessible than Event Viewer.

What's required to monitor event logs?

Activity: Required permissions:
Monitor local event logs * Splunk must run on Windows
* Splunk must run as the Local System user to read all local event logs
Monitor remote event logs * Splunk must run on Windows
AND
* Splunk must run on a universal forwarder that is installed on the server you wish to collect event logs from
OR
* Splunk must run as a domain or remote user with read access to WMI on the target server
* The user Splunk runs as must have read access to the desired event logs

Security and remote access considerations

Splunk collects event log data from remote machines using either WMI or a forwarder. Splunk recommends using a universal forwarder to send event log data from remote machines to an indexer. Review "Introducing the universal forwarder" in the Distributed Deployment Manual for information about how to install, configure and use the forwarder to collect event log data.

If you choose to install forwarders on your remote machines to collect event log data, then you can install the forwarder as the Local System user on these machines. The Local System user has access to all data on the local machine, but not on remote machines.

If you want Splunk to use WMI to get event log data from remote machines, then you must ensure that your network and Splunk instances are properly configured. You cannot install Splunk as the Local System user, and the user you install with determines the set of performance metrics Splunk will see. Review "Security and remote access considerations" in the "Monitor WMI-based data" topic in this manual for additional information on the requirements you must satisfy in order for Splunk to collect remote data properly using WMI.

By default, access to some Windows event logs is restricted, depending on which version of Windows you're running. In particular, the Security event logs by default can only be read by members of the local Administrators or global Domain Admins groups.

Collect event logs from a remote Windows machine

If you want Splunk to collect event logs from a remote machine, you have two choices:

  • Collect the logs remotely using WMI. When you select "Remote event log collections" in Splunk Web, you are using this option.
  • Install a universal forwarder on the machine from which you want to collect logs.

If you choose to collect event logs using WMI, you must install Splunk with an Active Directory domain user. Refer to "Considerations for deciding how to monitor remote Windows data" for additional information on collecting data from remote Windows machines. If the selected domain user is not a member of the Administrators or Domain Admins groups, then you must configure event log security to give the domain user access to the event logs.

To change event log security for access to the event logs from remote machines, you must:

For instructions on how to configure event log security permissions on Windows XP and Windows Server 2003/2003 R2, review this Microsoft Knowledge Base article. If you're running Windows Vista, Windows 7 or Windows Server 2008/2008 R2, use the wevtutil utility to set event log security.

Anomalous host names visible in event logs on some systems

On Windows Vista and Server 2008 systems, you might see some event logs with randomly-generated host names. This is the result of those systems logging events before the user has named the system, during the OS installation process.

This anomaly only occurs when collecting logs from the above-mentioned versions of Windows remotely over WMI.

Use Splunk Web to configure event log monitoring

Configure local event log monitoring

1. Click Manager in the upper right-hand corner of Splunk Web.

2. Under Data, click Data Inputs.

3. Click Local event log collections.

4. Click Add new to add an input.

5. Select one or more logs from the list of Available Logs and click to add to the list of Selected Logs.

Note 1: Select up to 63 logs from the list of Available Logs. Selecting more than 63 can cause Splunk to become unstable.

Note 2: Certain Windows Event Log channels (known as direct channels) do not allow for users to access - or subscribe to - them in order to monitor them. This is because events sent via these log channels are not actually processed by the Windows Event Log framework, and thus can't be forwarded or collected remotely. Often, these direct channels are logged directly to disk. Attempts to monitor these log channels will generate the error: "The caller is trying to subscribe to a direct channel which is not allowed."

6. Click Save.

Splunk adds and enables the input.

Configure remote event log monitoring

1. Click Manager in the upper right-hand corner of Splunk Web.

2. Under Data, click Data Inputs.

3. Click Remote event log collections.

4. Click Add new to add an input.

5. Enter a unique name for this collection.

6. Specify a hostname or IP address for the host from which to pull logs, and click Find logs... to get a list of logs from which to choose.

Note: Windows Vista offers many event log channels in addition to the standard set of channels defined in all versions of Windows. Depending on the CPU available to Splunk, selecting all or a large number of them can result in high load.

7. Optionally, provide a comma-separated list of additional servers from which to pull data.

8. Click Save.

Splunk adds and enables the input.

Use inputs.conf to configure event log monitoring

You can edit inputs.conf to configure event log monitoring. For more information on configuring data inputs with inputs.conf, read "Configure your inputs" in this manual.

Note: You can always review the defaults for a configuration file by looking at the examples in %SPLUNK_HOME%\etc\system\default or at the spec file in the Admin Manual.

To enable event log inputs by editing inputs.conf:

1. Create an inputs.conf in %SPLUNK_HOME%\etc\system\local and open it for editing.

2. Open %SPLUNK_HOME%\etc\system\default\inputs.conf and review it for the Windows event log inputs you want to enable.

3. Copy the Windows event log input stanzas you want to enable from %SPLUNK_HOME\etc\system\default\inputs.conf.

4. Paste the stanzas you copied into %SPLUNK_HOME%\etc\system\local\inputs.conf.

5. Make edits to the stanzas to collect the Windows event log data you desire.

6. Save %SPLUNK_HOME%\etc\system\local\inputs.conf and close it.

7. Restart Splunk.

The next section describes the specific configuration values for event log monitoring.

Event log monitor configuration values

Windows event log (*.evt) files are in binary format. They can't be monitored like a normal text file. The splunkd service monitors these binary files by using the appropriate APIs to read and index the data within the files.

Splunk uses the following stanzas in inputs.conf to monitor the default Windows event logs:

# Windows platform specific input processor.
[WinEventLog:Application]
disabled = 0 
[WinEventLog:Security]
disabled = 0 
[WinEventLog:System]
disabled = 0 

You can also configure Splunk to monitor non-default Windows event logs. Before you can do this, you must import them to the Windows Event Viewer. Once the logs are imported, you can add them to your local copy of inputs.conf, as follows:

[WinEventLog:DNS Server]
disabled = 0
[WinEventLog:Directory Service]
disabled = 0
[WinEventLog:File Replication Service]
disabled = 0

Note: Use the log properties' "Full Name:" to index. For example, to monitor Task Scheduler in Microsoft> Windows > TaskScheduler >Operational, right click on Operational and select properties. Use the "Full Name" to append to WinEventLog: stanza:

[WinEventLog:Microsoft-Windows-TaskScheduler/Operational]
disabled = 0

To disable indexing for an event log, add disabled = 1 below its listing in the stanza in %SPLUNK_HOME%\etc\system\local\inputs.conf.

Resolve Active Directory objects in event log files

If you want to specify whether or not Active Directory objects like globally unique identifiers (GUIDs) and security identifiers (SIDs) are resolved for a given Windows event log channel, you can use the evt_resolve_ad_obj attribute (1=enabled, 0=disabled) for that channel's stanza in your local copy of inputs.conf. The evt_resolve_ad_obj attribute is on by default for the Security channel.

For example:

[WinEventLog:Security]
disabled = 0
start_from = oldest
current_only = 0
evt_resolve_ad_obj = 1
checkpointInterval = 5

To specify a domain controller for the domain that Splunk should bind to in order to resolve AD objects, use the evt_dc_name attribute.

The string specified in the evt_dc_name attribute can represent either the domain controller's NetBIOS name, or its fully-qualified domain name (FQDN). Either name type can, optionally, be preceded by two backslash characters.

The following examples are correctly formatted domain controller names:

  • FTW-DC-01
  • \\FTW-DC-01
  • FTW-DC-01.splunk.com
  • \\FTW-DC-01.splunk.com

To specify the FQDN of the domain to bind to, use the evt_dns_name attribute.

For example:

[WinEventLog:Security]
disabled = 0
start_from = oldest
current_only = 0
evt_resolve_ad_obj = 1
evt_dc_name = ftw-dc-01.splunk.com
evt_dns_name = splunk.com
checkpointInterval = 5

Constraints

There are some things you must understand when using the evt_dc_resolve_obj attribute:

  • When you specify this attribute, Splunk first attempts to resolve SIDs and GUIDs using the domain controller (DC) specified in the attribute first. If it cannot resolve SIDs using this DC, it attempts to bind to the default DC to perform the translation.
  • If Splunk cannot contact a DC to translate SIDs, it then attempts to use the local machine for translation.
  • If none of these methods works, then Splunk prints the SID as it was captured in the event.
  • Splunk cannot translate SIDs that are not in the format S-1-N-NN-NNNNNNNNNN-NNNNNNNNNN-NNNNNNNNNN-NNNN.
  • If you discover that Splunk is not translating SIDs properly, review splunkd.log on your indexer for clues on what the problem might be.

Specify whether to index starting at earliest or most recent event

Use the start_from attribute to specify whether Splunk indexes events starting at the earliest event or the most recent. By default, Splunk starts with the oldest data and indexes forward. You can change this by setting this attribute to newest, telling Splunk to start with the newest data, and index backward. We don't recommend changing this setting, as it results in a highly inefficient indexing process.

Use the current_only attribute to specify whether or not you want Splunk to index all preexisting events in a given log channel. When set to 1, Splunk indexes only new events that appear from the moment Splunk was started. When set to 0, Splunk indexes all events.

For example:

[WinEventLog:Application]
disabled = 0
start_from = oldest
current_only = 1

Index exported event log (.evt or .evtx) files

To index exported Windows event log files, use the instructions for monitoring files and directories to monitor the directory that contains the exported files.

Constraints

  • As a result of API and log channel processing constraints on Windows XP and Server 2003 systems, imported .evt files from those systems will not contain the "Message" field. This means that the contents of the "Message" field will not appear in your Splunk index.
  • Splunk running on Windows XP and Windows Server 2003/2003 R2 cannot index .evtx files exported from systems running Windows Vista, 7 or Server 2008/2008 R2.
  • Splunk running on Windows Vista, 7, and Server 2008/2008 R2 can index both .evt and .evtx files.
  • If your .evt or .evtx file is not from a standard event log channel, you must make sure that any dynamic link library (DLL) files required by that channel are present on the computer on which you are indexing.
  • The language that a .evt or .evtx file will be indexed as is the primary locale/language of the Splunk computer that collects the file.

Caution: Do not attempt to monitor a .evt or .evtx file that is currently being written to; Windows will not allow read access to these files. Use the event log monitoring feature instead.

Note: When producing .evt or .evtx files on one system, and monitoring them on another, it's possible that not all of the fields in each event will expand as they would on the system producing the events. This is caused by variations in DLL versions, availability and APIs. Differences in OS version, language, Service Pack level and installed third party DLLs, etc. can also have this effect.

Answers

Have questions? Visit Splunk Answers and see what questions and answers the Splunk community has around Windows event logs.

This documentation applies to the following versions of Splunk: 4.2 , 4.2.1 , 4.2.2 , 4.2.3 , 4.2.4 , 4.2.5 , 4.3 , 4.3.1 , 4.3.2 , 4.3.3 , 4.3.4 , 4.3.5 , 4.3.6 , 4.3.7 , 5.0 , 5.0.1 , 5.0.2 , 5.0.3 , 5.0.4 , 5.0.5 , 5.0.6 View the Article History for its revisions.


Comments

More information about monitoring remote event log data and performance metrics, including information about wmi.conf, can be found in "Monitor WMI-based data" (http://docs.splunk.com/Documentation/Splunk/latest/Data/MonitorWMIdata) in this manual.

Malmoore, Splunker
October 4, 2012

Remote event log and remote performance monitoring inputs are stored in a separate file called wmi.conf. See $SPLUNK_HOME/etc/system/README/wmi.conf.spec for more information.

LincolnBowser
March 4, 2012

When you configure inputs within Manager, the inputs.conf files get created within the context of the app that is currently loaded at the time. In many cases, that is the 'Search' app, as it is the default app that is loaded when you start Splunk. The configuration files for that app get written to %SPLUNK_HOME%\etc\apps\search\local by default. You can move the files to %SPLUNK_HOME%\etc\system\local at your leisure.

For more information about configuration files, be sure to read "About configuration files" (http://docs.splunk.com/Documentation/Splunk/latest/Admin/aboutconfigurationfiles) in the Admin Manual.

Malmoore, Splunker
November 29, 2011

When you configure remote event log monitoring using the Manager, where are your inputs stored? Mine are not in the inputs.conf file. I would very much like to move management of this configuration information to a text file but for the life of me, I cannot find where this is located.

Tafiedler
November 29, 2011

You must be logged into splunk.com in order to post comments. Log in now.

Was this documentation topic helpful?

If you'd like to hear back from us, please provide your email address:

We'd love to hear what you think about this topic or the documentation as a whole. Feedback you enter here will be delivered to the documentation team.

Feedback submitted, thanks!