Splunk® App for VMware (Legacy)

Installation Guide

Acrobat logo Download manual as PDF


On August 31, 2022, the Splunk App for VMware will reach its end of life. After this date, Splunk will no longer maintain or develop this product. The functionality in this app is migrating to a content pack in Data Integrations. Learn about the Content Pack for VMware Dashboards and Reports.
This documentation does not apply to the most recent version of Splunk® App for VMware (Legacy). For documentation on the most recent version, go to the latest release.
Acrobat logo Download topic as PDF

Collect log data from ESXi hosts

You do not need to collect log data from ESXi hosts in order to see a working version of Splunk App for VMware.


About ESXi server logs

ESXi server logs let you troubleshoot events and host issues.

Splunk App for VMware accepts ESXi log data using syslogs from the following sources.

  • A Splunk Enterprise intermediate forwarder as the data collection point, which can be the data collection node OVA. When you use the intermediate forwarder to collect ESXi logs, Splunk Enterprise is the default log repository.
  • A syslog server with a Splunk Enterprise forwarder monitoring logs.

The VMware environment supports the following ports for syslog data collection.

  • TCP port 1514. Not supported on VMware vSphere 4.1.
  • UDP port 514. Requires Splunk Enterprise root privileges.


Configure Splunk App for VMware to receive syslog data

Prerequisites: Verifications

  • To configure ESXi log data collection, identify the machine to use as your data collection point. Verify that the ESXi hosts can forward data to that data collection point.
  • For the first installation, use an intermediate forwarder as your data collection point. Configure hosts to forward syslog data to the intermediate forwarder.

Step 1: Install a Splunk Universal Forwarder on your syslog server

  1. Download the Splunk Universal Forwarder from Download Splunk Universal Forwarder page. Select the forwarder version and the OS version that you need.
  2. See "Deployment overview" in Forwarding Data to install the universal forwarder.

Step 2: Create an inputs.conf file

Create an inputs.conf file in the system/local folder to monitor the ESXi hosts log files on the syslog server. Set the index and the source type before sending it to the intermediate forwarder.

1. For each monitor stanza in the inputs.conf file, specify the following settings:

The entry in the monitor stanza of the inputs.conf file is: 

[monitor:///var/log/.../syslog.log]
disabled = false
index = vmware-esxilog
sourcetype = vmw-syslog

2. Configure forwarding on your syslog server in outputs.conf to send data to your indexer or intermediate forwarder, which is the Splunk Enterprise instance on which Splunk_TA_esxilogs is installed. For more information about setting up forwarding for your indexers, see Configure forwarders with outputs.conf in Forwarding Data.

Step 3: Install and configure Splunk_TA_esxilogs

Install and configure Splunk_TA_esxilogs on the machine that receives log data from your syslog server.

Install Splunk_TA_esxilogs under $SPLUNK_HOME/etc/apps. This technology add-on is included in Splunk App for VMware. It collects syslog data from the ESXi hosts and maps the data into the dashboards in Splunk App for VMware.

Step 4: Configure Splunk_TA_esxilogs

1. Assign the host field (on the machine where Splunk_TA_esxilogs is installed). The Splunk App for VMware can not determine the originating host for the data when you use a syslog server as your data store and you forward that data to the Splunk indexer.

2. Optionally create an index time extraction that takes the actual host name from the event that passes through, so that the log files can be associated with the correct host. By default, the host name is that of the syslog server. This step is not required when you use an intermediate forwarder, as the Splunk App for VMware automatically assigns the host based on the original data source.

3. Assign the host field. Create a local version of props.conf and transforms.conf in the $SPLUNK_HOME/etc/apps/Splunk_TA_esxilogs/local/ directory and add the regular expressions to extract the host field.

In this example regular expression extraction in props.conf calls the set_host stanza of transforms.conf where the regular expression extraction extracts the host. The source and sourcetype fields are extracted by the settings in the props.conf and transforms.conf files in $SPLUNK_HOME/etc/apps/Splunk_TA_esxilogs/default. Do not override these fields in the local versions of these files.

Here's an example of the entry for props.conf: 

[vmw-syslog]
……
TRANSFORMS-vmsysloghost = set_host

Here's the example for transforms.conf 

[set_host]
REGEX = ^(?:\w{3}\s+\d+\s+[\d\:]{8}\s+([^ ]+)\s+)
DEST_KEY = MetaData:Host
FORMAT = host::$1

4. If the sourcetype is not correct, check the regular expressions in the stanzas [set_syslog_sourcetype] and [set_syslog_sourcetype_4x] in Splunk_TA_esxilogs/default/transforms.conf.

The following is an example of an entry in transforms.conf: 

[set_syslog_sourcetype]
REGEX = ^(?:\w{3}\s+\d+\s+[\d\:]{8}\s+[^ ]+\s+)?([A-Za-z\-]+)(?:[^:]*)
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::vmware:esxlog:$1

Where:

  • ^(?:\w{3}\s+\d+\s+[\d\:]{8}\s+[^ ]+\s+)? is used to extract the datetime field and host field
  • ([A-Za-z\-]+) is used to extract the sourcetype
  • (?:[^:]*) defines the limit. sourcetype is followed by : or [

Troubleshoot Splunk_TA_esxilogs

  • If the time is not extracted from the events, for example, Mar 26 19:00:20 esx1.abc.com Hostd:…, you can modify $SPLUNK_HOME/etc/apps/Splunk_TA_esxilogs/default/syslog_datetime.xml or you can use splunk datetime.xml and change the entry for DATETIME_CONFIG to /etc/datetime.xml in /local/props.conf.
  • If you use VMware vSphere ESX 4.x, remove the comment tags from the following stanzas in transforms.conf on the search head. This ensures that datetime extraction is the same in all regular expressions. These stanzas are only used during search time extraction.
[esx_hostd_fields_4x]
[esx_vmkernel_fields_4x]
[esx_generic_fields_4x]
  • If the correct fields do not display in the ESXi Log Browser, modify the regular expressions in the [esx_hostd_fields], [esx_vmkernel_fields], and [esx_generic_fields] stanzas.

The following example is from syslog_datetime.xml. 

[esx_hostd_fields]
REGEX = (?:^<(\d+)>)?<REPLACE WITH REGEX FOR SOURCETYPE EXTRACTION>: \[([^\s]+) (\w+) '([^']+)'(?: opID=([^\]]+))?\] ?(.*)
FORMAT = Pri::$1 Application::$2 Offset::$3 Level::$4 Object::$5 opID::$6 Message::$7
[esx_vmkernel_fields]
REGEX = (?:^<(\d+)>)?<REPLACE WITH REGEX FOR DATE TIME AND HOST FIELD EXTRACTION>:(vmkernel|vmkwarning):\s+(?:([\d\:\.]+)\s+)?(cpu\d+):(?:(\d+)\))?(?:\[([\:\w]+)\]\s+)?(.*)
FORMAT = Pri::$1 Type::$2 HostUpTime::$3 Cpu::$4 WorldId::$5 SubComp::$6 Message::$7
[esx_generic_fields]
REGEX = (?:^<(\d+)>)?<REPLACE WITH REGEX FOR SOURCETYPE EXTRACTION>:?\s*(.*)$
FORMAT = Pri::$1 Application::$2 Message::$3

Use an intermediate forwarder to configure Splunk to receive syslog data

Step 1: Set up your forwarder

1. Install Splunk Enterprise 6.0.x configured as a heavy forwarder or light forwarder on a machine identified as the intermediate forwarder. If Splunk Enterprise is installed as the heavy forwarder, index time extraction happens on this intermediate forwarder. This forwarder can be the data collection node OVA. We recommend a ratio of one intermediate forwarder to 100 ESXi hosts.

2. Set up forwarding to the port on which the Splunk indexers are configured to receive data. See "Set up forwarding" in Distributed Deployment.

4. Install the Splunk_forwarder_for_vmware package. Get the file splunk_forwarder_for_vmware-<version>-<build_number>.zip from the download package and add it to $SPLUNK_HOME.

5. Unzip the file and make sure that Splunk_TA_esxilogs is in the SPLUNK_HOME/etc/apps/ directory. Use UDP port 514. As the Splunk user on the intermediate forwarder, you must have root privileges to configure data inputs. If you do not have the required privileges, use TCP port 1514.

Step 2: Enable the ports to receive syslog data

Enable ports in Splunk Web using Settings or by modifying the inputs.conf file. In this example using Splunk Web, the TCP port is 1514.

1. Select Settings > Data Inputs.

2. Add TCP port 1514.

3. In the Setup screen enter the following information:

  • TCP port: 1514
  • Accept conditions from all hosts: yes
  • Set sourcetype: Manual
  • Source type: vmw-syslog

4. Select More Settings and enter the following information:

  • Set host: DNS
  • Set the destination index for the source: vmware-esxilog. This setting is the destination of the syslog data. Set the destination index for the source after you have installed the Splunk App for VMware components.

If you do not have access to Splunk Web, create an inputs.conf file in $SPLUNK_HOME/etc/apps/Splunk_TA_esxilogs/local/ and add the following:

[tcp://1514]
disabled = 0

Configure ESXi hosts to send data

Configure the ESXi hosts to forward log data to your syslog server or intermediate forwarders. Enable syslog data collection on the firewall on each host from which you want to collect syslog data.

Configure ESXi hosts using the vSphere Client

1. Select a host on the Hierarchy selector.

2. Click the Configuration tab.

3. In the Software section, click Advanced Settings.

4. In Advanced Settings, scroll down and select Syslog.

5. Change the setting Syslog.global.loghost to the machine receiving the data. For example, enter tcp://yourmachine.yourdomain:1514. vSphere version 4.1 forwards only to tcp. In this case, do not specify tcp://. ESXi hosts forward to UDP port 514 or TCP port 1514 by default. To forward to UDP port 514, make sure that the receiving machine is set up to do so. To forward to a different port, create a new outbound firewall rule as another Security Profile on the sending host.

6. Click OK.

7. In Software, click Security Profile.

8. In Firewall, click Properties.

9. In Firewall Properties Remote Access, select Syslog.

10. Click Firewall.

11. Select Allow connections from any IP address or specify the connections.

12. Click OK.

Set up a host profile

The VMware ESXi and vCenter Server documentation describes how to set up syslog from a host profile.

Configure all hosts remotely

Splunk App for VMware can configure hosts remotely when you use an intermediate forwarder to collect syslog data. See Configure data collection.

Last modified on 22 June, 2016
 

This documentation applies to the following versions of Splunk® App for VMware (Legacy): 3.1.1, 3.1.2, 3.1.3, 3.1.4, 3.2.0, 3.2.1, 3.2.2

Was this documentation topic helpful?


You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters