Configure Splunk for ESXi logs
To configure ESXi log data collection, identify the machine to use as your data collection point and check that the ESX/i hosts are set up to forward data to that data collection point.
- For a first time install, use an intermediate forwarder as your data collection point and manually configure hosts to forward syslog data to the intermediate forwarder.
- In a production installation, use the host profile to set up forwarding to your intermediate forwarders or syslog server.
Configure Splunk to receive syslog data
The Splunk App for VMware can receive ESXi log data via syslog from:
- a Splunk intermediate forwarder.
- a syslog server with a Splunk forwarder monitoring logs.
Note that VMware vSphere 4.1 only supports syslog data collection using a UDP port. Data collection using a TCP port is not supported.
The Splunk App for VMware supports the following ports for syslog data collection:
- TCP port 1514.
- UDP port 514.
WARNING: When you use an intermediate forwarder to collect ESXi logs, the Splunk App for VMware configures all hosts to send data to the identified intermediate forwarders. This overwrites the existing log host setting and replaces it with the new intermediate forwarder settings. To continue using your syslog server, see Use your syslog server in this topic.
When you use the intermediate forwarder to collect ESXi logs, Splunk Enterprise is your default log repository.
Use your syslog server
Use your syslog server, set up with a Splunk Universal Forwarder, to monitor, collect, and store ESXi log files coming from the ESXi hosts.
To use your syslog server to collect and store ESXi logs:
- Install a Splunk Universal Forwarder on your syslog server to forward the data from your syslog server to the Splunk instance receiving the log data. (This can be an indexer or an intermediate forwarder.)
- Configure forwarding on your syslog server, in the
outputs.conf
file, to send data to your indexer or intermediate forwarder (the Splunk instance on which Splunk_TA_esxilogs is installed). For more information about setting up forwarding for your indexers, see Configure forwarders with outputs.conf in the Forwarding Data Manual. - Install
Splunk_TA_esxilogs
under$SPLUNKHOME/etc/apps
on the machine that will receive log data from your syslog server. - Create an
inputs.conf
file in thesystem/local
folder (on the machine whereSplunk_TA_esxilogs
is installed) to monitor the esxi hosts log files. For each monitor stanza in theinputs.conf
file specify the sourcetype for the data (vmw-syslog) and the index (vmware-esxilog) for the data. See "Configure your inputs" in the Getting Data in manual for more information. - Assign the
host
field (on the machine whereSplunk_TA_esxilogs
is installed). This step is not required when you use an intermediate forwarder, as the Splunk App for VMware can automatically assign the host based on the original data source. The Splunk App for VMware can not determine the originating host for the data when you use a syslog server as your data store and you forward that data to the Splunk indexer. Create a local version ofprops.conf
andtransforms.conf
in the$SPLUNKHOME/etc/apps/Splunk_TA_esxilogs/local/
directory and add the regular expressions to extract the host field. For example, the regular expression extraction inprops.conf
calls theset_host
stanza oftransforms.conf
where the regular expression extraction extracts the host. The source and sourcetype fields are extracted by the settings in theprops.conf
andtransforms.conf
files in$SPLUNKHOME/etc/apps/Splunk_TA_esxilogs/default
. Do not override these fields in the local versions of these files.
Use an intermediate forwarder
To set up forwarding to an intermediate forwarder:
- Install Splunk 5.0.4 on a machine identified as the intermediate forwarder. (This can be the data collection node OVA.) We recommend a ratio of 1 intermediate forwarder to 100 ESXi hosts.
- Set up forwarding to the port on which the Splunk indexer(s) is configured to receive data. See Set up forwarding in the Distributed Deployment manual.
- Install the
Splunk_forwarder_for_vmware
package. Get the filesplunk_forwarder_for_vmware-<version>-<build_number>.zip
from the download package and put it in$SPLUNK_HOME
. - Unzip the file and check that
Splunk_TA_esxilogs
is in theSPLUNK_HOME/etc/apps/
directory. - Use UDP port 514. As the Splunk user on the intermediate forwarder, you must have root privileges to configure data inputs. If you do not have the required privileges, use TCP port 1514.
- Enable the ports to receive syslog data. You can do this in Splunk Web using Manager or Settings or by manually modifying your
inputs.conf
file. In this example that uses Splunk Web, we use TCP port 1514.- In Manager or Settings:
- Go to Data Inputs and add a new TCP port 1514.
- In the Setup screen enter the following information:
- TCP port:
1514
- Accept conditions from all hosts:
yes
- Set sourcetype:
Manual
- Source type:
vmw-syslog
- TCP port:
- Select More Settings and enter the following information:
- Set host:
DNS
- Set the destination index for the source: vmware-esxilog. This is where the syslog data is sent. Do this after you have installed your app components.
- Set host:
- If you do not have access to Splunk Web, create an
inputs.conf
file in$SPLUNKHOME/etc/apps/Splunk_TA_esxilogs/local/
and add the following:
- In Manager or Settings:
[tcp://1514]
disabled = 0
Configure ESXi Hosts to send data
Configure the ESXi hosts to forward log data to your syslog server or intermediate forwarder(s).
To manually configure ESXi hosts in the vSphere client:
- Select a host on the Hierarchy selector.
- Click the Configuration tab on the panel.
- Click Advanced Settings.
- In the modal dialog box, scroll down and select Syslog.
- Change the setting
Syslog.global.loghost
to the machine receiving the data. For example, entertcp://yourmachine.yourdomain:1514
. NOTE: vSphere version 4.1 only forwards to tcp. In this case, do not specifytcp://
. ESXi hosts only forward to UDP port 514 or TCP port 1514. To forward to UDP port 514 check that your receiving machine is set up to do so. - Click Save.
- Click Security Profile.
- Click Properties....
- In the modal dialog box click Firewall.
- Select Allow connections from any IP address or specify the connections allowed.
- Click Save.
Set up a Host profile
The VMware ESXi and vCenter Server documentation describes how to set up syslog from the Host Profile.
- See Set Up Syslog from the Host Profiles Interface in the VMware ESXi and vCenter Server 5 Documentation.
- See Set Up Syslog from the Host Profiles Interface in the vSphere Client 5.1
Configure all hosts remotely
The Splunk App for VMware can configure hosts remotely when you use an intermediate forwarder to collect syslog data. See Configure data collection.
Considerations when using tsidx namespaces | Collect Windows vCenter log data |
This documentation applies to the following versions of Splunk® App for VMware (Legacy): 3.0, 3.0.1, 3.0.2
Feedback submitted, thanks!