Collect VMware vCenter Server and ESXi host log data with Splunk App for Infrastructure

Configure your vCenter Server and ESXi hosts to send log data to indexers in your Splunk Enterprise deployment. Here are steps to use VMware vSphere Syslog Agent to send logs from vCenter Server Appliance and a universal forwarder to send logs from a Windows vCenter Server.

You can collect vCenter Server log data from a Windows vCenter Server and from vCenter Server Appliance. If you want to collect log data from a Windows vCenter Server, you have to install a Splunk universal forwarder on the vCenter Server. If you want to collect log data from a vCenter Server Appliance, use the VMware vSphere Syslog Agent.

Collect Windows vCenter Server logs

Configure a Splunk universal forwarder with Splunk_TA_vcenter to collect vCenter Server logs. You can run the universal forwarder on the same system that runs the DCN.

Follow these steps to collect logs from a Windows vCenter Server.

1. Download the Splunk ITSI version 4.4 package on Splunkbase and extract Splunk_TA_vcenter.
2. Install a Splunk universal forwarder. To download a universal forwarder, see download page. For information about installing a universal forwarder, see Install a Windows universal forwarder from an installer.
3. Configure the forwarder on your vCenter Server systems to send data to the indexer tier in your deployment. For information about configuring forwarding, see Configure forwarding with outputs.conf in the Splunk Universal Forwarder Forwarder Manual.
4. Change the default password for the universal forwarder. To change the default password, see Changing the admin default password in the Splunk Enterprise Admin Manual.
5. Copy the Splunk_TA_vcenter directory to the $SPLUNK_HOME\etc\apps directory.
6. Go to $SPLUNK_HOME\etc\apps\Splunk_TA_vcenter\local.
7. If an inputs.conf file doesn't exist yet, create the file and open it.
8. Add these stanzas to the file:

   [monitor://$ALLUSERSPROFILE\VMware\vCenterServer\logs\vws]
   disabled = 0
   index = vmware-vclog
   
   [monitor://$ALLUSERSPROFILE\VMware\vCenterServer\logs\vmware-vpx]
   blacklist = (.*(gz)$)|(\\drmdump\\.*)
   disabled = 0
   index = vmware-vclog
   
   [monitor://$ALLUSERSPROFILE\VMware\vCenterServer\logs\perfcharts]
   disabled = 0
   index = vmware-vclog
   

9. Restart splunkd:

   $SPLUNK_HOME\bin\splunk restart
   

Collect vCenter Server Appliance logs

SAI uses rsyslog to collect vCenter Server Appliance logs. These are the log types SAI collects:

• vpxd
• vpxd-profiler
• vpxd-alert

By default, use TCP port 1517 to send rsyslog data to SAI. Enable rsyslog in your vCenter Server Appliance and send data to your indexer tier. If you have a distributed indexer tier, forward rsyslog data to the Data Collection Node (DCN) so that the Splunk Enterprise instance on the DCN can load balance to your indexers properly.

Follow these steps to set up log collection for a vCenter Server Appliance.

1. On the system running vCenter Server Appliance, go to /etc/.
2. Open the rsyslog.conf file.
3. Find this line and enter the IP address or hostname of your indexer. If you have a distributed indexer deployment, enter the IP address or hostname of the DCN collecting metrics, tasks, inventory, and event data.

    *.* @@<IP/HOSTNAME>:1517;vclogtemplate
   

4. After you save your changes and close the file, restart rsyslog:

   $ service syslog restart
   

5. Go to $SPLUNK_HOME/etc/apps/Splunk_TA_vcenter/default/inputs.conf and copy this stanza:

   #[tcp://1517]
   #connection_host = dns
   #index = vmware-vclog
   #sourcetype = vclog
   #disabled = 0
   

6. Go to Splunk/etc/apps/Splunk_TA_vcenter/local.
7. If an inputs.conf file doesn't exist yet, create the file and open it.
8. Paste the stanza from $SPLUNK_HOME/etc/apps/Splunk_TA_vcenter/default/inputs.conf into the local inputs.conf file.
9. Enable the stanza by uncommenting it. Remove the # from the beginning of each line in the stanza.
10. Restart splunkd:

    $SPLUNK_HOME/bin/splunk restart
    

These are the parameters in each stanza in rsyslog.conf:

File properties	Description
$InputFileName	Used to monitor specific files.
$InputFileTag	Used to set the prefix in each event data. Set $InputFileTag so your Splunk platform deployment can recognize sourcetype of different logs.
$InputFileStateFile	Used to keep track of which parts of the monitored file are already processed. Must be unique.
$InputFileSeverity	Used to set the type of log the user wants.
$InputRunFileMonitor	Used to activate the monitoring.

Here is an example rsyslog.conf file:

$template vclogtemplate,"%syslogtag% %rawmsg%"

$ModLoad imfile
$InputFileName /var/log/vmware/vpxd/vpxd.log
$InputFileTag vpxd
$InputFileStateFile state-vpxd
$InputFileSeverity all
$InputRunFileMonitor

$ModLoad imfile
$InputFileName /var/log/vmware/vpxd/vpxd-profiler.log
$InputFileTag vpxd-profiler
$InputFileStateFile state-vpxd-profiler
$InputFileSeverity all
$InputRunFileMonitor

$ModLoad imfile
$InputFileName /var/log/vmware/vpxd/vpxd-alert.log
$InputFileTag vpxd-alert
$InputFileStateFile state-vpxd-alert
$InputFileSeverity all
$InputRunFileMonitor

$ModLoad imfile
$InputFileName /var/log/vmware/vws/watchdog-vws/watchdog-vws-syslog.log
$InputFileTag vws
$InputFileStateFile state-vws
$InputFileSeverity all
$InputRunFileMonitor

$ModLoad imfile
$InputFileName /var/log/vmware/perfcharts/stats.log
$InputFileTag stats
$InputFileStateFile state-stats
$InputFileSeverity all
$InputRunFileMonitor

 *.* @@<IP/HOSTNAME>:1517;vclogtemplate

Collect ESXi host logs

Configure an ESXi host to forward log data to the DCN that's collecting metrics, task, inventory, and event data from the vCenter Server the host is running in.

To collect log data from an ESXi host, configure syslog on the ESXi host. By default, EXSi hosts send syslog data over ports UDP 514 or TCP 1514. For more information about receiving data from UDP or TCP ports, see Get data from TCP and UDP ports in the Splunk Enterprise Getting Data In guide.

For information about sending log data with syslog, see Configure Syslog on ESXi Hosts on the VMware Docs website. For Syslog.global.LogHost, specify the DCN or other server you want to send syslog data to.