Splunk® Supported Add-ons

Splunk Add-on for VMware

Download manual as PDF

Download topic as PDF

Configure the Splunk Add-on for VMware to collect data from vCenter logs

vCenter logs contain information about access to the vCenter environment, audit information (who assigned permissions, added/edited/removed VMs), and health information about vCenter's processes.

For vCSA servers, vCSA's native syslog forwarding is used to pass this information to your Splunk platform. Nothing is installed onto the vCSA servers to collect this data. Windows-based vCenter environments require a Splunk platform forwarder and splunk_TA_vcenter.

Prepare to collect data

Set up a vCenter Server user account

Obtain VMware vCenter Server account credentials for each vCenter Server system.

These credentials allow the Splunk Add-on for VMware read-only API access to the appropriate metrics on each vCenter Server system in the environment. the Splunk App for VMware uses the credentials when the DCN polls vCenter Server systems for performance, hierarchy, inventory, task, and event data. These credentials are required for DCN configuration. You can use existing vCenter Server account credentials, or create a new account for Splunk App for VMware to access the vCenter Server data.

If you encounter issues setting the correct permissions for vCenter Server accounts, see "Permissions in vSphere."

You must have a user account to authenticate with vCenter. Your role determines access privileges. If you use ActiveDirectory for authentication on your Windows OS (vCenter) machines, see Create users in ActiveDirectory in this topic.

If you add a new vCenter Server user as administrator, the user automatically gets an Administrator role in vSphere.

Create a local user on your Windows OS (vCenter) machine

  1. Log into the Windows OS with an administrator account.
  2. In the Start menu, click Control Panel.
  3. In the User Accounts screen, click Add or remove user accounts.
  4. In the Manage Accounts window, click Create a new account.
  5. Enter a name for the account (example: splunksvc).
  6. In vSphere, select Standard user.
  7. Click Create Account.
  8. In the Manage Accounts screen, click on the new user.
  9. In the Change an Account screen, click Create a password and assign the user a password. The new user account displays as a Standard user and the account shows that it is Password protected.
  10. Verify that you now have a local Windows user compatible with the vSphere permissions system.

Create users in Active Directory

For machines that participate in an Active Directory (AD) domain, create a service account in the given domain using the appropriate control panel in Windows Server. Most VMware environments use a single Active Directory domain for authentication. However, if you use multiple AD domains, then create a service account in each domain that your VMware environment uses.

How you create a service account within Active Directory depends on your environment. Contact your AD administrator to learn how to do this for your environment.

Create roles on each vCenter server in your environment

  1. Open the vSphere client and connect to the vCenter server.
  2. Log in with administrative privileges.
  3. Click Home in the path bar.
  4. Under Administration click Roles.
  5. Click Add Role.
  6. In the Add new Role dialog box, enter a name for the role (for example, splunkreader).
  7. Select the appropriate permissions for the role.

For information about collecting data via the VMware API, see Configure the Splunk Add-on for VMware to collect data from vCenter Server systems using the VMware API.

Configure DCNs to honor TLS protocols

You may need to set your DCN's to honor TLS protocols when making requests to the vCenter APIs.

  1. On your DCN, Navigate to $SPLUNK_HOME/etc/system/local, and open web.conf with a text editor. If there is no web.conf create the file.
  2. Add the below stanzas to your web.conf file.
[settings]
sslVersions = tls1.2
cipherSuite = AES256-SHA256

Validate and patch vCenter Server systems, add WSDL files

If you use vCenter Server 5.0 and 5.0.1, apply a patch to manage a known issue with the servers. See known issues in the release notes for details on acquiring and applying the patch.

If you use vSphere 5.0 or 5.0 update 1, be sure to add two missing WSDL files that the app needs to make API calls to vCenter. Access the VMware Knowledge Base for detailed installation instructions. The missing files are:

  • reflect-message.xsd
  • reflect-types.xsd

vCenter Log Collection (Windows vCenter and vCSA)

Collect Windows VMware vCenter Server log data

Use the Splunk Add-on for VMware vCenter to collect vCenter Server log data. Use a Splunk Universal Forwarder to forward the log data from your Windows vCenter Server to the indexer.

  1. Install a Splunk forwarder.
  2. Configure forwarding. Configure the forwarder on your vCenter Server systems to send data to your indexers. Configure the forwarder in the outputs.conf file for each forwarder installed on a vCenter Server system. See Configure forwarding with outputs.conf.
  3. Change your Splunk password.
  4. Install Splunk_TA_vcenter.
    • Get the file Splunk_TA_vcenter-<version>-<build_number>.zip from the download package and install it on your vCenter Server systems.
    • Unzip the file, "Splunk_TA_vcenter-<version>-<build_number>.zip", into the apps directory under %SPLUNK_HOME%\etc\apps. When installing on a universal forwarder the path is C:\Program Files\SplunkUniversalForwarder\etc\apps otherwise it is C:\Program Files\Splunk\etc\apps.
  5. On the system that you've installed installed the Splunk Enterprise forwarder, install Splunk_TA_vCenter.
  6. Copy the inputs.conf file from $SPLUNK_HOME/etc/Splunk_TA_vCenter/default.
  7. Paste $SPLUNK_HOME/etc/Splunk_TA_vCenter/default into the $SPLUNK_HOME/etc/Splunk_TA_vCenter/local folder.
  8. Open the inputs.conf file.
  9. Change the log path to the location of the vCenter Server Appliance logs data, C:\ProgramData\VMware\vCenterServer\logs. Edit the following stanzas in the inputs.conf file:
  10. Windows vCenter server 6.x

    [monitor://$ALLUSERSPROFILE\VMware\vCenterServer\logs\vws]
    disabled = 0
    index = vmware-vclog
    
    [monitor://$ALLUSERSPROFILE\VMware\vCenterServer\logs\vmware-vpx]
    blacklist = (.*(gz)$)|(\\drmdump\\.*)
    disabled = 0
    index = vmware-vclog
    
    [monitor://$ALLUSERSPROFILE\VMware\vCenterServer\logs\perfcharts]
    disabled = 0
    index = vmware-vclog
    
  11. (Optional) If you configured Splunk Enterprise as a heavy or light forwarder, and you want to monitor the license file and and tomcat configuration files, edit the following stanzas in the props.conf file.
    • Copy the $SPLUNK_HOME/etc/Splunk_TA_vCenter/default/props.conf file.
    • Paste $SPLUNK_HOME/etc/Splunk_TA_vCenter/default/props.conf into the $SPLUNK_HOME/etc/Splunk_TA_vCenter/local folder.
    • Open the local props.conf file.
    • Change the log path to that in which the vCenter Server Appliance logs data. Adjust the following stanzas:

    Windows vCenter server

    [source::(?-i)...\\VMware\\vCenterServer\\logs\\cim-diag.log(?:.\d+)?]
    [source::(?-i)...\\VMware\\vCenterServer\\logs\\sms.log(?:.\d+)?]
    [source::(?-i)...\\VMware\\vCenterServer\\logs\\stats.log(?:.\d+)?]
    [source::(?-i)...\\VMware\\vCenterServer\\logs\\vim-tomcat-shared.log(?:.\d+)?]
    [source::(?-i)...\\VMware\\vCenterServer\\logs\\vpxd-\d+.log(?:.\d+)?]
    [source::(?-i)...\\VMware\\vCenterServer\\logs\\vpxd-alert-\d+.log(?:.\d+)?]
    [source::(?-i)...\\VMware\\vCenterServer\\logs\\vpxd-profiler-\d+.log(?:.\d+)?]
    [source::(?-i)...\\VMware\\vCenterServer\\logs\\vws.log(?:.\d+)?]
    [source::(?-i)...\\VMware\\vCenterServer\\logs\\vpxd.cfg]
    
    • Change the licenses path to the vCenter Server Appliance licenses path:
    [source::(?-i)...\\VMware\\vCenterServer\\licenses]
    
    • Change the tomcat conf path to the vCenter Server Appliance tomcat conf path:
    [source::(?-i)...\\VMware\\Infrastructure\\tomcat\\conf] 
    • Change the path to the vCenter Server Appliance path:
    [source::...\\Application Data\\VMware\\…]
    [source::...\\VMware\\Infrastructure\\…]
    
  12. Restart Splunk Enterprise. See "Start and stop Splunk" in the Admin Manual.
  13. In %SPLUNK_HOME%\bin run the command splunk restart. Alternatively, select Start > Administrative Tools > Services > Splunkd restart in Windows services.

The Splunk Add-on for VMware collects log data from your Windows vCenter Server systems and forwards the data from vCenter Server to your Splunk platform indexers or combined indexer search heads.

Collect VMware vCenter Server Appliance (vCSA) log data

Use the Splunk Add-on for VMware to collect logs from the VMware vCenter Server Appliance. the Splunk Add-on for VMware stores VMware vCenter Server Appliance logs in /var/log/vmware.

Export vCenter logs to an external system

  1. Install a Splunk forwarder.
  2. Enable the VMware vCenter Server Appliance to store log files on NFS storage on a system on which you have installed Splunk Enterprise as a heavy forwarder or as a light forwarder. See NFS Storage on the VMware vCenter Server Appliance in the VMware vSphere documentation.
  3. On the system on which you have installed the Splunk Enterprise forwarder, install Splunk_TA_vCenter.
  4. Copy the inputs.conf file from $SPLUNK_HOME/etc/Splunk_TA_vCenter/default then paste it into the $SPLUNK_HOME/etc/Splunk_TA_vCenter/local folder and open file.
  5. Change the log path to the location that the vCenter Server Appliance logs data (/var/log/vmware/). Edit the following stanzas in the inputs.conf file: Linux server appliance 6.x
    [monitor:///var/log/vmware/vws]
    disabled = 0
    index = vmware-vclog
    
    [monitor:///var/log/vmware/vpxd]
    blacklist = (.*(gz)$)|(\\drmdump\\.*)
    disabled = 0
    index = vmware-vclog
    
    [monitor:///var/log/vmware/perfcharts]
    disabled = 0
    index = vmware-vclog
    

    Linux server appliance 5.x (not supported from 3.4.5)

    [monitor:///var/log/vmware/vpx]
    blacklist = (.*(gz)$)|(\\drmdump\\.*)
    disabled = 0
    index = vmware-vclog
    
  6. (Optional) If you configured Splunk Enterprise as a heavy/light forwarder and you want to monitor the license file and tomcat configuration files, edit the following stanzas in the props.conf file:
    • Copy the $SPLUNK_HOME/etc/Splunk_TA_vCenter/default/props.conf file, then past into the $SPLUNK_HOME/etc/Splunk_TA_vCenter/local folder. Open the local props.conf file.
    • Change the log path to that in which the vCenter Server Appliance logs data. Adjust the following stanzas:

    Linux server appliance 6.x
    [source::(?-i).../var/log/vmware/perfcharts/stats.log(?:.\d+)?]
    [source::(?-i).../var/log/vmware/vpxd/vpxd-\d+.log(?:.\d+)?]
    [source::(?-i).../var/log/vmware/vpxd/vpxd-alert-\d+.log(?:.\d+)?]
    [source::(?-i).../var/log/vmware/vpxd/vpxd-profiler-\d+.log(?:.\d+)?]
    

    Linux server appliance 5.x (not supported from 3.4.5)

    [source::(?-i).../var/log/vmware/vpx/stats.log(?:.\d+)?]
    [source::(?-i).../var/log/vmware/vpx/vpxd-\d+.log(?:.\d+)?]
    [source::(?-i).../var/log/vmware/vpx/vpxd-alert-\d+.log(?:.\d+)?]
    [source::(?-i).../var/log/vmware/vpx/vpxd-profiler-\d+.log(?:.\d+)?]
    [source::(?-i).../var/log/vmware/vpx/vws.log(?:.\d+)?]
    
  7. Start Splunk Enterprise.

Forward VMware vCenter Linux appliance logs to Splunk Enterprise

To forward VMware vCenter Linux appliance logs to your Splunk Enterprise indexers or search head, install a Splunk Enterprise forwarder on the VMware vCenter Linux appliance. Access to vCSA shell access must be enabled.

  1. Install a Splunk forwarder on the VMware vCenter Server Appliance.
  2. Install Splunk_TA_vCenter on the Splunk platform forwarder.
    1. Get the Splunk_TA_vcenter-<version>-<build_number>.zip file from the download package and place it on vCenter.
    2. Unzip the Splunk App for VMware package:
      cd /opt/splunkforwarder
      Splunk_TA_vcenter-<version>-<build_number>.zip"
    3. Verify that you successfully extracted the Splunk_TA_vcenter/… in the $SPLUNK_HOME/etc/apps directory.
  3. Copy the inputs.conf file from $SPLUNK_HOME/etc/Splunk_TA_vCenter/default then paste it into the $SPLUNK_HOME/etc/Splunk_TA_vCenter/local folder and open file.
  4. (Optional) If you configured Splunk Enterprise as a heavy forwarder and you want to monitor the license file and and tomcat configuration files, copy the contents of the $SPLUNK_HOME/etc/Splunk_TA_vCenter/default/props.conf file and paste it into the $SPLUNK_HOME/etc/Splunk_TA_vCenter/local folder.
  5. Start the Splunk Universal Forwarder.

Collect vCenter Server Appliance logs via syslog

Syslog type Supported vCSA version Log types
syslog-ng 5.5 vpxd, vpxd-profiler, vpxd-alert
rsyslog 6.x vpxd, vpxd-profiler, vpxd-alert

Syslog-ng on vCenter 5.5

vCenter 5.5 is not supported from VMware 3.4.5

Enable syslog forwarding using syslog-ng for vCSA 5.5 logs.

  1. Open your vCenter deployment, and navigate to /etc/syslog-ng/.
  2. In /etc/syslog-ng/, open the syslog-ng.conf file.
  3. In the syslog-ng.conf file, replace <IP/HOSTNAME> with the IP address of the hostname of the machine where you want to receive the vCSA logs. Example:
    # vpxd source log
    source vclog {
        file("/var/log/vmware/vpx/vpxd.log" follow-freq(60) log-prefix("vpxd ") flags(no-parse));
        file("/var/log/vmware/vpx/vpxd-alert.log" follow-freq(60) log-prefix("vpxd-alert ") flags(no-parse));
        file("/var/log/vmware/vpx/vpxd-profiler.log" follow-freq(60) log-prefix("vpxd-profiler ") flags(no-parse));    
        file("/var/log/vmware/vpx/vws.log" follow-freq(60) log-prefix("vws ") flags(no-parse));
        file("/var/log/vmware/vpx/stats.log" follow-freq(60) log-prefix("stats ") flags(no-parse));
        file("/var/log/vmware/vpx/cim-diag.log" follow-freq(60) log-prefix("cim-diag ") flags(no-parse));
        file("/var/log/vmware/vpx/sms.log" follow-freq(60) log-prefix("sms ") flags(no-parse));
        file("/var/log/vmware/vpx/cim-diag.log" follow-freq(60) log-prefix("cim-diag ") flags(no-parse));
        file("/var/log/vmware/vpx/vmware-vpxd.log" follow-freq(60) log-prefix("vmware-vpxd ") flags(no-parse));
    };
    
    # Remote Syslog Host
    destination remote_syslog {
        tcp("<IP/HOSTNAME>" port(1517) template("${MSG} \n") template-escape(no));
    };
    
    # Log vCenter Server vpxd log remotely
    log {
        	source(vclog);
        	destination(remote_syslog);	 
    };
    
  4. After changing the conf file, restart the syslog service for the changes to take effect. service syslog restart
  5. Navigate to Splunk/etc/apps/Splunk_TA_vcenter/ and create a local folder.
  6. In Splunk/etc/apps/Splunk_TA_vcenter/local, create an inputs.conf file.
  7. Navigate to Splunk/etc/apps/Splunk_TA_vcenter/default/inputs.conf and copy the below stanza.
    #[tcp://1517]
    #connection_host = dns
    #index = vmware-vclog
    #sourcetype = vclog
    #disabled = 0
    
  8. Navigate to Splunk/etc/apps/Splunk_TA_vcenter/local/inputs.conf, and paste the copied stanza into the local version of inputs.conf.
  9. Enable the copied stanza in local/inputs.conf by uncommenting it. Note: Since TCP port 1514 is used for receiving ESXi logs, the 1517 port is used, by default, for vclogs. Other open ports can be used.
File properties Description
follow-freq Used to set the polling interval in seconds.
log-prefix Used to set the prefix in each event data. Set log-prefix so your Splunk platform deployment can recognize sourcetype of different logs.
flags Used to forward the log without any parsing.

For more information on configuration details, see the syslog-ng Open Source Edition Administrator Guide

Rsyslog on vCenter 6.x

Enable syslog forwarding using rsyslog for vCSA 6.x logs.

  1. Open your vCenter deployment, and navigate to /etc/.
  2. In /etc/, open the rsyslog.conf file.
  3. In the rsyslog.conf file, replace <IP/HOSTNAME> with the IP address of the hostname of the machine where you want to receive the vCSA logs. Example:
    $template vclogtemplate,"%syslogtag% %rawmsg%"
    
    $ModLoad imfile
    $InputFileName /var/log/vmware/vpxd/vpxd.log
    $InputFileTag vpxd
    $InputFileStateFile state-vpxd
    $InputFileSeverity all
    $InputRunFileMonitor
    
    $ModLoad imfile
    $InputFileName /var/log/vmware/vpxd/vpxd-profiler.log
    $InputFileTag vpxd-profiler
    $InputFileStateFile state-vpxd-profiler
    $InputFileSeverity all
    $InputRunFileMonitor
    
    $ModLoad imfile
    $InputFileName /var/log/vmware/vpxd/vpxd-alert.log
    $InputFileTag vpxd-alert
    $InputFileStateFile state-vpxd-alert
    $InputFileSeverity all
    $InputRunFileMonitor
    
    $ModLoad imfile
    $InputFileName /var/log/vmware/vws/watchdog-vws/watchdog-vws-syslog.log
    $InputFileTag vws
    $InputFileStateFile state-vws
    $InputFileSeverity all
    $InputRunFileMonitor
    
    $ModLoad imfile
    $InputFileName /var/log/vmware/perfcharts/stats.log
    $InputFileTag stats
    $InputFileStateFile state-stats
    $InputFileSeverity all
    $InputRunFileMonitor
    
     *.* @@<IP/HOSTNAME>:1517;vclogtemplate
    
  4. After changing the conf file, restart the syslog service for the changes to take effect. service syslog restart
  5. Navigate to Splunk/etc/apps/Splunk_TA_vcenter/ and create a local folder.
  6. In Splunk/etc/apps/Splunk_TA_vcenter/local, create an inputs.conf file.
  7. Navigate to Splunk/etc/apps/Splunk_TA_vcenter/default/inputs.conf and copy the below stanza.
    #[tcp://1517]
    #connection_host = dns
    #index = vmware-vclog
    #sourcetype = vclog
    #disabled = 0
    
  8. Navigate to Splunk/etc/apps/Splunk_TA_vcenter/local/inputs.conf, and paste the copied stanza into the local version of inputs.conf.
  9. Enable the copied stanza in local/inputs.conf by uncommenting it. Note: Since TCP port 1514 is used for receiving ESXi logs, the 1517 port is used, by default, for vclogs. Other open ports can be used.
File properties Description
$InputFileName Used to monitor specific files.
$InputFileTag Used to set the prefix in each event data. Set $InputFileTag so your Splunk platform deployment can recognize sourcetype of different logs.
$InputFileStateFile Used to keep track of which parts of the monitored file are already processed. Must be unique.
$InputFileSeverity Used to set the type of log the user wants.
$InputRunFileMonitor Used to activate the monitoring.

For more information on configuration details, see the text file input module page.

Last modified on 12 June, 2020
PREVIOUS
Configure the Splunk Add-on for VMware to collect data
  NEXT
Configure the Splunk Add-on for VMware to collect log data from ESXi hosts

This documentation applies to the following versions of Splunk® Supported Add-ons: released


Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters