Requirements for collecting data

Networking

The FA VM must be able to connect to each vCenter machine and each ESX/i host in the vSphere environment for data to be collected and splunked. Splunk for VMware itself, as it monitors the vSphere environment, generates events (user session events) when it connects to the resource in that environment.

VM Considerations

Each FA VM contains Splunk configured as a Heavy Forwarder. The FA VM is also initially configured with a single CPU and default memory constraints. We recommend that you configure your Solution to maximize the use of the FA VM as this keeps the number of FA VMs running to a minimum, which uses less resources and is easier to manage and maintain over time.

When you run multiple engines, each process runs independently of the other, so each one is allocated its own CPU time, memory, and socket connections to their respective target machines. How "hot" (resource intensive) your FA VMs run depends upon:

• The size of your environment.
• The number of concurrent engine processes you run.
• The number of forwarder appliances you run.

Execution Times

The time it takes for an action (in engine.conf) to run, directly depends upon the data type being collected and the quantity of data being collected. For example, collecting hierarchy data from a VC that manages a large number of ESX/i hosts and VMs takes longer than collecting the same data from a smaller environment. Also, in an environment that has a large number of active users continuously generating tasks, collection of task data will take longer due to the higher volume of tasks.

All data collection modules are designed to collect a certain amount of data (often configurable) and then release control to another module. In this way no single action consumes all of the engine's processing time and "starves" other modules. As a result data collection can take time as it is often collected in stages.

Host value settings

Host is referred to in a number of contexts in the Splunk for VMware solution. Setting the specific host values correctly in the Solution is important as these values impact the values of the "host" fields in Splunk events. An incorrect host setting can cause data to be indexed incorrectly, resulting in broken searches and incorrect representation of the data in in the Splunk App for VMware dashboards and views.

There are "ESX/i hosts", "host" settings in configuration files, values of "host" fields in the Splunk events, and "OS hostname" properties that different machines have (VC, ESX/i hosts, FA VMs, and so on). See the table below for examples of where and how the host setting is used:

File	Description
$SPLUNK_HOME/etc/system/local/inputs.conf	For each FA VM deployed in the solution, set this host setting to a unique OS hostname.
$SPLUNK_HOME/etc/system/local/server.conf	Set the serverName in this file to the value set in the system/local/inputs.conf file in each FA VM.
$SPLUNK_HOME/etc/apps/splunk_for_vmware/local/inputs.conf	Set the host setting in this file to value set in the system/local/inputs.conf file in each FA VM.
$SPLUNK_HOME/etc/apps/splunk_for_vmware/local/engine.conf	This is where all engine.conf files reside. The engine.conf file contains many host related fields: For each FA VM, set the FA VM to have the same value as the host setting in the system/local/inputs.conf file. This is the unique OS hostname. This is the value used in the fa field in Splunk events generated by the engine. Set the host setting in VC stanzas to the VC instance name. This is the name of the root node in the vSphere Client VC's "Hosts and Clusters" view. Set the host setting in an unmanaged ESX/i host stanza to the ESX/i host's "fully qualified hostname".

Note: engine.pm gets the correct host value for all managed ESX/i host stanzas directly from vCenter. You do not need to explicitly set the value.

Assign correct values to ESX/i host fields

An ESX/i host field must be given the correct value to enable data to be collected from the host and for the Splunk App for VMware to accurately display the data. This is very important for unmanaged hosts because it directly sets the "host" field value in the Splunk events that contain data coming from that host. You must know the ESX/i host's "fully qualified hostname". For unmanaged ESX/i hosts this is the value for the "host" setting in the engine.conf stanzas for unmanaged hosts.

The "fully qualified hostname" is the internal value that an ESX/i host uses for its "hostname". To get the internal "hostname" for an ESX/i host you must point the vSphere Client directly at the ESX/i host .

To get the fully qualified host name:

1. Open the vSphere Client.
   1. Enter an IP address or hostname for the ESX/i host itself (not the vCenter).
   2. Enter the splunk administrator login credentials for a local ESX/i user account. Use credentials (splunkadmin / changeme) you previously created.
2. When the client has connected to the host and logged in, click Home > Inventor > Inventory to look at the main inventory screen.
3. In vSphere Client, click on the root node in the tree in the navigation panel, then elect the "Configuration" tab.
4. In the Configuration tab, in the "Software" panel, click "DNS and Routing". This shows the "Host Identification" information "Name" and "Domain" properties.

   The "Name" is the internal OS hostname (esxhost1).

   The "Domain" is the configured DNS domain (company.com).

5. Set the fields to the correct value (if not already set):

   * In the panel, click "Properties".

   * In the Properties window, enter the name and the domain values for your environment.

   *Click "ok" to make the change.

6. You can now construct the fully qualified hostname for the ESX/i host. Combine the internal OS hostname and domain using a "." separator, for example esxhost1.company.com.

Note: Use the "fully qualified hostname" for the ESX/i host to set up the inputs.conf file and the engine.conf files when configuring the Solution.

Get the ESX/i host name in the VC

engine.pm learns about vCenters using blank stanzas. A blank VC stanza is a stanza in engine.conf that points to a vCenter machine but has no action=value field set for it. To populate the dashboards in the Splunk App for VMware engine.pm must have access to VC(s) to collect the data. These situations and when a "blank VC stanza" needs to be applied is summarized below:

1. ESX/i hostname in vCenter: engine.pm automatically sets the "host" field on Splunk events for managed ESX/i hosts. It gets the host name (the "ESX/i hostname in VC") by contacting the VC managing that ESX/i host. To gather this informtion, engine.pm must have access to the ESX/i host's managing VC. You can see a managed ESX/i host's name in VC by pointing the vSphere Client at VC and looking at the list of ESX/i hosts in the tree view on the left panel of the "Hosts and Clusters" inventory view.
2. Moid mapping: engine.pm uses data from the VC to perform "moid mapping" on managed ESX/i hosts. As performance data comes directly from the managed ESX/i hosts, it must also include a moid mapping step for managed ESX/i hosts because the data lives in a different "namespace" from VC. Moid mapping enables the performance data for a given managed ESX/i host to be uniquely identified using the special values ("moids") in the VC but not on the ESX/i hosts.

If a managed ESX/i host stanza is in an engine.conf file, but there is no stanza for its associated VC, then you must add a blank VC stanza to engine.conf. Blank VC stanzas are used in the engine.conf files so that engine.pm can gather all of the data it needs. Note: Set the "interval" value to 1 in the blank VC stanzas to avoid unnecessary delays in data collection. Follow the instructions carefully to avoid "can't access VC" data problems. Blank VC stanzas are not needed for unmanaged ESX/i hosts.