Configure data collection

Once you have service account(s) created and the Splunk Forwarder Virtual Appliance for VMware (FA VM) is configured, you are now ready to create the configuration files that are responsible for collecting data from the target machines in your VMware environment.

The Splunk Forwarder Virtual Appliance for VMware (FA VM) sends data to your Splunk indexers after you configure the engine.conf file(s) and inputs.conf files.

Run enginebuilder.py to automatically create the engine.conf files that specify how data is collected in your VMware environment from vCenter. It reads the engine.template file that contains key information about your environment.

enginebuilder.py is located in the FA VM in the directory: $SPLUNK_HOME/etc/apps/Splunk_TA_vmware/bin. As the splunkadmin user, you already have the configuration builder in your path, so you can run it from any directory location. Note that the engine.conf files that the tool creates are output to the same directory from which the tool is invoked.

enginebuilder.py does the following:

1. It create engine.conf files by data type: engine<datatype>.conf.
2. It create the inputs.conf file that starts the engine instances using engine.conf files.
3. It checks permissions on vCenter server and all ESX/i hosts.

Set up engine.template

Note: Always stop Splunk before changing the engine.conf file. This is to avoid saving incomplete copies of engine.conf while editing it, which prevents the engine from generating errors based on an incorrect configuration. The engine periodically reads the engine.conf file and sees changes that are made to it. For more information about starting or stopping Splunk, see "Start and stop Splunk" in the Splunk Admin Manual.

1. ssh to the FA VM as splunkadmin
2. Gather the environment parameters for the engine.template file. Before you run the tool, the engine.template file must contain the following information:
   1. The username for the service account created to access vCenter (vcuser).
   2. The password for the vcuser account above (vcpwd).
   3. The IP or hostname of the vCenter Server (vc).
   4. The username for the service account created to access ESX/i hosts (hostuser).
   5. The password for the hostuser account above (hostpwd).
   6. A comma separated list of values containing ESX/i host IPs or hostnames. You can use“*” to generate files that cover ALL of the ESX/i hosts managed by the given VC (host_csv).
   7. perfInstanceData: This option is set to OFF by default. When turned ON it provides fine-grained control over the amount and kinds of performance data that you want the engine to collect.

   Important: enginebuilder.py assumes that all ESX/i hosts in the host_csv field use the same service account username and password (hostuser, hostpwd). If your ESX/i hosts do not use the same service account credentials, you may need to run the tool multiple times or generate the FA VM configuration files manually.

3. Create a local directory in $SPLUNK_HOME/etc/apps/Splunk_TA_vmware

   mkdir $SPLUNK_HOME/etc/apps/Splunk_TA_vmware/local/

4. In the FA VM, go to the local directory where you want to generate the configuration files:

   cd $SPLUNK_HOME/etc/apps/Splunk_TA_vmware/local

5. Copy the engine.template file.

   cp ../default/engine.template .

6. Edit the engine.template file to include the correct environment parameter values (such as vCenter name, host name, user name, and so on). When you run enginebuilder.py it uses the information in engine.template to create the configuration files (engine.conf files) for the FA VM. You can include multiple VCs in the file.
   1. For large scale environments you can create a template file that covers multiple vCenters by copying all of the entries in engine.template and pasting them (together), below the current group of values. Continue to do this untll you have covered all of the VCs you want to add to the configuration.

The following is a sample engine.template with 2 vCenters. The first with a subset of hosts in that vCenter and the second with all hosts.

vcuser=splunkuservc
vcpwd=splunkuser123
vc=vc1.company.com
hostuser=splunksvc
hostpwd=splunkuser123
host_csv=esx1.company.com,esx2.company.com,esx3.company.com
perfInstanceData=OFF

vcuser=splunkuservc
vcpwd=splunkuser123
vc=vc1.company.com
hostuser=splunksvc
hostpwd=splunkuser123
host_csv=*
perfInstanceData=OFF

Run enginebuilder.py

1. Run enginebuilder.py with the appropriate options. It reads the engine.template file and generates all of your engine.conf files and inputs.conf. We recommend that you run this command using the -c option to automatically check the credentials for all logins that you are using with the App. The configuration files are created even if the logins are not valid.
2. Ensure you are logged in as splunkadmin
3. Go to $SPLUNK/etc/apps/Splunk_TA_vmware/local directory (you should have the engine.template file that you modified in that directory.
4. To run enginebuilder.py from the local directory run it as following:
   1. $ enginebuilder.py [argument list]
5. You should always run enginebuilder with the -c argument. This checks the validity of the defined user credentials on your vCenter and ESX/i hosts.
6. Decide whether you need further optimization of performance data collection
   1. It is recommended that each engine-perf<numer>.conf file that gets generated, should only contain monitoring details of hosts that amount to around 300 VMs. It is therefore important to know what your average VM to host ratio is and then split up performance data collection across several engine-perf<number>.conf files.
   2. Use the -l argument to achieve this. E.g. If you have a ratio of 30 VMs per host then at 10 hosts you will reach the 300 VM limit per engine-perf<number>.conf file. Suppose now you had 20 hosts that would be approximately 600 VMs and you will need 2 of these engine-perf<number>.conf files. In this scenario you will need to use the argument -l 10 limiting each engine-perf<number>.conf to 10 hosts (and therefore 300 VMs)
7. Decide on the number of hosts you will be monitoring with this FA VM. The limit should be 20 (or 30 if you have increased the FA VM resources) hosts per FA VM.
   1. Use the -f argument to split data collection across multiple FA VMs. e.g. -f 20 to limit data collection to 20 hosts for a FA VM.
   2. The engine.conf files for the other FA VMs will be packaged into a tar.gz file which you will need to copy over, and you will need to use the -u argument to unpack the tar.gz when on the other FA VMs.

$ enginebuilder.py -c

or 

$ enginebuilder.py -c -l 10
(not required in small test environments of 1-10 hosts)

or 

$ enginebuilder.py -c -l 10 -f 30
(not required in small test environments of 1-10 hosts)

Start data collection

1. Collecting information from vCenter can take some time. After running enginebuilder.py you will have many engine.conf files and an inputs.conf files.
2. Your FA VM is now ready to run. Start splunk:

   splunk start

You now have an FA VM that is configured for your environment and ready to work.! When Splunk starts, the engine instance is started by the simple inputs.conf file. The engine looks for the engine.conf files in the Splunk_TA_vmware/local directory and starts collecting data. Now you can validate your setup. See "Validate your installation" in this manual.