Deploy a Data Collection Node

Deploy a Data Collection Node (DCN) to collect data from VMware vSphere vCenter Servers. You can deploy a DCN with the Splunk VMware OVA for ITSI or manually convert a heavy forwarder into a DCN.

After you deploy a DCN, configure data collection with a Data Collection Scheduler (DCS) from the Add Data page in the Splunk App for Infrastructure. For information about setting up a DCS, see Deploy a Data Collection Scheduler.

If you're collecting VMware vCenter Server data and migrate Python versions, you have to update your Data Collection Node configuration. For more information, see Update a Data Collection Node after migrating Python versions.

Prerequisites

• You viewed the DCN requirements. See Data Collection Node requirements and limits.

Deploy a DCN with the Splunk VMware OVA for ITSI

Deploy the Splunk VMware OVA for ITSI in a vCenter Server. When you deploy the OVA, it creates a virtual machine running a Splunk heavy forwarder that's configured to be a DCN. For information about specifications for the OVA, see Install and configure the OVA in the Install and Configure the Splunk VMware OVA for ITSI guide.

Follow these steps to deploy the OVA and set up the DCN.

1. Download the Splunk VMware OVA for ITSI

Download the Splunk VMware OVA for ITSI from Splunkbase.

2. Install the Splunk Add-on for Infrastructure inside the OVA

Install the Splunk Add-on for Infrastructure from Splunkbase inside the OVA for index time extractions.

3. Deploy the DCN in a vCenter Server

Use the OVA to create a virtual machine in your vCenter Server to run the DCN. For information about deploying an OVA, see Deploy an OVF or OVA Template on the VMware website.

For information about DCN requirements, see Data Collection Node requirements and limits.

After you deploy the OVA, the vCenter Server automatically assigns an IP address to the DCN via DHCP. For the most reliable connection, configure a static IP address for the DCN if possible.

Follow these steps to set up the DCN after you deployed the OVA in a vCenter Server.

1. SSH into the virtual machine that's running the DCN.
2. Log in to the virtual machine as the root user:

   username: root
   password: changemenow
   

3. Change the password:

   # passwd
   

4. Run the dcn-network-config command to test the network configuration for the DCN. Press Enter for each setting you don't want to change. When you finish, the dcn-network-config command tests your network configuration. These are the settings you can modify:

   IPv4 address
   IPv4 address of the default gateway
   Netmask
   DNS/Nameserver
   Hostname
   

4. Configure the DCN

1. Log in to the virtual machine with these user credentials:

   username: splunk
   password: changeme
   

2. Run the dcn-splunk-config command and enter a new password for the admin user for the universal forwarder. Before you change the password for the admin user, the default password is changeme.
3. Configure these settings:

   Setting	Description
   Enter comma separated Indexers(<host>:<port>)[]	Enter the IP address and port of each indexer you want to forward data to. For more information about forwarding data directly to indexers, see Connect forwarders directly to peer nodes in the Splunk Enterprise Managing Indexers and Clusters of Indexers guide.
   Enter license master(https://<host>:<port>)[self]	You don't have to configure the DCN as a license slave to collect data from the VMware vCenter Server. Press Enter to continue without providing a license manager.

4. Save your changes and restart Splunk.

Configure a heavy forwarder to be a DCN

Follow these steps to configure an existing heavy forwarder as a DCN. If you use a heavy forwarder that's performing other tasks, you may run into performance issues. For the best performance, configure a heavy forwarder to be a dedicated DCN, especially if you want to monitor vCenter Servers that are near or at the maximum number of ESXi hosts and virtual machines a DCN can manage. For information about DCN requirements and limits, see Data Collection Node requirements and limits.

Make sure that you're running a compatible version of Splunk Enterprise for the instance you configure as a DCN. For version requirements, see Version compatibility.

1. Install the add-ons

Install the following add-ons on the heavy forwarder. Download the Splunk ITSI package on Splunkbase and extract the add-ons. To install the add-ons, copy the directories from the vmware_ta_itsi parent directory in the ITSI package to $SPLUNK_HOME/etc/apps.

Add-on	Description
Splunk_TA_vmware	Runs a Python-based API data collection engine and performs search-time tagging of VMware data.
SA-Hydra	Runs worker processes to collect VMware data from vCenter Servers.
Splunk_TA_esxilogs	Runs props and transforms for ESXi log data you forward to the DCN. If you don't forward ESXi log data to the DCN and instead forward ESXi log data directly to the indexer tier, you don't have to install this add-on on the DCN.
Splunk_TA_vcenter	Runs props and transforms for vCenter Server log data you forward to the DCN. If you don't forward vCenter Server log data to the DCN and instead forward vCenter Server log data directly to the indexer tier, you don't have to install this add-on on the DCN.

Install the following add-on on the heavy forwarder. Download the Splunk Add-on for Infrastructure package on Splunkbase and install it under $SPLUNK_HOME/etc/apps.

Add-on	Description
Splunk_TA_Infrastructure	Contains all the index time extractions for VMware.

2. Configure forwarding

Configure the heavy forwarder to send data to your indexer or distributed indexer environment.

Follow these steps to configure forwarding:

1. Enable forwarding on the heavy forwarder. For information about deploying a heavy forwarder, see Set up forwarding in the Splunk Enterprise Forwarding Data guide.
2. Specify the indexer or group of indexers you want to forward data to. There are two options to do this:
   1. Manually specify each indexer you want to forward data to. For more information about forwarding data directly to indexers, see Connect forwarders directly to peer nodes in the Splunk Enterprise Managing Indexers and Clusters of Indexers guide.
   2. Configure indexer discovery to forward data to indexers. For information about how to use indexer discovery, see Use indexer discovery to connect forwarders to peer nodes.

Configure additional settings for a DCN

After you deploy the DCN, you can configure settings to change the limit for the count of metrics the DCN collects from a VMware vCenter Server, change the NTP server pool list, and disable NTP on the data collection node.

Change the limit for the count of metrics the DCN collects

As of VMware vCenter Server version 5.5 Update 2d, there's a 64 limit count of performance metrics that the vpxd.stats.maxQueryMetrics function collects. the vCenter Server calculates the count of performance metrics by multiplying the number of metrics by the number of virtual machines that you're querying. For example, if you query 10 metrics from eight virtual machines, that's a query size of 80.

If you hit the limit, you'll see a message like this:

Request processing is restricted by administrator.

For instructions on how to change the limit, see Performance charts are empty and displays the error: Request processing is restricted by administrator (2107096) on the VMware Knowledge Base website.

Change the NTP server pool list

A system uses the Network Time Protocol (NTP) to synchronize its time with another reference time source. If you're experiencing time synchronization issues between the indexer, DCN, and vCenter Server, change the NTP servers that the DCN uses.

1. On the instance running the DCN, go to /etc/ntp.conf. These are the following values:

    # Use public servers from the pool.ntp.org project.
    # Please consider joining the pool (http://www.pool.ntp.org/join.html).
    server 0.centos.pool.ntp.org
    server 1.centos.pool.ntp.org
    server 2.centos.pool.ntp.org
   

2. Replace the default values in the file with your NTP server values.
3. Restart ntpd:

   $ sudo service ntpd restart
   

Disable NTP on the DCN

If you deployed a DCN in a vCenter Server and it doesn't have internet access, disable NTP on the DCN. If you disable NTP, enable VMware Tools Clock Synchronization to establish the time for the DCN using the ESXi host.

1. Stop ntpd:

   $ sudo service ntpd stop
   

2. Configure ntpd so that it doesn't run when the system starts:

   $ sudo chkconfig ntpd off
   

3. Enable VMware Tools Clock Synchronization:

   $ vmware-toolbox-cmd timesync enable
   

4. Confirm that VMware Tools Clock Synchronization is enabled:

   $ vmware-toolbox-cmd timesync status