Is data coming in

Have you installed the Solution correctly and do you have the right data coming into Splunk? Do you know if your data is displayed correctly in the views and dashboards of the App? In this section we will validate the integrity of your data by looking at your environment in the Splunk App for VMware.

Is your data being forwarded to Splunk

To check that you have correctly set up your forwarders to forward data:

1. In the App, Select Solution Administration > Admin Data Health.
2. In the "Forwarder Appliance" menu, check that all the forwarder appliances that you have as part of the Solution are included in the list.
3. Select each forwarder appliance in the list individually, and check that the Forwarder Appliance summary displays data for each.
4. Verify that your VI-Perl SDK has a value for each forwarder appliance. If it does not, then you must reinstall the Perl API package as it did not install correctly.
5. Check that all vCenters from where you installed the vCenter add-on show up in the "Virtual Center" menu.
6. Select each vCenter individually to ensure that the Virtual Center summary shows data for all vcenters.

Are you collecting the correct type of data

Inventory Data

To check that inventory data (used to populate the drop downs) is collected correctly, look at the Inventory Data health views.

To check inventory data:

1. In the App, select Solution Administration > Inventory Data Health.
2. For all data centers, folders, clusters, resource pools, host systems, and virtual machines, check that the correct number of each resource is displayed in the "Number of Unique entities seen in the last 2 hours".
3. Set the time range to 60 minutes.
4. Ignore the Bucket Span for Count Charts.
5. Check that data is displayed in all bar charts.

Hierarchy Data

To check that hierarchy data (used to populate the drop downs) is collected correctly, look at the Hierarchy Data health views.

To check hierarchy data:

1. In the App, select Solution Administration > Hierarchy Data Health.
2. Check that data is displayed in the bar chart. If data is displayed, then hierarchy data is being collected at a regular set interval.
3. Check the "Overview of virtual centers" panel. Ensure that the correct number of hosts within each vCenter and data center is shown in the data. This should directly reflect your environment.

Performance Data

To check that performance data is collected correctly, look at the Performance Data health views.

To check performance data:

1. In the App, select Solution Administration > ESX/ESXi Performance data health.
2. Check the "Overview of performance data by host" view. This view shows the type of memory metrics being received for each host. The "Check Performance data" section covers similar information, except that it checks on the ClusterComputeResourcePerf performance data source. You don't have to check this view.

Searches to run to validate performance data results

1. Check that data has been coming in for the last 15 minutes:

   index=vmware sourcetype=vmware:perf

2. Four types of data are displayed - ClusterComputeResourcePerf, HostSystemPerf, ResourcePoolPerf, and VirtualMachinePerf.

   index=vmware sourcetype=vmware:perf | stats count by source

3. Shows a breakdown of all hosts that are sending performance data and the types of data they are sending. ClusterComputeResourcePerf should only be returned by the Virtual Center hosts.

   index=vmware sourcetype=vmware:perf | stats values(source) by host

ESX/ESXi Log data

To check that ESX/ESXi Log data is collected correctly, look at the ESX/ESXi Log Data Health views.

To check ESX/ESXi Log data:

1. In the App, select Solution Administration > ESX/ESXi Log Data Health.
2. Look at the the "Log event volume" view to see if you are receiving ESX/ESXi log data.

To check ESX log data by host:

1. To ensure that you are collecting ESX/ESXi log data for each ESX/ESXi host you are monitoring, run:

   index=vmware sourcetype=vmware:esxlog:* | stats count by host

2. To ensure that you are collecting ESXi log data for each ESXi host you are monitoring, run:

   index=vmware sourcetype=vmware:esxilog:* | stats count by host

Tasks and Events Data

To check that Tasks and Events data is collected correctly, look at the Tasks and Events Data Health views.

To check Tasks and Events data:

1. In the App, select Solution Administration > Tasks and Event Health.
2. Check the "Overview of tasks" and "Overview of events" views. If data is displayed in on the bar charts in both views, then tasks and event data is being collected. If these views appear empty ( if you have not been using your environment and you have not performed any recent tasks) increase the time range and see if selecting a longer period of time causes data to display.

Check tasks and events by host:

1. To display all the hosts (including VCs) from which you are receiving task data, run:

   index=vmware sourcetype=vmware:task | stats count by host

   1. Check that all the hosts included in your splunked environment are listed.
2. To display all the events (including VCs) from which you are receiving task data, run

   index=vmware sourcetype=vmware:event | stats count by host

   1. Check that all the events in your splunked environment are listed.

VC Log Data

To check that VC Log data is collected correctly, look at the Virtual Center Server Log Data Health views.

To check VC Log data:

1. In the App, select Solution Administration > Virtual Center Server Log Data Health.
2. For all vCenter servers from which data is being collected, look at "Virtual center forwarding status" to see that data is being received.
3. Check that each of the three bar charts displays three different types of Virtual Center logs.

Verify dashboard health

Verify Inventory dashboards

To verify inventory dashboards:

1. In the App, select Inventory > Virtual Machines by ESX/i Hosts.
2. Look at "VMs on Host All". This view combines your inventory data with your hierarchy and inventory data.
3. Check that it has all of your VCs, the corresponding hosts in the vCenter, and the VMs on the hosts. Additional information about the VMs is also available. You can do a comparison of this data by looking at your vSphere client.

1. Select a host from the menu to display host summary information.
2. Select Inventory > Datastore Summary.
3. Check that data is displayed on both views.

1. Select Inventory > Inventory Hierarchy
2. Select a Virtual Center from the "Virtual Center" drop down to see the inventory for the VC selected.

Select Inventory > VM Snapshots

1. Check that you can see data for all virtual centers in the "VM's with snapshots present in Inventory". Click on a particular row and for the selected VM two new views "Snapshot details" and "Snapshot history" appear at the bottom.

Verify Performance dashboards

1. Select Performance > ESX/i Metrics by Type and Instance
2. Check that the data is being displayed for the selected hosts and corresponding VMs.
3. Go through all metrics for a host to see which ones produce data. All metrics do not necessarily populate.

Note: This dashboard is the superset of all the performance metrics dashboards in this section.

Verify Tasks dashboards

1. Check the ESX/i Host task overview.
2. Check Task Overview.

Verify Troubleshooting dashboards

1. Check that there is data in the ESX/i Log browser.
2. Check that there is data in the vCenter Log browser.