Troubleshoot your deployment
This documentation does not apply to the most recent version of Splunk. Click here for the latest version.
Contents
Troubleshoot your deployment
The Deployment Monitor is a great tool for troubleshooting your deployment. Among other features, the monitor's home dashboard includes a set of warnings that provide immediate notice of any abnormalities in your system.
Forwarders behaving badly
The bottom half of the home dashboard provides a number of categories of warnings. Warnings do not necessarily indicate problems in your system, but they do indicate areas that you might need to investigate further.
To get detailed information on any particular warning, click on the green arrow to the right of the warning. For example, you might see a warning that says "5 missing forwarders." To see a list of these missing forwarders, click on the green arrow. You can then drill down within the list for a more detailed look at individual forwarders.
To create an alert that corresponds to a warning, click the Configure Alerting link to its right.
These warnings are just the first step in troubleshooting your deployment. They can indicate serious conditions or, on the other hand, completely benign situations. Depending on your system, there might be excellent reasons for a forwarder to go missing or for other forwarders to be sending less data than expected.
Types of warnings
Warnings indicate possibly unusual behavior in indexers and forwarders. Here are the possible warnings, along with their likely causes and suggested remedies.
Indexer warnings
| Warning | Likely causes | Possible remedies |
|---|---|---|
| N idle indexer(s) | Possible problem with indexer, network, data sources, or forwarders sending data to the indexer. | Drill down into the list of indexers and into the detailed information for each indexer to determine root cause and remedy. |
| N overloaded indexer(s) | Indexer cannot keep up with incoming data flow. This can result in poor search performance and data latency. | Add more indexers or filter incoming data. |
Forwarder warnings
| Warning | Likely causes | Possible remedies |
|---|---|---|
| N missing forwarder(s) | Splunk failure, network outage, or issue with underlying system. Trivial causes include laptops or virtualized hosts going on/off line. | Drill down into the list of missing forwarders and into the detailed information for each missing forwarder to determine root cause and remedy. |
| N quiet forwarder(s) | Forwarder believes a source has gone silent. | Determine whether the problem lies with the source or with the forwarder. |
| N forwarder(s) sending less than expected | Can indicate a problem with the underlying system. | Drill down into the list of forwarders and into the detailed information for each forwarder to determine root cause and remedy. |
| N forwarder(s) sending more than expected | Can indicate an attack, data dump due to application crash, or other system problem. Other possibilities include bad Splunk configuration or a new rogue data source configured by a user. Too much data can result in license violations or indexers being unavailable for searches. | Drill down into the list of forwarders and into the detailed information for each forwarder to determine root cause and remedy. |
Source type warnings
| Warning | Likely causes | Possible remedies |
|---|---|---|
| N missing source type(s) | A particular source on your forwarders is misconfigured. A source type isn't being correctly applied to a source, perhaps because the source data has changed. | Drill down into the list of missing source types and into the detailed information for each missing source type to determine root cause and remedy. |
| N source type(s) sending less than expected | A source type isn't being correctly applied to some of the data from a source. Network issues are preventing data from reaching a Splunk. | Drill down into the list of source types and into the detailed information for each source type to determine root cause and remedy. |
| N source type(s) sending more than expected | Data from one source is being incorrectly source-typed. A system is in an error loop and is sending many repeated messages. | Drill down into the list of source types and into the detailed information for each source type to determine root cause and remedy. |
This documentation applies to the following versions of Splunk: 4.2 , 4.2.1 , 4.2.2 , 4.2.3 , 4.2.4 , 4.2.5 , 4.3 , 4.3.1 , 4.3.2 , 4.3.3 , 4.3.4 , 4.3.5 , 4.3.6 View the Article History for its revisions.