HDP-About Hadoop Monitor

The Hadoop Monitor app provides centralized monitoring of current and past Hadoop MapReduce jobs and tasks. It allows you to monitor your entire Hadoop deployment, efficiently and quickly, no matter what its size. In addition to tracking a variety of data about jobs and tasks, the Hadoop Monitor provides ready access to Hadoop warning and error messages.

Here are some typical use cases for the app:

• Hadoop user submits a job and wants to track it.
• Hadoop administrator needs to monitor the health of the entire infrastructure.
• Administrator needs to determine cause of Hadoop job failure. The Hadoop Monitor provides visibility into the entire cluster, so you can quickly determine which nodes are contributing to job failure. You can also easily determine whether certain nodes are underutilized or are prone to failure.
• Administrator needs to scale the Hadoop system. The Hadoop Monitor can help with capacity planning, as it offers visibility into current loads across the slave nodes.

How does it work?

The Hadoop Monitor app runs on a central Splunk instance. To feed Hadoop data to the app, you run a lightweight version of Splunk, called a forwarder, on each Hadoop node. The forwarders send JobTracker and TaskTracker data to the central Splunk instance, which indexes the data on an ongoing basis. The app runs searches against the data and presents the results through its GUI.

For a more detailed description, see "What a Hadoop Monitor deployment looks like".

How to get Hadoop Monitor

The Hadoop Monitor app is available on Splunkbase.

<<Back to the documentation landing page