Splunk DFS Manager overview
The Splunk DFS Manager app enables you to run Data Fabric Search (DFS). The app bundles, installs, and configures Apache Spark for your DFS deployment automatically. Additionally, the app helps you to continuously and seamlessly manage, configure, and monitor your Spark cluster and customize the resource allocation within your DFS deployment from any search head through a scalable and adaptive user interface to run a DFS search.
The Splunk DFS Manager only supports Apache Spark as the compute cluster for a DFS deployment. Splunk provides support only for a Spark cluster that is deployed using the Splunk DFS Manager app. If you install your Spark cluster manually, Splunk isn't responsible for support or maintenance of the compute cluster.
You can use the app irrespective of your deployment scenario and install Spark on a standalone search head, standalone indexer, search head cluster, or an indexer cluster.
The Splunk DFS Manager app provides the following capabilities:
- High availability that allows an alternate search head captain to restart DFS master, in case a search head captain fails
- Customize resource usage by leveraging workload management (WLM) to allocate CPU and memory
- Add DFS workers on all or selected search peers
- Remove a DFS worker
- Restrict adding DFS workers to a particular site in a multi-site environment
- Monitor the health and resource usage of the Spark cluster
- Enable Spark security
The default Spark master port is 7777 to launch DFS jobs and connect DFS workers. The default Spark Web UI port is 8009 to view the Spark cluster.
Allow only the search heads in the search head cluster and the search peers to access the Spark master and the Spark UI port. You can block all other nodes from accessing the Spark master and the Spark Web UI port by using iptable and firewall rules.
Contact Splunk Services if you run into issues trying to configure your Spark cluster using the Splunk DFS Manager app.
Discuss the app for Splunk DFS Manager on Splunk Answers.
If you have a support contract, submit a case using the Splunk Support Portal.
For general Splunk platform support, see the Support and Services page.
Install Splunk DFS Manager
This documentation applies to the following versions of Splunk® DFS Manager: 1.0.0