Splunk architecture and processes
This topic discusses Splunk's internal architecture and processes at a high level. If you're looking for information about third-party components used in Splunk, refer to the credits section in the Release notes.
A Splunk server runs two processes (installed as services on Windows systems) on your host,
splunkdis a distributed C/C++ server that accesses, processes and indexes streaming IT data. It also handles search requests.
splunkdprocesses and indexes your data by streaming it through a series of pipelines, each made up of a series of processors.
- Pipelines are single threads inside the
splunkdprocess, each configured with a single snippet of XML.
- Processors are individual, reusable C or C++ functions that act on the stream of IT data passing through a pipeline. Pipelines can pass data to one another via queues.
splunkdsupports a command line interface for searching and viewing results.
- Pipelines are single threads inside the
splunkwebis a Python-based application server based on CherryPy that provides the Splunk Web user interface. It allows users to search and navigate data stored by Splunk servers and to manage your Splunk deployment through a Web interface.
splunkd can both communicate with your Web browser via Representational state transfer (REST):
splunkdalso runs a Web server on port 8089 with SSL/HTTPS turned on by default.
splunkwebruns a Web server on port 8000 without SSL/HTTPS by default.
On Windows systems,
splunkweb.exe is a third-party, open-source executable that Splunk renames from
pythonservice.exe. Since it is a renamed file, it does not contain the same file version information as other Splunk for Windows binaries.
Splunk and Windows in Safe Mode
splunkweb, nor the
SplunkForwarder services starts if Windows is in Safe Mode. Additionally, if you attempt to start Splunk from the Start Menu while in Safe Mode, Splunk does not alert you to the fact that its services are not running.
Additional processes for Splunk on Windows
On Windows instances of Splunk, in addition to the two services described above, there are additional processes that Splunk uses when you create specific data inputs on a Splunk instance. These scripted inputs run when configured by certain types of Windows-specific data input.
splunk.exe is the control application for the Windows version of Splunk. It provides the command line interface (CLI) for the program, and allows you to start, stop, and configure Splunk, similar to the *nix
splunk.exe requires an elevated context to run because of how it controls the
splunkweb processes. Splunk might not function correctly if this executable is not given the appropriate permissions on your Windows system. This is not an issue if you install Splunk as the Local System user.
splunk-admon.exe is spawned by
splunkd whenever you configure an Active Directory (AD) monitoring input.
splunk-admon's purpose is to attach to the nearest available AD domain controller and gather change events generated by AD. Splunk then stores these events in the desired index.
splunk-perfmon.exe (new for version 4.2) runs when you configure Splunk to monitor performance data on the local machine. This service attaches to the Performance Data Helper libraries, which query the performance libraries on the system and extract performance metrics both instantaneously and over time.
splunk-regmon.exe runs when you configure a Registry monitoring input in Splunk. This scripted input initially writes a baseline for the Registry as it currently exists (if desired), then monitors changes to the Registry over time. Those changes come back into Splunk as searchable events.
You can use this utility to test defined event log collections, and it outputs events as they are collected for investigation. Splunk has a Windows event log input processor built into the engine.
When you configure a performance monitoring, event log or other input against a remote computer, this program starts up. Depending on how you configure the input, either it attempts to attach to and read Windows event logs as they come over the wire, or it executes a Windows Query Language (WQL) query against the Windows Management Instrumentation (WMI) provider on the specified remote machine(s). Splunk then stores the events.
Estimate your storage requirements
About Splunk Enterprise licenses
This documentation applies to the following versions of Splunk® Enterprise: 4.3, 4.3.1, 4.3.2, 4.3.3, 4.3.4, 4.3.5, 4.3.6, 4.3.7, 5.0, 5.0.1, 5.0.2, 5.0.3, 5.0.4, 5.0.5, 5.0.6, 5.0.7, 5.0.8, 5.0.9, 5.0.10, 5.0.11, 5.0.12, 5.0.13, 5.0.14, 5.0.15, 5.0.16, 5.0.17, 5.0.18