All DSP releases prior to DSP 1.4.0 use Gravity, a Kubernetes orchestrator, which has been announced end-of-life. We have replaced Gravity with an alternative component in DSP 1.4.0. Therefore, we will no longer provide support for versions of DSP prior to DSP 1.4.0 after July 1, 2023. We advise all of our customers to upgrade to DSP 1.4.0 before July 1, 2023 in order to continue to receive full product support from Splunk.
HTTP Event Collector and the
If you already use Splunk HTTP Event Collector (HEC) to ingest data in the Splunk platform, you can update your data ingestion workflow to use DSP HEC and send your data to DSP instead. Send the events and metrics from your HTTP client to DSP so that you can process your data before sending it to the Splunk platform for indexing.
DSP HEC doesn't support the same tokens as Splunk HEC, so you'll need to create a new token and configure your HTTP client to use it. You can then use the Splunk DSP Firehose source function in a data pipeline to receive data from the HTTP client. DSP HEC uses the DSP API Gateway port to connect to the Splunk Data Stream Processor. See Get data from HTTP clients into DSP using the Splunk HEC API for instructions on how to redirect your Splunk HEC workflow to DSP.
DSP HEC supports the following Splunk HEC endpoints:
You can send metrics and events to any of these endpoints. See HTTP Event Collector REST API endpoints and Format events for HTTP Event Collector in the Splunk Enterprise Getting Data In manual to learn more about the differences between these endpoints and when to use them. The maximum supported metric or event payload size for all
/services/collector endpoints is 5 MB. If you send a metric or event payload that is larger than 5 MB, you will receive an
HTTP 413 Request entity exceeded size limit error message in the response body.
Differences between Splunk HEC and DSP HEC
|Allows events and metrics to be written directly to the Splunk platform.
|Allows events and metrics to be written to DSP. See Connecting your DSP pipeline to a Splunk index if the final destination for the ingested data is the Splunk platform.
|Splunk Indexer error codes can be returned directly to the HTTP client.
|Splunk Indexer error codes return an Invalid Data Format error in DSP HEC.
|Each HEC token is associated with a set of authorized indexes. An error is returned if an event refers to another index.
|DSP HEC can't directly control which index an event is written to. You can set default values for the
index fields in the DSP HEC tokens, but you'll also need to configure your pipeline to route the data according to the index value. See Connecting your DSP pipeline to a Splunk index for more information on configuring index routing.
|A typical Splunk HEC token looks like this:
|The DSP HEC token format is
dsphec:sha256:UUID. A typical DSP HEC token looks like this:
|Asynchronous event acknowledgment is supported via the
/services/collector/ack API endpoint.
|DSP HEC does not support the asynchronous ACK protocol or the
/services/collector/ack endpoint. If an HTTP 200 response is received from DSP HEC, the events in the request have been delivered to the Splunk DSP Firehose and are available for processing in your pipeline. No ACK is necessary.
|Raw events are supported via the
/services/collector/raw API endpoint.
|Raw events are not supported.
|MINT formatted data is supported via the
/services/collector/mint API endpoint.
|MINT formatted data is not supported.
|Uses port 8088 to connect to Splunk Enterprise.
|Uses port 443 to connect to the Splunk Data Stream Processor API services.
|The maximum payload size (
max_content_length) defaults to 2 GB.
|The maximum supported event or metric payload size for all
/services/collector endpoints is 5 MB.
To learn more about Splunk HEC, see the following pages from the Splunk Enterprise Getting Data In manual:
Formatting data into the Splunk Infrastructure Monitoring metrics schema
Get data from HTTP clients into using the Splunk HEC API
This documentation applies to the following versions of Splunk® Data Stream Processor: 1.4.0, 1.4.1, 1.4.2, 1.4.3