HTTP Event Collector and the
If you already use Splunk HTTP Event Collector (HEC) to ingest data in the Splunk platform, you can update your data ingestion workflow to use DSP HEC and send your data to DSP instead. Send the events and metrics from your HTTP client to DSP so that you can process your data before sending it to the Splunk platform for indexing.
DSP HEC doesn't support the same tokens as Splunk HEC, so you'll need to create a new token and configure your HTTP client to use it. You can then use the Splunk DSP Firehose source function in a data pipeline to receive data from the HTTP client. DSP HEC uses the DSP API Gateway port to connect to the Splunk Data Stream Processor. See Get data from HTTP clients into DSP using the Splunk HEC API for instructions on how to redirect your Splunk HEC workflow to DSP.
DSP HEC supports the following Splunk HEC endpoints:
You can send metrics and events to any of these endpoints. See HTTP Event Collector REST API endpoints and Format events for HTTP Event Collector in the Splunk Enterprise Getting Data In manual to learn more about the differences between these endpoints and when to use them. The maximum supported metric or event payload size for all
/services/collector endpoints is 5 MB. If you send a metric or event payload that is larger than 5 MB, you will receive an
HTTP 413 Request entity exceeded size limit error message in the response body.
Differences between Splunk HEC and DSP HEC
|Splunk HEC||DSP HEC|
|Allows events and metrics to be written directly to the Splunk platform.||Allows events and metrics to be written to DSP. See Connecting your DSP pipeline to a Splunk index if the final destination for the ingested data is the Splunk platform.|
|Splunk Indexer error codes can be returned directly to the HTTP client.||Splunk Indexer error codes return an Invalid Data Format error in DSP HEC.|
|Each HEC token is associated with a set of authorized indexes. An error is returned if an event refers to another index.||DSP HEC can't directly control which index an event is written to. You can set default values for the |
|A typical Splunk HEC token looks like this:
||The DSP HEC token format is |
|Asynchronous event acknowledgment is supported via the
||DSP HEC does not support the asynchronous ACK protocol or the |
|Raw events are supported via the
||Raw events are not supported.|
|MINT formatted data is supported via the
||MINT formatted data is not supported.|
|Uses port 8088 to connect to Splunk Enterprise.||Uses port 31000 to connect to the Splunk Data Stream Processor API services.|
|The maximum payload size (
||The maximum supported event or metric payload size for all |
To learn more about Splunk HEC, see the following pages from the Splunk Enterprise Getting Data In manual:
Formatting data into the Splunk Infrastructure Monitoring metrics schema
Get data from HTTP clients into DSP using the Splunk HEC API
This documentation applies to the following versions of Splunk® Data Stream Processor: 1.2.0, 1.2.1-patch02, 1.2.1, 1.2.2-patch02, 1.2.4, 1.2.5, 1.3.0, 1.3.1
Feedback submitted, thanks!