Send events to a DSP data pipeline using the DSP HTTP Event Collector

You can send events and metrics data to a DSP data pipeline using the DSP HTTP Event Collector (DSP HEC). The DSP HEC supports the Splunk HTTP Event Collector (HEC) /services/collector, /services/collector/event, and /services/collector/event/1.0 endpoints allowing you to quickly redirect your existing Splunk HEC workflow into DSP and ingest your data through the Read from Splunk Firehose data source function.

DSP HEC does not share tokens with Splunk HEC. You must create a DSP HEC token with the Ingest REST API or create a DSP HEC token with SCloud, and then configure your HTTP clients with the DSP HEC token to send data to the DSP Firehose.

DSP HEC uses the DSP API Gateway port to connect to the Splunk Data Stream Processor.

Differences between Splunk Enterprise HEC and DSP HEC

Splunk Enterprise HEC	DSP HEC
Allows events and metrics to be written directly to Splunk Enterprise	Allows events and metrics to be written to DSP. See Sending data from DSP to the Splunk platform if the final destination for the ingested data is Splunk Enterprise.
Splunk Indexer error codes can be returned directly to the HTTP client.	Splunk Indexer error codes return an Invalid Data Format error in DSP HEC.
Each HEC token is associated with a set of authorized indexes. An error is returned if an event refers to another index.	DSP HEC can't directly control which index an event is written to. You can set default values for index fields in the DSP HEC tokens, and you must configure the index routing in your DSP pipeline. See Sending data from DSP to the Splunk platform for more information on configuring index routing.
A typical Splunk HEC token looks like this: ef976ef0-dc7b-46b9-aa2e-c407cad884e2	DSP HEC token format is dsphec:sha256:UUID. A typical DSP HEC token looks like this: dsphec:e9da86d351cf9a7642d8c50195c3f466220911a15c177809bd1161a51e8c5f24:14c813f1-33ab-426b-8350-1b3e7f1e83f8
Asynchronous event acknowledgment is supported via the /services/collector/ack API endpoint.	DSP HEC does not support the asynchronous ACK protocol or the /services/collector/ack endpoint. If an HTTP 200 response is received from DSP HEC, the events in the request have been delivered to the DSP firehose and are available for processing in your DSP pipeline. No ACK is necessary.
Raw events are supported via the /services/collector/raw API endpoint.	Raw events are not supported.
MINT formatted data is supported via the /services/collector/mint API endpoint.	MINT formatted data is not supported.
Uses port 8088 to connect to Splunk Enterprise.	Uses port 31000 to connect to the Splunk Data Stream Processor API services.

Example workflow: Use Splunk HEC to send data to a DSP pipeline

1. Create a pipeline using the DSP UI and set the source function to Read from DSP Firehose and configure the pipeline to send data from DSP to the Splunk platform.
2. Create a DSP HEC token with the Ingest REST API or create a DSP HEC token with SCloud.
3. Update the base URL and token in the HTTP client used in your current Splunk HEC workflow and start sending data to your DSP pipeline.
   • Set the URL to https://<DSP_HOST>:31000.
   • Set the token to Authorization: Splunk <dsphec-token>.
4. Use DSP to transform and troubleshoot your data and then send that data to Splunk Enterprise or Splunk Cloud for indexing.

See also

See Set up and use HTTP Event Collector in Splunk Web for more information on setting up HEC in Splunk Enterprise.

See Format events for HTTP Event Collector for more information on formatting events for Splunk HEC.

See Send metrics to a metrics index for more information on formatting metrics for Splunk HEC.