Splunk® Data Stream Processor

Connect to Data Sources and Destinations with DSP

DSP 1.2.0 is impacted by the CVE-2021-44228 and CVE-2021-45046 security vulnerabilities from Apache Log4j. To fix these vulnerabilities, you must upgrade to DSP 1.2.4. See Upgrade the Splunk Data Stream Processor to 1.2.4 for upgrade instructions.

On October 30, 2022, all 1.2.x versions of the Splunk Data Stream Processor will reach its end of support date. See the Splunk Software Support Policy for details.
This documentation does not apply to the most recent version of Splunk® Data Stream Processor. For documentation on the most recent version, go to the latest release.

Set the default field values in DSP HEC

When you send data to through the DSP HTTP Event Collector (HEC), if the data doesn't include values for the source, sourcetype, index, or time fields, then default values are used. The time field defaults to the ingest time of the event, but you can specify the default values for the source, sourcetype, and index fields.

The source, sourcetype, index, and time values in your requests to the Splunk HEC API endpoints map to DSP record fields as follows:

Request body field DSP record field
source source
sourcetype source_type
index An index key in the attributes field.
time timestamp

Precedence order of default values

Default values for source, sourcetype, and index are taken from the following settings in the following precedence order:

  1. The query string of the requests to the Splunk HEC API endpoints.
  2. The metadata of your DSP HEC token.

For example, if "source": "source2" is defined in your token, and you use that token to send data to the following endpoint which defines source=source1:

https://<DSP_HOST>:31000/services/collector?source=source1

Then the value of the source field in the resulting DSP record would be source1.

If you used an endpoint that didn't define source=source1, then the source value in the DSP record would be source2 instead.

Use query string parameters to specify default values

You can use the following query string parameters in your request to the Splunk HEC API endpoint to set default values for all events in the request. These default values are used only if the request body doesn't include source, sourcetype, or index values.

Parameter Data type Description
source string Sets a default source field value for all events in the request.
sourcetype string Sets a default sourcetype field value for all events in the request.
index string Sets a default index field value for all events in the request.

The following example demonstrates how to set default values in the query string of a request to the Splunk HEC API /services/collector endpoint:

https://<DSP_HOST>:31000/services/collector?source=source1&sourcetype=sourcetype1&index=index1

The following example shows a Splunk HEC API request with a specific sourcetype set in the request body. In this case, the sourcetype in the request body is used instead of the default sourcetype in the query string. In the resulting DSP record, the source_type field is set to sourcetype_name.

curl -k "https://<DSP_HOST>:31000/services/collector/event?source=curl&sourcetype=http&index=default" \
    -H "Authorization: Splunk <dsp-hec-token>" \
    -d '{
         "sourcetype": "sourcetype_name", 
         "event": "Hello, world!"
       }'

Use the metadata of the DSP HEC token to specify default values

You can use the metadata of your DSP HEC token to set default values for any events that are sent to the using that token. These default values are used only if both the query string and the body of the request to the Splunk HEC API endpoint don't include source, sourcetype, or index values.

The following example demonstrates how to use the Ingest service to create a token with a default sourcetype value:

curl -X POST "https://<DSP_HOST>:31000/default/ingest/v1beta2/collector/tokens" \
    -H "Authorization: Bearer <accessToken>" \
    -H "Content-Type: application/json" \
    -d '{
          "name": "<dsp-hec-token-name>",
          "sourcetype": "sourcetype_name"
         }'

As an alternative, you can use the Splunk Cloud Services CLI to create this token:

./scloud ingest post-collector-tokens --name <dsp-hec-token-name> --sourcetype "default_source_type"

The following example demonstrates how to use this token in a Splunk HEC API request to POST an event that doesn't specify a sourcetype value:

curl -k "https://<DSP_HOST>:31000/services/collector/event?source=curl&index=default" \
    -H "Authorization: Splunk <dsp-hec-token>" \
    -d '{
         "event": "Hello, world!"
       }'

In the resulting DSP record, the source_type field is set to sourcetype_name.

Use the ingest time of an event as the default value of the time field

If an event doesn't contain a time value, the timestamp field in the resulting DSP record defaults to the time when the event was ingested into DSP. This time is given in epoch time format in milliseconds.

The following example shows a Splunk HEC API request that doesn't include a time value in the request body.

curl -k "https://<DSP_HOST>:31000/services/collector/event" \
    -H "Authorization: Splunk <dsp-hec-token>" \
    -d '{
         "event": "Hello, world!"
       }'

If this event was ingested into DSP on May 19, 2021 at 6:05:57.586 PM PDT, then the timestamp field in the resulting DSP would be 1621472757586.

Last modified on 13 January, 2023
Create and manage DSP HEC tokens through the Splunk Cloud Services CLI   Connecting syslog data sources to your DSP pipeline

This documentation applies to the following versions of Splunk® Data Stream Processor: 1.2.0, 1.2.1-patch02, 1.2.1, 1.2.2-patch02, 1.2.4, 1.2.5, 1.3.0, 1.3.1


Was this topic useful?







You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters