On October 30, 2022, all 1.2.x versions of the Splunk Data Stream Processor will reach its end of support date. See the Splunk Software Support Policy for details.
Set the default field values in DSP HEC
When you send data to through the DSP HTTP Event Collector (HEC), if the data doesn't include values for the source
, sourcetype
, index
, or time
fields, then default values are used. The time
field defaults to the ingest time of the event, but you can specify the default values for the source
, sourcetype
, and index
fields.
The source
, sourcetype
, index
, and time
values in your requests to the Splunk HEC API endpoints map to DSP record fields as follows:
Request body field | DSP record field |
---|---|
source
|
source
|
sourcetype
|
source_type
|
index
|
An index key in the attributes field.
|
time
|
timestamp
|
Precedence order of default values
Default values for source
, sourcetype
, and index
are taken from the following settings in the following precedence order:
- The query string of the requests to the Splunk HEC API endpoints.
- The metadata of your DSP HEC token.
For example, if "source": "source2"
is defined in your token, and you use that token to send data to the following endpoint which defines source=source1
:
https://<DSP_HOST>:31000/services/collector?source=source1
Then the value of the source
field in the resulting DSP record would be source1
.
If you used an endpoint that didn't define source=source1
, then the source
value in the DSP record would be source2
instead.
Use query string parameters to specify default values
You can use the following query string parameters in your request to the Splunk HEC API endpoint to set default values for all events in the request. These default values are used only if the request body doesn't include source
, sourcetype
, or index
values.
Parameter | Data type | Description | |
---|---|---|---|
source | string | Sets a default source field value for all events in the request.
| |
sourcetype | string | Sets a default sourcetype field value for all events in the request.
| |
index | string | Sets a default index field value for all events in the request.
|
The following example demonstrates how to set default values in the query string of a request to the Splunk HEC API /services/collector
endpoint:
https://<DSP_HOST>:31000/services/collector?source=source1&sourcetype=sourcetype1&index=index1
The following example shows a Splunk HEC API request with a specific sourcetype
set in the request body. In this case, the sourcetype
in the request body is used instead of the default sourcetype
in the query string. In the resulting DSP record, the source_type
field is set to sourcetype_name
.
curl -k "https://<DSP_HOST>:31000/services/collector/event?source=curl&sourcetype=http&index=default" \ -H "Authorization: Splunk <dsp-hec-token>" \ -d '{ "sourcetype": "sourcetype_name", "event": "Hello, world!" }'
Use the metadata of the DSP HEC token to specify default values
You can use the metadata of your DSP HEC token to set default values for any events that are sent to the using that token. These default values are used only if both the query string and the body of the request to the Splunk HEC API endpoint don't include source
, sourcetype
, or index
values.
The following example demonstrates how to use the Ingest service to create a token with a default sourcetype
value:
curl -X POST "https://<DSP_HOST>:31000/default/ingest/v1beta2/collector/tokens" \ -H "Authorization: Bearer <accessToken>" \ -H "Content-Type: application/json" \ -d '{ "name": "<dsp-hec-token-name>", "sourcetype": "sourcetype_name" }'
As an alternative, you can use the Splunk Cloud Services CLI to create this token:
./scloud ingest post-collector-tokens --name <dsp-hec-token-name> --sourcetype "default_source_type"
The following example demonstrates how to use this token in a Splunk HEC API request to POST an event that doesn't specify a sourcetype
value:
curl -k "https://<DSP_HOST>:31000/services/collector/event?source=curl&index=default" \ -H "Authorization: Splunk <dsp-hec-token>" \ -d '{ "event": "Hello, world!" }'
In the resulting DSP record, the source_type
field is set to sourcetype_name
.
Use the ingest time of an event as the default value of the time field
If an event doesn't contain a time
value, the timestamp
field in the resulting DSP record defaults to the time when the event was ingested into DSP. This time is given in epoch time format in milliseconds.
The following example shows a Splunk HEC API request that doesn't include a time
value in the request body.
curl -k "https://<DSP_HOST>:31000/services/collector/event" \ -H "Authorization: Splunk <dsp-hec-token>" \ -d '{ "event": "Hello, world!" }'
If this event was ingested into DSP on May 19, 2021 at 6:05:57.586 PM PDT, then the timestamp
field in the resulting DSP would be 1621472757586
.
Create and manage DSP HEC tokens through the Splunk Cloud Services CLI | Connecting syslog data sources to your DSP pipeline |
This documentation applies to the following versions of Splunk® Data Stream Processor: 1.2.0, 1.2.1-patch02, 1.2.1, 1.2.2-patch02, 1.2.4, 1.2.5, 1.3.0, 1.3.1
Feedback submitted, thanks!