Splunk Cloud

Getting Data In

Download manual as PDF

Download topic as PDF

Set up and use HTTP Event Collector

The HTTP Event Collector (HEC) lets you send data and application events to a Splunk deployment over the HTTP and Secure HTTP (HTTPS) protocols. HEC uses a token-based authentication model. You can generate a token and then configure a logging library or HTTP client with the token to send data to HEC in a specific format. This process eliminates the need for a Splunk forwarder when you send application events.

After you enable HEC, as a developer, you can use HEC tokens in your app to send data to HEC. You do not need to include Splunk credentials in your app or supported files.

For information about HEC on Splunk Cloud, see Getting data into HTTP Event Collector on the Splunk Dev Portal.

HEC functionality varies based on Splunk software type

HTTP Event Collector runs on Splunk Enterprise, and self-service and managed Splunk Cloud. How it works depends on the type of Splunk instance you have.

HEC and Splunk Enterprise

HEC offers full configurability and functionality on the Splunk Enterprise platform on-premises. It offers the following additional benefits on Splunk Enterprise over the other Splunk types:

  • HEC can accept events that you send to it over the HTTP protocol in addition to the HTTPS protocol.
  • HEC can forward events to another Splunk indexer with an optional forwarding output group.
  • You can use the deployment server to distribute HEC tokens across indexers in a distributed or clustered indexer deployment.


For instructions on how to enable and manage HEC on Splunk Enterprise, see Configure HTTP Event Collector on Splunk Enterprise.

HEC and self-service Splunk Cloud

HEC offers similar functionality on self-service Splunk Cloud instances as it does on Splunk Enterprise. The following exceptions apply:

  • You cannot make changes to configuration files, because Splunk Cloud does not provide that access.
  • You cannot change the network port that HEC listens on for connections.
  • You cannot forward data that HEC receives to another set of Splunk indexers as Splunk Cloud does not support forwarding output groups.

For instructions on how to enable and manage HEC on self-service Splunk Cloud, see Configure HTTP Event Collector on self-service Splunk Cloud.

HEC and managed Splunk Cloud

HEC offers an experience on Splunk Cloud deployments that Splunk manages that is similar to the experience on self-service Splunk Cloud. The following exceptions apply:

  • You cannot make changes to configuration files, because Splunk Cloud does not provide that access.
  • You must file a ticket with Splunk Support to enable HEC.
  • You cannot make changes to global settings. You can only make settings changes to tokens that you create.
  • You cannot forward data that HEC receives to another set of Splunk indexers as Splunk Cloud does not support forwarding output groups.
  • The index that you choose to store events that HEC receives must already exist. You cannot create a new index during the setup process.
  • After you create tokens, you can monitor progress of the token as it is deployed across your managed Splunk Cloud instance.

For instructions on how to enable and manage HEC on managed Splunk Cloud, see Configure HTTP Event Collector on managed Splunk Cloud.

About Event Collector tokens

Tokens are entities that let logging agents and HTTP clients connect to the HEC input. Each token has a value, which is a 32-bit number that agents and clients use to authenticate their connections to HEC. When the clients connect, they present this token value. If HEC receives a valid token, it accepts the connection and the client can deliver its payload of application events in either text or JavaScript Object Notation (JSON) format.

HEC receives the events and Splunk Cloud indexes them based on the configuration of the token. HEC uses the source, source type, and index that was specified in the token. If a forwarding output group configuration exists on a Splunk Enterprise instance, HEC forwards the data to indexers in that output group.

Configure HTTP Event Collector on Splunk Enterprise

Enable HTTP Event Collector

Before you can use Event Collector to receive events through HTTP, you must enable it. For Splunk Enterprise, enable HEC through the Global Settings dialog box.

  1. Click Settings > Data Inputs.
  2. Click HTTP Event Collector.
  3. Click Global Settings.
    63 HTTPEC GlobalSettings.png
  4. In the All Tokens toggle button, select Enabled.
  5. (Optional) Choose a Default Source Type for all HEC tokens. You can also type in the name of the source type in the text field above the drop-down before choosing the source type.
  6. (Optional) Choose a Default Index for all HEC tokens.
  7. (Optional) Choose a Default Output Group for all HEC tokens.
  8. (Optional) To use a deployment server to handle configurations for HEC tokens, click the Use Deployment Server check box.
  9. (Optional) To have HEC listen and communicate over HTTPS rather than HTTP, click the Enable SSL checkbox.
  10. (Optional) Enter a number in the HTTP Port Number field for HEC to listen on. Note: Confirm that no firewall blocks the port number that you specified in the HTTP Port Number field, either on the clients or the Splunk instance that hosts HEC.
  11. Click Save.

Create an Event Collector token

To use HEC, you must configure at least one token.

  1. Click Settings > Add Data.
  2. Click monitor.
  3. Click HTTP Event Collector.
  4. In the Name field, enter a name for the token.
  5. (Optional) In the Source name override field, enter a source name for events that this input generates.
  6. (Optional) In the Description field, enter a description for the input.
  7. (Optional) In the Output Group field, select an existing forwarder output group.
  8. (Optional) If you want to enable indexer acknowledgment for this token, click the Enable indexer acknowledgment checkbox.
  9. Click Next.
  10. (Optional) Confirm the source type and the index for HEC events.
  11. Click Review.
  12. Confirm that all settings for the endpoint are what you want.
  13. If all settings are what you want, click Submit. Otherwise, click < to make changes.
  14. (Optional) Copy the token value that Splunk Web displays and paste it into another document for reference later.

For information about HEC tokens, see About Event Collector tokens.

For information about defining forwarding output groups, see Configure forwarders with outputs.conf. You can also set up forwarding in Splunk Web, which generates a default output group called default-autolb-group.

For information on index acknowledgement, see Enable indexer acknowledgement. Indexer acknowledgement in HTTP Event Collector is not the same indexer acknowledgement capability that comes with Splunk Cloud.

Modify an Event Collector token

63F HTTPEC EditToken.png

You can make changes to an HEC token after you have created it.

  1. Click Settings > Data Inputs.
  2. Click HTTP Event Collector.
  3. Locate the token that you want to change in the list.
  4. In the Actions column for that token, click Edit. You can also click the link to the token name.
  5. (Optional) Edit the description of the token by entering updated text in the Description field.
  6. (Optional) Update the source value of the token by entering text in the Source field.
  7. (Optional) Choose a different source type by selecting it in the Source Type drop-down.
    1. Choose a category.
    2. Select a source type in the pop-up menu that appears.
    3. (Optional) You can also type in the name of the source type in the text box at the top of the drop-down.

    63 HTTPEC EditToken ST.png

  8. (Optional) Choose a different index by selecting it in the Available Indexes pane of the Select Allowed Indexes control.
  9. (Optional) Choose a different output group from the Output Group drop-down.
  10. (Optional) Choose whether you want indexer acknowledgment enabled for the token.
  11. Click Save.

Delete an Event Collector token

You can delete an HEC token. Deleting an HEC token does not affect other HEC tokens, nor does it disable HEC.

You cannot undo this action. Clients that use this token to send data to your Splunk deployment can no longer authenticate with the token. You must generate a new token and change the client configuration to use the new token.

  1. Click Settings > Data Inputs.
  2. Click HTTP Event Collector.
  3. Locate the token that you want to delete in the list.
  4. In the Actions column for that token, click Delete.
  5. In the Delete Token dialog box, click Delete.

Enable and disable Event Collector tokens

You can enable or disable a single HEC token from within the HEC management page. Changing the status of one token does not change the status of other tokens. To enable or disable all tokens, use the Global Settings dialog. See Enable the HTTP Event Collector.

To toggle the active status of an HEC token:

  1. Click Settings > Data Inputs.
  2. Click HTTP Event Collector.
  3. In the Actions column for that token, click the Enable link, if the token is not active, or the Disable link, if the token is active. The token status toggles immediately and the link changes to Enable or Disable based on the changed token status.

Configure HTTP Event Collector on self-service Splunk Cloud

Enable HTTP Event Collector

  1. Click Settings > Data Inputs.
  2. Click HTTP Event Collector.
  3. Click Global Settings.
    66C HTTPEC GlobalSettings.png
  4. In the All Tokens toggle button, select Enabled.
  5. (Optional) Choose a Default Source Type for all HEC tokens. You can also type in the name of the source type in the text field above the drop-down before choosing the source type.
  6. (Optional) Choose a Default Index for all HEC tokens.
  7. Click Save.

Create an Event Collector token

To use HEC, you must configure at least one token.

  1. Click Settings > Add Data.
  2. Click monitor.
  3. Click HTTP Event Collector.
  4. In the Name field, enter a name for the token.
  5. (Optional) In the Source name override field, enter a name for a source to be assigned to events that this endpoint generates.
  6. (Optional) In the Description field, enter a description for the input.
  7. (Optional) If you want to enable indexer acknowledgment for this token, click the Enable indexer acknowledgment checkbox.
  8. Click Next.
  9. (Optional) Make edits to source type and confirm the index where you want HEC events to be stored. See Modify input settings.
  10. Click Review.
  11. Confirm that all settings for the endpoint are what you want.
  12. If all settings are what you want, click Submit. Otherwise, click < to make changes.
  13. (Optional) Copy the token value that Splunk Web displays and paste it into another document for reference later.

For information about HEC tokens, see About Event Collector tokens.

For information on index acknowledgement, see Enable indexer acknowledgement. Indexer acknowledgement in HTTP Event Collector is not the same indexer acknowledgement capability in Splunk Cloud.

Modify an Event Collector token

66C HTTPEC EditToken.png

You can make changes to an HEC token after you have created it.

  1. Click Settings > Data Inputs.
  2. Click HTTP Event Collector.
  3. Locate the token that you want to change in the list.
  4. In the Actions column for that token, click Edit. You can also click the link to the token name.
  5. (Optional) Edit the description of the token by entering updated text in the Description field.
  6. (Optional) Update the source value of the token by entering text in the Source field.
  7. (Optional) Choose a different source type by selecting it in the Source Type drop-down.
    1. Choose a category.
    2. Select a source type in the pop-up menu that appears.
    3. (Optional) You can also type in the name of the source type in the text box at the top of the drop-down.

    63 HTTPEC EditToken ST.png

  8. (Optional) Choose a different index by selecting it in the Available Indexes pane of the Select Allowed Indexes control.
  9. (Optional) Choose whether you want indexer acknowledgment enabled for the token.
  10. Click Save.

Delete an Event Collector token

You can delete an HEC token. Deleting an HEC token does not affect other HEC tokens, nor does it disable the HEC endpoint.

You cannot undo this action. Clients that use this token to send data to your Splunk deployment can no longer authenticate with the token. You must generate a new token and change the client configuration to use the token.

  1. Click Settings > Data Inputs.
  2. Click HTTP Event Collector.
  3. Locate the token that you want to delete in the list.
  4. In the Actions column for that token, click Delete.
  5. In the Delete Token dialog, click Delete.

Enable and disable Event Collector tokens

You can enable or disable an HEC token from within the HEC management page. Changing the status of one token does not change the status of other tokens. To enable or disable all tokens, use the Global Settings dialog. See Enable the HTTP Event Collector.

  1. Click Settings > Data Inputs.
  2. Click HTTP Event Collector.
  3. In the Actions column for that token, click the Enable link, if the token is not active, or the Disable link, if the token is active. The token status toggles and the link changes to Enable or Disable based on the changed token status.

Configure HTTP Event Collector on managed Splunk Cloud

Enable HTTP Event Collector

Call Splunk Support and open a ticket to enable HEC.

Create an Event Collector token

To use HEC, you must configure at least one token. In managed Splunk Cloud instances, the token is distributed across the deployment. The token is not ready for use until distribution has completed.

  1. Click Settings > Add Data.
  2. Click monitor.
  3. Click HTTP Event Collector.
  4. In the Name field, enter a name for the token.
  5. (Optional) In the Source name override field, enter a name for a source to be assigned to events that this endpoint generates.
  6. (Optional) In the Description field, enter a description for the input.
  7. (Optional) If you want to enable indexer acknowledgment for this token, click the Enable indexer acknowledgment checkbox.
  8. Click Next.
  9. (Optional) Make edits to source type and confirm the index where you want HEC events to be stored. See Modify input settings.
  10. Click Review.
  11. Confirm that all settings for the endpoint are what you want.
  12. If all settings are what you want, click Submit. Otherwise, click < to make changes.
  13. (Optional) Copy the token value that Splunk Web displays and paste it into another document for reference later.
  14. (Optional) Click Track deployment progress to see progress on how the token has been deployed to the rest of the Splunk Cloud deployment. When you see a status of "Done", you can then use the token to send data to HEC.

For information about HEC tokens, see About Event Collector tokens.

For information on index acknowledgement, see Enable indexer acknowledgement. Indexer acknowledgement in HTTP Event Collector is not the same indexer acknowledgement capability in Splunk Cloud.

Check Event Collector token distribution status

66M HTTPEC DeployStatus.png

You can check the distribution status of an HEC token from the HEC token page. When a distribution is in progress, the page displays "Operation in progress" and a progress bar. Otherwise the page displays "Last deployment status."

  1. Click Settings > Data Inputs.
  2. Click HTTP Event Collector.
  3. Click Operation in progress or Last deployment status.
  4. View the status of the token distribution.
  5. Click Close.

Modify an Event Collector token

You can make changes to an HEC token after it has been created.

  1. Click Settings > Data Inputs.
  2. Click HTTP Event Collector.
  3. Locate the token that you want to change in the list.
  4. In the Actions column for that token, click Edit. You can also click the link to the token name.
  5. (Optional) Edit the description of the token by entering updated text in the Description field.
  6. (Optional) Update the source value of the token by entering text in the Source field.
  7. (Optional) Choose a different source type by selecting it in the Source Type drop-down.
    1. Choose a category.
    2. Select a source type in the pop-up menu that appears.
    3. (Optional) You can also type in the name of the source type in the text box at the top of the drop-down.
  8. (Optional) Choose a different index by selecting it in the Available Indexes pane of the Select Allowed Indexes control.
  9. (Optional) Choose whether you want indexer acknowledgment enabled for the token.
  10. Click Save.

Delete an Event Collector token

You can delete an HEC token. Deleting an HEC token does not affect other HEC tokens, nor does it disable the HEC endpoint.

You cannot undo this action. Clients that use this token to send data to your Splunk deployment can no longer authenticate with the token. You must generate a new token and change the client configuration to use the new value.

  1. Click Settings > Data Inputs.
  2. Click HTTP Event Collector.
  3. Locate the token that you want to delete in the list.
  4. In the Actions column for that token, click Delete.
  5. In the Delete Token dialog, click Delete.

Enable and disable Event Collector tokens

You can enable or disable a token from within the HEC management page. Changing the active status of one token does not change the status of other tokens.

  1. Click Settings > Data Inputs.
  2. Click HTTP Event Collector.
  3. In the Actions column for a token, click the Enable link, if the token is not active, or the Disable link, if the token is active. The token status toggles and the link changes to Enable or Disable based on the changed token status.

Send data to HTTP Event Collector

You must satisfy all of the following conditions when you send data to HEC:

  • HEC must be enabled
  • You must have at least one active HEC token available
  • You must use an active token to authenticate into HEC
  • You must format the data that goes to HEC in a certain way. See Format events for HTTP Event Collector

There are several options for sending data to HTTP Event Collector:

  • You can make an HTTP request using your favorite HTTP client and send your JSON-encoded events.
  • As a developer, you can use the Java, JavaScript (node.js), and .NET logging libraries in your application to send data to HEC. These libraries are compatible with popular logging frameworks. See Java, JavaScript (Node.js), and .NET on the Splunk Dev Portal.

Send data to HTTP Event Collector on Splunk Enterprise

You send data to a specific Uniform Resource Indicator (URI) for HEC.

The standard form for the HEC URI in Splunk Enterprise is as follows:

<protocol>://<host>:<port>/<endpoint>

Where:

  • <protocol> is either http or https
  • <host> is the Splunk instance that runs HEC
  • <port> is the HEC port number, which is 8088 by default, but you can change in the HEC Global Settings
  • <endpoint> is the HEC endpoint you want to use. In many cases, you use the /services/collector endpoint for JavaScript Object Notation (JSON)-formatted events or the services/collector/raw endpoint for raw events

Send data to HTTP Event Collector on Splunk Cloud instances

Depending on the type of Splunk Cloud that you use, you must send data using a specific URI for HEC.

The standard form for the HEC URI in self-service Splunk Cloud is as follows:

<protocol>://input-<host>:<port>/<endpoint>

The standard form for the HEC URI in managed Splunk Cloud is as follows:

<protocol>://http-inputs-<host>:<port>/<endpoint>

Where:

  • <protocol> is either http or https
  • <host> is the Splunk instance that runs HEC
  • <port> is the HEC port number
    • 8088 on self-service Splunk Cloud instances
    • 443 on managed Splunk Cloud instances
  • <endpoint> is the HEC endpoint you want to use. In many cases, you use the /services/collector endpoint for JavaScript Object Notation (JSON)-formatted events or the services/collector/raw endpoint for raw events
  • For self-service Splunk Cloud plans, you must pre-pend the hostname with input-
  • For managed Splunk Cloud plans, pre-pend the hostname with http-inputs-

If you do not include these prefixes before your Splunk Cloud hostname when you send data, the data cannot reach HEC.

Example of sending data to HEC with an HTTP request

The following example makes a HTTP POST request to the HEC on port 8088 and uses HTTPS for transport. This example uses the curl command to generate the request, but you can use a command line or other tool that better suits your needs.

You can configure the network port and HTTP protocol settings independently of settings for other instances of HEC in your Splunk Enterprise or self-service Splunk Cloud deployment.

The following cURL command uses an example HTTP Event Collector token (B5A79AAD-D822-46CC-80D1-819F80D7BFB0), and uses https://localhost as the hostname. Replace these values with your own before running this command.

JSON request and response

When you make a JSON request to send data to HEC, you must specify the "event" key in the command.

curl -k  https://hec.example.com:8088/services/collector/event -H "Authorization: Splunk B5A79AAD-D822-46CC-80D1-819F80D7BFB0" -d '{"event": "hello world"}'
{"text": "Success", "code": 0}

More information on HEC for developers

PREVIOUS
Get data from TCP and UDP ports
  NEXT
How Splunk Enterprise handles syslog data over UDP

This documentation applies to the following versions of Splunk Cloud: 6.6.0, 6.6.1, 6.6.3


Comments

Hi Pdehlke,

The pertinent information from that page has been added to this page. Thank you for the heads up.

Malmoore, Splunker
October 17, 2017

For cloud instances, you will have to specify an HEC-specific URI for your inputs. This is not documented on this page at all, but can be found here: http://dev.splunk.com/view/event-collector/SP-CAAAE7G

Pdehlke
October 16, 2017

Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters