Splunk® Enterprise

Getting Data In

Download manual as PDF

Download topic as PDF

Set up and use HTTP Event Collector

HTTP Event Collector (HEC) is an endpoint that lets you send application events to your Splunk deployment using the HTTP or Secure HTTP (HTTPS) protocols. HEC uses an authentication model based on tokens that you generate. You then configure a logging library or HTTP client with this token to send data to HEC in a specific format. This process eliminates the need for a forwarder when sending application events.

HEC was created with application developers in mind, so that all it takes is a few lines of code added to an app for the app to send data. Also, HEC is token-based, so you never need to hard-code your Splunk credentials in your app or supporting files.

HEC runs as a separate app called splunk_httpinput and stores its input configuration in $SPLUNK_HOME/etc/apps/splunk_httpinput/local.

For more information about getting started with HEC on Splunk Enterprise, see Getting data into HTTP Event Collector on Splunk Dev Portal.

About Event Collector Tokens

Tokens are entities that let logging agents and clients connect to the HTTP Event Collector endpoint. Each token has a token value: a 32-bit number that agents and clients use to authenticate their connections to HEC. When they connect, they present this token value. If HEC has the token value configured and it is active, HEC accepts the connection and the agent can then begin delivering its payload of application events in JavaScript Object Notation (JSON) format.

HEC receives the events and Splunk Enterprise indexes them based on the configuration of the token that the agent used to connect, using the source, source type, and index that was specified in the token. If a forwarding output group configuration exists, the application events are forwarded to other indexers as the output group defines them.

Configure HTTP Event Collector in Splunk Web

Enable HTTP Event Collector

Before you can use Event Collector to receive events through HTTP, you must enable it. If your Splunk deployment is a managed Splunk Cloud deployment, HEC must be enabled by Splunk Support before you can use it. For Splunk Enterprise, enable HEC through the "Global Settings" dialog box in the HEC management page as follows:

1. From the system bar, click Settings > Data Inputs.

2. On the left side of the page, click HTTP Event Collector. The HEC management page loads.

3. In the upper right corner, click Global Settings.

63 HTTPEC GlobalSettings.png

4. In the All Tokens toggle button, select Enabled.

5. To set the source type for all HEC tokens, select a category from the Default Source Type drop-down, then select the source type you want. You can also type in the name of the source type in the text field above the drop-down before choosing the source type.

6. To set the default index for all HEC tokens, choose an index in the Default Index drop-down.

7. To set the default forwarding output group for all HEC tokens, choose an output group from the Default Output Group drop-down.

8. To use a deployment server to handle configurations for HEC tokens, click the Use Deployment Server check box.

9. To have HEC listen and communicate over HTTPS rather than HTTP, click the Enable SSL checkbox.

10. To specify the port number that HEC listens on, enter a number in the HTTP Port Number field.

Note: To ensure that proper communication happens between logging agents and HEC, confirm that no firewall blocks the port number specified in the HTTP Port Number field, either on the agents, the Splunk instance that hosts HEC, or in between.

11. To save your settings, click Save. The dialog box disappears and Splunk Web saves the global settings and returns you to the HEC management page.

Create an Event Collector token

To use the HTTP Event Collector, you must configure at least one token. The token is what clients and agents use when they connect to Event Collector to send data.

1. From the Settings menu, select Add Data.

2. Select monitor, and then in the left pane, select HTTP Event Collector. The right pane populates with fields for HEC end point.

3. In the Name field, enter a name for the token that describes its purpose and that you will remember.

4. (Optional) In the Source name override field, enter a name for a source to be assigned to events that this endpoint generates.

5. (Optional) In the Description field, enter a description for the input.

6. (Optional) In the Output Group field, select an existing forwarder output group by picking it in the drop-down list.

Note: Define output groups in outputs.conf. See Configure forwarders with outputs.conf. You can also set up forwarding in Splunk Web, which generates a default output group called default-autolb-group.

7. (Optional) If you want to enable indexer acknowledgment for this token, click the Enable indexer acknowledgment checkbox.

Note: Indexer acknowledgement is verification from the indexer that events have been indexed. Indexer acknowledgement in HTTP Event Collector is not the same indexer acknowledgement capability described in Protect against loss of in-flight data in the Forwarding Data manual. For more information about indexer acknowledgement in HTTP Event Collector, see Enable indexer acknowledgement.

8. Click Next. The Input Settings page displays.

9. Make edits to source type and confirm the index where you want HEC events to be stored. See "Modify input settings."

10. Click Review. Confirm that all settings for the endpoint are what you want. If you need to change settings, click the gray < button at the top of the page.

11. If all settings are what you want, click Next. The success page loads and displays the token value that Event Collector generated. You can copy this token value from the displayed field and paste it into another document for reference later. See "About Event Collector tokens."

Modify an Event Collector token

63F HTTPEC EditToken.png

You can make changes to an HEC token after you have created it. Visit the HEC management page and edit a token to change any of its characteristics, including its name, description, default source type, default index, and output group.

To change the properties of a token:

1. Go to the HEC management page. From the Settings menu, select Data Inputs.

2. Select HTTP Event Collector.

3. Locate the token that you want to change in the list.

4. In the Actions column for that token, click Edit. You can also click the link to the token name.

5. Edit the description of the token by entering updated text in the Description field.

6. (Optional) Update the source value of the token by entering text in the Source field.

7. (Optional) Choose a different source type by selecting it in the Source Type drop-down. First choose a category, then select a source type in the pop-up menu that appears. You can also type in the name of the source type in the text box at the top of the drop-down.

63 HTTPEC EditToken ST.png

8. (Optional) Choose a different index by selecting it in the Available Indexes pane of the Select Allowed Indexes control. The index moves to the Selected Indexes pane of the control.

9. (Optional) Choose a different output group from the Output Group drop-down.

10. (Optional) Choose whether or not you want indexer acknowledgment enabled for the token.

11. Click Save. The dialog closes and Splunk Web returns you to the HEC management page.

Delete an Event Collector token

You can also delete an HEC token if you don't plan to use it any more. Deleting an HEC token does not affect other HEC tokens, nor does it disable the HEC endpoint.

Caution: You cannot undo this action. Agents that use this token to send data to your Splunk deployment will no longer be able to authenticate with the token. You must generate a new token and change the agent configuration to use the new token value.

To delete an HEC token:

1. Go to the HEC management page. From the Settings menu, select Data Inputs.

2. Select HTTP Event Collector.

3. Locate the token that you want to delete in the list.

4. In the Actions column for that token, click Delete.

5. In the Delete Token dialog, click Delete. Splunk Enterprise deletes the token and returns you to the HEC management page.

Enable and disable Event Collector tokens

You can enable or disable a single HEC token from within the HEC management page. Changing the status of one token does not change the status of other tokens. To enable or disable all tokens, use the Global Settings dialog. See "Enable the HTTP Event Collector."

To toggle the active status of an HEC token:

1. Go to the HEC management page.

2. Locate the token whose status you want to toggle.

3. In the Actions column for that token, click the Enable link (if the token is active) or the Disable link (if the token is inactive.) The token status toggles immediately and the link changes to Enable or Disable based on the changed token status.

Making use of HTTP Event Collector from a developer perspective

You have several options within your developer environment for using HTTP Event Collector. You can use our Java, JavaScript (Node.js) and .NET logging libraries, which are compatible with popular logging frameworks. Or you can make an HTTP request using your favorite HTTP client and send your JSON-encoded events.

Making an HTTP call with the command line using a curl command in your operating system is an easy way to test this out.

Example:

Note: This POST request is made to port 8088 and uses HTTPS for transport. The port and HTTP protocol settings can be configured independently of settings for any other servers in your deployment.
JSON

The following cURL statement uses an example HTTP Event Collector token (B5A79AAD-D822-46CC-80D1-819F80D7BFB0), and uses https://localhost as the hostname. Replace these values with your own before executing this statement.

JSON Request
curl -k  https://localhost:8088/services/collector/event -H "Authorization: Splunk B5A79AAD-D822-46CC-80D1-819F80D7BFB0" -d '{"event": "hello world"}'

Note: the key "event" is required.

JSON Response
{"text": "Success", "code": 0}

More information

You can find more developer-related content about using HTTP Event Collector in the Splunk Developer Portal. For a complete walkthrough of using HTTP Event Collector, see "HTTP Event Collector walkthrough".

PREVIOUS
Get data from TCP and UDP ports
  NEXT
How Splunk Enterprise handles syslog data over UDP

This documentation applies to the following versions of Splunk® Enterprise: 6.4.0, 6.4.1, 6.4.2, 6.4.3, 6.4.4, 6.4.5, 6.5.0, 6.5.1


Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters