Splunk® Enterprise

Getting Data In

Acrobat logo Download manual as PDF


Splunk Enterprise version 7.0 is no longer supported as of October 23, 2019. See the Splunk Software Support Policy for details. For information about upgrading to a supported version, see How to upgrade Splunk Enterprise.
This documentation does not apply to the most recent version of Splunk® Enterprise. For documentation on the most recent version, go to the latest release.
Acrobat logo Download topic as PDF

About the Splunk HTTP Event Collector

The Splunk HTTP Event Collector (HEC) helps you send data to Splunk Enterprise and Splunk Cloud. HEC lets you send data, application logs, and metrics over HTTP (or HTTPS) directly to Splunk Enterprise or Splunk Cloud from your application. HEC operates with tokens, which means you do not need to embed Splunk Enterprise or Splunk Cloud credentials in your app or supporting files.

To get started with using HEC, see one of the following topics:

How HTTP Event Collector works

The basics of using Splunk HTTP Event Collector follow:

  1. Turn on HTTP Event Collector by enabling its endpoint. HEC is not enabled by default.
  2. Generate an HEC token.
  3. On the client that will log to HEC, create an HTTP POST request, and set its authentication header or key/value pair to include the HEC token.
  4. POST data to the HEC token receiver.

For a walkthrough of HEC, see Use the HTTP Event Collector in Splunk Web.

You can send any kind of data to Splunk Enterprise and Splunk Cloud through HTTP Event Collector. Event data can be raw text or formatted within a JavaScript Object Notation (JSON) object. You can simplify the process by using one of the following logging libraries:

These libraries automatically package and send data to HEC in the correct format. HEC supports assigning different sourcetypes, indexes, and groups of indexers ("output groups"), so you can fine-tune where and how your data gets consumed by Splunk Enterprise or Splunk Cloud. You can use a deployment server to deploy HTTP Event Collector configuration files.

Following is a sample event, formatted in JSON according to the HEC protocol. For more information about the contents of each event packet, see Format events for HTTP Event Collector.

{
    "time": 1426279439, 
    "host": "localhost",
    "source": "datasource",
    "sourcetype": "txt",
    "index": "main",
    "event": { "hello": "world" }
}

HTTP Event Collector data flow

HEC takes data in as follows:

  1. On the data source, data is packaged using an agent such as:
    1. a Splunk logging library (Splunk logging for Java, Splunk logging for .NET, or Splunk logging for JavaScript)
    2. a JavaScript request library
    3. the Java Apache HTTP client
    4. some other client
  2. Each HTTP request is assigned the same unique token in its authorization header (or auth key/value pair), which has been generated with the management endpoint on the Splunk Enterprise or Splunk Cloud instance, using any of the following:
    1. the HTTP Event Collector UI
    2. a cURL command
    3. the Splunk Enterprise command-line interface (CLI)
    4. (managed Splunk Cloud customers only) a Splunk Support ticket
  3. The HTTP request, each of which includes the token, is sent to the appropriate Splunk Enterprise or Splunk Cloud endpoint.
  4. The token is verified against the list of known good tokens. If it's valid, an affirmative (OK) response is returned to the sender and the data is accepted by Splunk Enterprise or Splunk Cloud.
  5. Splunk Enterprise or Splunk Cloud sends the event data from the HTTP request to indexers to be indexed.

HTTP Event Collector workflow

There are three major workflows in HTTP Event Collector:

End user

An end user of HTTP Event Collector (most often a third-party app developer) simply needs to add a few lines of code to his or her app to enable it to log to HEC in Splunk Enterprise or Splunk Cloud. The easiest way to do this is to integrate the Splunk logging for Java, Splunk logging for JavaScript, or Splunk logging for .NET library into the app. If the user doesn't want to use one of the libraries, he or she must manually configure a mechanism to send event data over HTTP (or HTTPS) to the HTTP Event Collector REST API endpoint on the Splunk server.

Token admin

The token admin can be the Splunk Enterprise or e-commerce Splunk Cloud admin, or a different person who does not necessarily have experience with Splunk Enterprise or Splunk Cloud. Tokens are required for HTTP Event Collector to accept data that is sent to its port or endpoint. A token admin uses the Splunk Enterprise or Splunk Cloud management UI or Command Line Interface (CLI) to create, edit, disable, enable, and remove tokens. The token admin can also use the REST API token management endpoints to directly edit token configurations, and can enable or disable the HTTP Event Collector endpoints themselves.

Managed service Splunk Cloud customers must open a support ticket to administer HEC tokens.

Enabling HTTP Event Collector on Splunk Enterprise and e-commerce Splunk Cloud also adds a new capability: edit_token_http specifically enables roles to create and edit HTTP Event Collector tokens.

Splunk Enterprise or Splunk Cloud admin

On the Splunk Enterprise or e-commerce Splunk Cloud instance on which HTTP Event Collector is running, the admin can choose what do with and where to send the data that is sent from clients. For example, the admin can specify indexes, sourcetypes, and output groups. To do this, the admin edits the HTTP Event Collector endpoint.

Last modified on 15 December, 2020
PREVIOUS
Monitor Windows network information
  NEXT
Set up and use HTTP Event Collector in Splunk Web

This documentation applies to the following versions of Splunk® Enterprise: 7.0.0, 7.0.1, 7.0.2, 7.0.3, 7.0.4, 7.0.5, 7.0.6, 7.0.7, 7.0.8, 7.0.9, 7.0.10, 7.0.11, 7.0.13, 7.1.0, 7.1.1, 7.1.2, 7.1.3, 7.1.4, 7.1.5, 7.1.6, 7.1.7, 7.1.8, 7.1.9, 7.1.10


Was this documentation topic helpful?


You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters