Overview of getting data into Splunk Cloud

This topic provides an overview of the methods available to you for adding data to your Splunk Cloud deployment. For detailed information about what Splunk Cloud can index, see the Getting Data In manual.

Type of data that Splunk Cloud accepts

Splunk Cloud accepts a wide variety of data, including IT streaming, machine, and historical data such as Windows event logs, web server logs, live application logs, network feeds, system metrics, change monitoring, message queues, and archive files. Splunk Cloud can monitor relational databases and third-party infrastructures such as DB2, Cisco, Active Directory, Hadoop, and so on.

Splunk Cloud can monitor Windows-specific inputs such as the following:

• Windows Event Log
• Windows Registry
• WMI data
• Active Directory data
• Performance monitoring data

Splunk Cloud can monitor other kinds of data sources. For example:

• First-in, first-out (FIFO) queues
• Scripted inputs
• Modular inputs

Splunk offers apps and add-ons, with pre-configured inputs for specific types data sources, such as Cisco security data and Blue Coat data.

Options for getting data into Splunk Cloud

You can get data into your Splunk Cloud deployment as follows:

• Forward data from data sources
• Install Splunk apps and add-ons
• Send data using HTTP protocol

Forward data

Splunk forwarders send data from a datasource to your Splunk Cloud deployment for indexing, which makes the data searchable. Forwarders are lightweight processes, so they can usually run on the machines where the data originates. To forward data to Splunk Cloud, you typically use the Splunk universal forwarder.

For forwarder installation instructions, see the topic for your data source platform:

• Get Windows Data into Splunk Cloud
• Get *nix data into Splunk Cloud
• Forward data from files and directories to Splunk Cloud

The following diagram illustrates the topology of forwarding data from your corporate network to Splunk Cloud using the universal forwarder.

If you need to anonymize or otherwise preprocess data before it exits your enterprise network, or if a specific app or add-on that you are using does not support universal forwarders, use a heavy forwarder. For more information about heavy forwarders, see the Splunk Forwarding Data manual.

Note: By default, the universal forwarder can forward a maximum of 256 KB of data per second. As a best practice, do not exceed this limit. For more information, read Possible thruput limits in the Splunk Enterprise Troubleshooting Manual.

Use apps to get data in

Splunk apps and add-ons extend the capability and simplify the process of getting data into your Splunk platform deployment.

Apps typically target specific data types and handle everything from configuring the inputs to generating useful views of the data. For example, the Splunk App for Windows Infrastructure provides data inputs, searches, reports, alerts, and dashboards for Windows host management. The Splunk App for Unix and Linux offers the same for Unix and Linux environments. There is a wide range of apps to handle specific types of application data, including the following:

• Splunk DB Connect
• Splunk Stream
• Splunk Add-on for Amazon Web Services
• Splunk Add-on for Google Cloud Platform

Apps and add-ons that contain a data collection component should be installed on forwarders for their data collection functions.

Add data using HTTP protocol

To send HTTP events to Splunk Cloud, you can use the Splunk Java, JavaScript (Node.js), and .NET logging libraries, which are compatible with popular logging frameworks. For test and development purposes, you can use an HTTP client such as the curl utility to send events encoded in JSON.

In addition, you can send data directly to Splunk Cloud using HTTPS. To ensure that your credentials are never transmitted from your on-premises systems to Splunk Cloud, this feature uses token-based authentication. For a detailed discussion of the HTTP Event Collector, see Set Up and Use the HTTP Event Collector in the Getting Data In manual.