Data Manager

User Manual

Acrobat logo Download manual as PDF


This documentation does not apply to the most recent version of Data Manager. For documentation on the most recent version, go to the latest release.
Acrobat logo Download topic as PDF

Data ingestion mechanisms and intervals in Data Manager

The following table shows the data ingestion mechanisms and intervals in Data Manager. Use this table to verify the timing for how often your data is processed.

Data sources Data ingestion mechanism Data interval
Amazon API Gateway Pushed from Amazon CloudWatch Log Groups to Amazon Kinesis Data Firehose to the HTTP Event Collector (HEC). Immediately as soon as AWS makes data available on CloudWatch.
AWS CloudHSM Pushed from Amazon CloudWatch Log Groups to Amazon Kinesis Data Firehose to the HTTP Event Collector (HEC). Immediately as soon as AWS makes data available on CloudWatch.
Amazon Web Services (AWS) CloudTrail Pushed from Amazon CloudWatch Log Groups to Amazon Kinesis Data Firehose to the HTTP Event Collector (HEC). Immediately as soon as AWS makes data available on CloudWatch.
Amazon DocumentDB Pushed from Amazon CloudWatch Log Groups to Amazon Kinesis Data Firehose to the HTTP Event Collector (HEC). Immediately as soon as AWS makes data available on CloudWatch.
Amazon Elastic Kubernetes Service (Amazon EKS) Pushed from Amazon CloudWatch Log Groups to Amazon Kinesis Data Firehose to the HTTP Event Collector (HEC). Immediately as soon as AWS makes data available on CloudWatch.
Amazon GuardDuty Pushed from Amazon EventBridge to Amazon Kinesis Data Firehose to the HTTP Event Collector (HEC). Immediately as soon as AWS makes data available on EventBridge. By default, AWS makes the GuardDuty Findings available to EventBridge every 6 hours. These settings can be changed to every hour or every 15 minutes.
AWS Lambda Pushed from Amazon CloudWatch Log Groups to Amazon Kinesis Data Firehose to the HTTP Event Collector (HEC). Immediately as soon as AWS makes data available on CloudWatch.
AWS Metadata - AWS Identity and Access Management (IAM) Users AWS Lambda makes AWS API calls and ingests to the HTTP Event Collector (HEC) directly. Polling for existing IAM users on a scheduled rate every 3 hours.
AWS Metadata - Elastic Compute Cloud (Amazon EC2) Instances AWS Lambda makes AWS API calls and ingests to the HTTP Event Collector (HEC) directly. Polling for existing EC2 instances on a scheduled rate every 3 hours. New EC2 Instance creation events are ingested immediately to the Splunk platform.
AWS Metadata - EC2 Security Groups AWS Lambda makes AWS API calls and ingests to the HTTP Event Collector (HEC) directly. Polling for existing EC2 Security Groups on a scheduled rate every 3 hours. New EC2 Security Group creation events are ingested immediately to the Splunk platform.
AWS Metadata - Network ACLs AWS Lambda makes AWS API calls and ingests to the HTTP Event Collector (HEC) directly. Polling for existing Network ACLs on a scheduled rate every 3 hours. New Network ACL creation events are ingested immediately to the Splunk platform.
AWS IAM Access Analyzer Pushed from Amazon EventBridge to Amazon Kinesis Data Firehose to the HTTP Event Collector (HEC). Immediately as soon as AWS makes data available on EventBridge.
AWS IAM Credential Report AWS Lambda makes AWS API calls and ingests to the HTTP Event Collector (HEC) directly. Fetches and ingests the IAM Credential Report every 5 hours.
Amazon Relational Database Service (Amazon RDS) Pushed from Amazon CloudWatch Log Groups to Amazon Kinesis Data Firehose to the HTTP Event Collector (HEC). Immediately as soon as AWS makes data available on CloudWatch.
AWS Security Hub Pushed from Amazon EventBridge to Amazon Kinesis Data Firehose to the HTTP Event Collector (HEC) Immediately as soon as AWS makes data available on EventBridge.
AWS Virtual Private Cloud (VPC) Flow Logs Pushed from Amazon CloudWatch Log Groups to Amazon Kinesis Data Firehose to the HTTP Event Collector (HEC). Immediately as soon as AWS makes data available on CloudWatch.
Microsoft Azure Active Directory Data is pushed from Azure function to the HTTP Event Collector (HEC). Immediately as soon as Azure makes data available on Azure Event Hubs.
Microsoft Azure Activity Logs Data is pushed from Azure function to the HTTP Event Collector (HEC). Immediately as soon as Azure makes data available on Azure Event Hubs.
GCP Audit Logs Data is pushed from a Pub/Sub topic to the HTTP Event Collector (HEC). Immediately as soon as GCP makes data available on your GCP Project or Folder/Organization.
GCP Access Transparency Logs Data is pushed from a Pub/Sub topic to the HTTP Event Collector (HEC). Immediately as soon as GCP makes data available on your GCP Project or Folder/Organization.
Last modified on 07 September, 2022
PREVIOUS
Overview of source types for Data Manager
  NEXT
AWS prerequisites for Data Manager

This documentation applies to the following versions of Data Manager: 1.7.0


Was this documentation topic helpful?


You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters