Data Manager

User Manual

This documentation does not apply to the most recent version of Data Manager. For documentation on the most recent version, go to the latest release.

Data ingestion mechanisms and intervals in Data Manager

The following table shows the data ingestion mechanisms and intervals in Data Manager. Use this table to verify the timing for how often your data is processed.

Data Sources Data Ingestion Mechanism Data Interval
Amazon API Gateway Pushed from Amazon CloudWatch Log Groups to Amazon Kinesis Data Firehose to the HTTP Event Collector (HEC). Immediately as soon as AWS makes data available on CloudWatch.
AWS CloudHSM Pushed from Amazon CloudWatch Log Groups to Amazon Kinesis Data Firehose to the HTTP Event Collector (HEC). Immediately as soon as AWS makes data available on CloudWatch.
Amazon Web Services (AWS) CloudTrail Pushed from Amazon CloudWatch Log Groups to Amazon Kinesis Data Firehose to the HTTP Event Collector (HEC). Immediately as soon as AWS makes data available on CloudWatch.
Amazon DocumentDB Pushed from Amazon CloudWatch Log Groups to Amazon Kinesis Data Firehose to the HTTP Event Collector (HEC). Immediately as soon as AWS makes data available on CloudWatch.
Amazon Elastic Kubernetes Service (Amazon EKS) Pushed from Amazon CloudWatch Log Groups to Amazon Kinesis Data Firehose to the HTTP Event Collector (HEC). Immediately as soon as AWS makes data available on CloudWatch.
Amazon GuardDuty Pushed from Amazon EventBridge to Kinesis Data Firehose to HEC. Immediately as soon as AWS makes data available on EventBridge. By default, AWS makes the GuardDuty Findings available to CloudWatch EventBridge every 6 hours. These settings can be changed to every 1 hour or every 15 minutes.
AWS Lambda Pushed from Amazon CloudWatch Log Groups to Amazon Kinesis Data Firehose to the HTTP Event Collector (HEC). Immediately as soon as AWS makes data available on CloudWatch.
AWS Metadata - AWS Identity and Access Management (IAM) Access Analyzer Users AWS Lambda makes AWS API calls and ingests to HEC directly. Polling for existing IAM users on a scheduled rate every 1 hour.
AWS Metadata - Elastic Compute Cloud (Amazon EC2) Instances Lambda makes AWS API calls and ingests to HEC directly. Polling for existing EC2 instances on a scheduled rate every 3 hours. New EC2 Instance creation events are ingested immediately to Splunk.
AWS Metadata - EC2 Security Groups Lambda makes AWS API calls and ingests to HEC directly. Polling for existing EC2 Security Groups on a scheduled rate every 3 hours. New EC2 Security Group creation events are ingested immediately to Splunk.
AWS Metadata - Network ACLs Lambda makes AWS API calls and ingests to HEC directly. Polling for existing Network ACLs on a scheduled rate every 3 hours. New Network ACL creation events are ingested immediately to Splunk.
AWS IAM Access Analyzer Pushed from Cloudwatch to EventBridge to Kinesis Firehose to HEC. Immediately as soon as AWS makes data available on EventBridge.
AWS IAM Credential Report Lambda makes AWS API calls and ingests to HEC directly. Fetches and ingests the IAM Credential Report every 4 hours.
Amazon Relational Database Service (Amazon RDS) Pushed from Amazon CloudWatch Log Groups to Amazon Kinesis Data Firehose to the HTTP Event Collector (HEC). Immediately as soon as AWS makes data available on CloudWatch.
AWS Security Hub Pushed from EventBridge to Kinesis Data Firehose to HEC. Immediately as soon as AWS makes data available on Amazon EventBridge.
Last modified on 13 January, 2022
Overview of source types for Data Manager   AWS Kinesis Data Firehose prerequisites for Data Manager

This documentation applies to the following versions of Data Manager: 1.3.1


Was this topic useful?







You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters