Data ingestion mechanisms and intervals in Data Manager

The following table shows the data ingestion mechanisms and intervals in Data Manager. Use this table to verify the timing for how often your data is processed.

Data sources	Data ingestion mechanism	Data interval
Amazon API Gateway	Pushed from Amazon CloudWatch Log Groups to Amazon Kinesis Data Firehose to the HTTP Event Collector (HEC).	Immediately as soon as AWS makes data available on CloudWatch.
AWS CloudHSM	Pushed from Amazon CloudWatch Log Groups to Amazon Kinesis Data Firehose to the HTTP Event Collector (HEC).	Immediately as soon as AWS makes data available on CloudWatch.
Amazon Web Services (AWS) CloudTrail	Pushed from Amazon CloudWatch Log Groups to Amazon Kinesis Data Firehose to the HTTP Event Collector (HEC).	Immediately as soon as AWS makes data available on CloudWatch.
Amazon DocumentDB	Pushed from Amazon CloudWatch Log Groups to Amazon Kinesis Data Firehose to the HTTP Event Collector (HEC).	Immediately as soon as AWS makes data available on CloudWatch.
Amazon Elastic Kubernetes Service (Amazon EKS)	Pushed from Amazon CloudWatch Log Groups to Amazon Kinesis Data Firehose to the HTTP Event Collector (HEC).	Immediately as soon as AWS makes data available on CloudWatch.
Amazon GuardDuty	Pushed from Amazon EventBridge to Amazon Kinesis Data Firehose to the HTTP Event Collector (HEC).	Immediately as soon as AWS makes data available on EventBridge. By default, AWS makes the GuardDuty Findings available to EventBridge every 6 hours. These settings can be changed to every hour or every 15 minutes.
AWS Lambda	Pushed from Amazon CloudWatch Log Groups to Amazon Kinesis Data Firehose to the HTTP Event Collector (HEC).	Immediately as soon as AWS makes data available on CloudWatch.
AWS Metadata - AWS Identity and Access Management (IAM) Users	AWS Lambda makes AWS API calls and ingests to the HTTP Event Collector (HEC) directly.	Polling for existing IAM users on a scheduled rate every 3 hours.
AWS Metadata - Elastic Compute Cloud (Amazon EC2) Instances	AWS Lambda makes AWS API calls and ingests to the HTTP Event Collector (HEC) directly.	Polling for existing EC2 instances on a scheduled rate every 3 hours. New EC2 Instance creation events are ingested immediately to the Splunk platform.
AWS Metadata - EC2 Security Groups	AWS Lambda makes AWS API calls and ingests to the HTTP Event Collector (HEC) directly.	Polling for existing EC2 Security Groups on a scheduled rate every 3 hours. New EC2 Security Group creation events are ingested immediately to the Splunk platform.
AWS Metadata - Network ACLs	AWS Lambda makes AWS API calls and ingests to the HTTP Event Collector (HEC) directly.	Polling for existing Network ACLs on a scheduled rate every 3 hours. New Network ACL creation events are ingested immediately to the Splunk platform.
AWS IAM Access Analyzer	Pushed from Amazon EventBridge to Amazon Kinesis Data Firehose to the HTTP Event Collector (HEC).	Immediately as soon as AWS makes data available on EventBridge.
AWS IAM Credential Report	AWS Lambda makes AWS API calls and ingests to the HTTP Event Collector (HEC) directly.	Fetches and ingests the IAM Credential Report every 5 hours.
Amazon Relational Database Service (Amazon RDS)	Pushed from Amazon CloudWatch Log Groups to Amazon Kinesis Data Firehose to the HTTP Event Collector (HEC).	Immediately as soon as AWS makes data available on CloudWatch.
AWS Security Hub	Pushed from Amazon EventBridge to Amazon Kinesis Data Firehose to the HTTP Event Collector (HEC)	Immediately as soon as AWS makes data available on EventBridge.
AWS Virtual Private Cloud (VPC) Flow Logs	Pushed from Amazon CloudWatch Log Groups to Amazon Kinesis Data Firehose to the HTTP Event Collector (HEC).	Immediately as soon as AWS makes data available on CloudWatch.
Microsoft Azure Active Directory	Data is pushed from Azure function to the HTTP Event Collector (HEC).	Immediately as soon as Azure makes data available on Azure Event Hubs.
Microsoft Azure Activity Logs	Data is pushed from Azure function to the HTTP Event Collector (HEC).	Immediately as soon as Azure makes data available on Azure Event Hubs.
GCP Audit Logs	Data is pushed from a Pub/Sub topic to the HTTP Event Collector (HEC).	Immediately as soon as GCP makes data available on your GCP Project or Folder/Organization.
GCP Access Transparency Logs	Data is pushed from a Pub/Sub topic to the HTTP Event Collector (HEC).	Immediately as soon as GCP makes data available on your GCP Project or Folder/Organization.
AWS CloudTrail (S3)	Connect Amazon S3 to your Splunk Cloud deployment as a pull-based data source.	Configurable with the Splunk Web timerange picker.
AWS S3 access logs (S3)	Connect Amazon S3 to your Splunk Cloud deployment as a pull-based data source.	Configurable with the Splunk Web timerange picker.
AWS Elastic Load Balancer (ELB) access logs (S3)	Connect Amazon S3 to your Splunk Cloud deployment as a pull-based data source.	Configurable with the Splunk Web timerange picker.
AWS CloudFront (CF) access logs (S3)	Connect Amazon S3 to your Splunk Cloud deployment as a pull-based data source.	Configurable with the Splunk Web timerange picker.