Data Manager

User Manual

Configure Custom Logs in Data Manager

In your Amazon Web Services (AWS) deployment, use Amazon CloudWatch Logs to store, access and monitor logs from custom log sources. In Data Manager, use the Amazon CloudWatch Logs Custom Logs data source to ingest AWS custom logs into your Splunk Cloud platform instance.

For more information see the Enabling logging from certain AWS services topic in the AWS documentation.

Configure custom source types in Data Manager

A custom source type is a default field that identifies the data structure of an event. A source type determines how the Splunk platform formats the data during the indexing process.

Your custom source type serves as the source type for events ingested through this input. Custom source types are only supported by the Custom Logs data source. The aws:cloudwatchlogs: prefix is added to the beginning of your custom source type by default.

Configure log groups in Data Manager

Onboard log groups by specific log group names, or bulk ingest all log groups by region, or by selected common log group prefixes. Log groups cannot be onboarded more than once.

Create a log group in CloudWatch Logs

A log group is created when you install a CloudWatch Logs agent on an Amazon EC2 instance process. Log groups can also be created in the CloudWatch console.

CloudWatch Logs automatically receive log events from some AWS services. Users can also send log events to CloudWatch Logs.

For more information, see the Working with log groups and log streams topic in the Amazon CloudWatch Logs user guide.

Configure Custom Logs in Data Manager

Perform the following steps to configure custom logs in Data Manager

Click Expand to review the steps to configure custom logs in Data Manager


  1. On the Data Management page, click New Data Input.
    The Data Management page lists the status of your data inputs.
  2. On the Choose Cloud Data Platform page, select Amazon Web Services, and click Next.
    The Choose Cloud Data Platform page lists the available data sources.
  3. On the AWS Data Onboarding page, select Amazon CloudWatch Logs - Custom Logs, and click Next.
  4. On the Prerequisites for Onboarding Amazon CloudWatch Logs - Custom Logs page,
    The prerequisites page lists the prerequisites for this data input.
    1. Navigate to the Create a Role step.
      1. Click View Role Policy, and review the sample role policy.

      2. The prerequisites page lists the role policy for this data input.

      3. Click View Trust Relationship and review the sample trust relationship.
  5. Click Next.
  6. On the Input Amazon CloudWatch Logs Data Information - Custom Logs page,
    1. Enter a Data Input Name.
    2. Enter an AWS Data Account ID.
    3. In the Selected Data Sources section, select a data destination for your Custom Logs from the dropdown menu.
    4. In the Select Regions section, select the us-east-1 region.
    5. In the Enter a Custom Source Type section, enter a custom source type name. The aws:cloudwatchlogs: prefix is added to the beginning of your custom source type by default.
    6. In the Onboard log groups section, click Add groups.
      1. On the Onboard log groups page, select the log groups that you want to onboard from the dropdown menu for each available region.
        The Onboard log groups page lists the available log groups for each available region for your data input.
      2. Click Save.
    7. Click Review Data Input
  7. On the Review Data Input page, review your data input selections, and click Next.
  8. On the Setup Data Ingestion page,
    1. Navigate to the Download the CloudFormation Stack Template section, and click the Data Ingestion Template button to download the CloudFormation Stack Template that you will run in every region in your AWS deployment to establish resources for sending data from that region.
    2. In the Choose a Method to Run the Template on Your Accounts and Regions section, select either the AWS CLI or the AWS Console method, and perform the listed steps in order to run the template on your AWS account and regions.
      AWS CLI steps

      The Setup Data Ingestion page lists the steps to deploy your CloudFormation Template using the AWS CLI.
      AWS Console steps

      The Setup Data Ingestion page lists the steps to deploy your CloudFormation Template using the AWS Console.
      If you choose the AWS Console method, navigate to step four, and copy the listed Stack Name, which will be used when you navigate to your AWS deployment to create your CloudFormation stack.
    3. Once you have created your CloudFormation stack, and have run the CloudFormation template on your accounts and regions, click Review Finish Setup and Monitor Data Input.
  9. On the Data Management page, you can see the status of your data input.
Last modified on 05 September, 2024
Configure AWS for onboarding from multiple accounts   Verify the data input for AWS in Data Manager

This documentation applies to the following versions of Data Manager: 1.11.0


Was this topic useful?







You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters