Configure Metadata inputs for the Splunk Add-on for AWS

The Description input was deprecated in version 6.2.0 of the Splunk Add-on for AWS. The Metadata input has been added as a replacement. To continue data collection for the Description input, move your workloads to the Metadata input.

Complete the steps to configure Metadata inputs for the Splunk Add-on for Amazon Web Services (AWS):

1. You must manage accounts for the add-on as a prerequisite. See Manage accounts for the Splunk Add-on for AWS.
2. Configure AWS services for the Metadata input.
3. Configure AWS permissions for the Metadata input.
4. Configure Metadata inputs either through Splunk Web or configuration files.

Configure Metadata permissions

Required permissions for IAM: GetUser, ListUsers, GetAccountPasswordPolicy, ListAccessKeys, GetAccessKeyLastUsed, ListPolicies, GetPolicyVersion, ListUserPolicies, ListAttachedUserPolicies

Required permissions for EC2 resources: DescribeInstances, DescribeReservedInstances, DescribeSnapshots, DescribeRegions, DescribeKeyPairs, DescribeNetworkAcls, DescribeSecurityGroups, DescribeSubnets, DescribeVolumes, DescribeVpcs, DescribeImages, DescribeAddresses

Required permissions for Lambda: ListFunctions

Required permissions for RDS: DescribeDBInstances

Required permissions for CloudFront, if you are in a region that supports CloudFront: ListDistributions

Required permissions for ELB: DescribeLoadBalancers, DescribeInstanceHealth, DescribeTags, DescribeTargetGroups, DescribeTargetHealth

Required permissions for S3: ListAllMyBuckets, GetAccelerateConfiguration, GetBucketCORS, GetLifecycleConfiguration, GetBucketLocation, GetBucketLogging, GetBucketTagging

See the following sample inline policy to configure Metadata input permissions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeInstances",
        "ec2:DescribeReservedInstances",
        "ec2:DescribeSnapshots",
        "ec2:DescribeRegions",
        "ec2:DescribeKeyPairs",
        "ec2:DescribeNetworkAcls",
        "ec2:DescribeSecurityGroups",
        "ec2:DescribeSubnets",
        "ec2:DescribeVolumes",
        "ec2:DescribeVpcs",
        "ec2:DescribeImages",
        "ec2:DescribeAddresses",
        "lambda:ListFunctions",
        "rds:DescribeDBInstances",
        "cloudfront:ListDistributions",
        "iam:GetUser",
        "iam:ListUsers",
        "iam:GetAccountPasswordPolicy",
        "iam:ListAccessKeys",
        "iam:GetAccessKeyLastUsed", 
        "iam:ListPolicies",
        "iam:GetPolicyVersion",
        "iam:ListUserPolicies",
        "iam:ListAttachedUserPolicies",
        "elasticloadbalancing:DescribeLoadBalancers",
        "elasticloadbalancing:DescribeInstanceHealth",
        "elasticloadbalancing:DescribeTags",
        "elasticloadbalancing:DescribeTargetGroups",
        "elasticloadbalancing:DescribeTargetHealth",
        "elasticloadbalancing:DescribeListeners",
        "s3:ListAllMyBuckets",
        "s3:GetAccelerateConfiguration",
        "s3:GetBucketCORS",
        "s3:GetLifecycleConfiguration",
        "s3:GetBucketLocation",
        "s3:GetBucketLogging",
        "s3:GetBucketTagging"
      ],
      "Resource": [
        "*"
      ]
    }
  ]
}

Configure a Metadata input using Splunk Web

To configure inputs in Splunk Web:

1. Click Splunk Add-on for AWS in the navigation bar on Splunk Web home.
2. Click Create New Input > Metadata.
3. Fill out the fields as described in the following table:

Argument in configuration file	Field in Splunk Web	Description
account	AWS Account	The AWS account or EC2 IAM role the Splunk platform uses to access your Metadata data. In Splunk Web, select an account from the drop-down list. In aws_metadata_tasks.conf, enter the friendly name of one of the AWS accounts that you configured on the Configuration page or the name of the automatically discovered EC2 IAM role.
regions	AWS Regions	The AWS regions for which you are collecting Metadata data. In Splunk Web, select one or more regions from the drop-down list. In aws_metadata_tasks.conf, enter one or more valid AWS region IDs, separated by commas. See https://docs.aws.amazon.com/general/latest/gr/rande.html#d0e371.
apis	APIs/Interval (seconds)	APIs you want to collect data from, and intervals for each API, in the format of <api name>/<api interval in seconds>,<api name>/<api interval in seconds>. The default value in Splunk Web is ec2_volumes/3600,ec2_instances/3600,ec2_reserved_instances/3600,ebs_snapshots/3600,elastic_load_balancers/3600,vpcs/3600,vpc_network_acls/3600,cloudfront_distributions/3600,vpc_subnets/3600,rds_instances/3600,ec2_key_pairs/3600,ec2_security_groups/3600 . This value collects from all of the APIs supported in this release. Set your intervals to 3,600 seconds (1 hour) or longer to avoid rate limiting errors.
aws_iam_role	Assume Role	The IAM role to assume, see Manage accounts for the Splunk Add-on for AWS.
sourcetype	Source type	A source type for the events. Enter aws:metadata.
index	Index	The index name where the Splunk platform puts the Metadata data. The default is main.

Configure a Metadata input using configuration files

To configure a Metadata input using configuration files, create $SPLUNK_HOME/etc/apps/Splunk_TA_aws/local/aws_metadata_tasks.conf using the following template:

[<name>]
account = <value>
regions = <values split by commas>
apis = <value>
aws_iam_role = <value>
sourcetype = <value>
index = <value>

Here is an example stanza that collects metadata data from all supported APIs:

[desc:splunkapp2]
account = splunkapp2
regions = us-west-2
apis = ec2_volumes/3600, ec2_instances/3600, ec2_reserved_instances/3600, ebs_snapshots/3600, classic_load_balancers/3600, application_load_balancers/3600, vpcs/3600, vpc_network_acls/3600, cloudfront_distributions/3600, vpc_subnets/3600, rds_instances/3600, ec2_key_pairs/3600, ec2_security_groups/3600, ec2_images/3600, ec2_addresses/3600, lambda_functions/3600, s3_buckets/3600, iam_users/3600, iam_list_policies/3600
aws_iam_role = iam_users
sourcetype = aws:metadata
index = default