Configure Billing inputs for the Splunk Add-on for AWS

Complete the steps to configure Billing inputs for the Splunk Add-on for Amazon Web Services (AWS):

1. You must manage accounts for the add-on as a prerequisite. See Manage accounts for the Splunk Add-on for AWS.
2. Configure AWS services for the Billing input.
3. Configure AWS permissions for the Billing input.
4. Configure Billing inputs either through Splunk Web or configuration files.

If you want to collect both a Monthly report and a Detailed report, configure two billing inputs: one for the Monthly report and another for the Detailed report. This way, you can configure the interval and the report_file_match_regex values for a specific report type rather than having the values you enter there apply to both report types.

After you configure your Billing inputs, see Access billing data for the Splunk Add-on for AWS for more information about data collection behavior and how to access the preconfigured reports included in the add-on.

Configure AWS services for the Billing input

The Splunk Add-on for AWS collects Billing Metrics through CloudWatch and Billing Reports by collecting them from an S3 bucket.

To enable AWS to produce Billing Metrics in CloudWatch, turn on Receive Billing Alerts in the Preferences section of the Billing and Cost Management console.

To enable Billing Reports, turn on Receive Billing Reports in the Preferences section of the Billing and Cost Management console. Verify your S3 bucket in the Billing and Cost Management console and select the report types that you want to collect.

For more details on managing your AWS Billing Reports, see https://docs.aws.amazon.com/cur/latest/userguide/detailed-billing.html.

Configure AWS permissions for the Billing input

You need these required permissions for the S3 bucket to collect your Billing Reports:

• Get*
• List*

In the Resource section of the policy, specify the Amazon Resource Names (ARNs) of the S3 buckets that contain billing reports for your accounts. ListAllMyBuckets is required when you use an asterisk (*) character.

See the following sample inline policy to configure Billing input permissions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:Get*",
        "s3:List*"
      ],
      "Resource": ""arn:aws:s3:::<your bucket name>"
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:ListAllMyBuckets"
      ],
      "Resource": "*"
    }
  ]
}

For more information and sample policies, see http://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/billing-permissions-ref.html.

Configure a Billing input using Splunk Web

To configure inputs using Splunk Web:

1. Click Splunk Add-on for AWS in the navigation bar on Splunk Web home.
2. Click Create New Input > Billing.
3. Fill out the fields as described in the following table:

Argument in configuration file	Field in Splunk Web	Description
AWS input configuration
aws_account	AWS account	The AWS account or EC2 IAM role the Splunk platform uses to access your Billing data. In Splunk Web, select an account from the drop-down list. In inputs.conf, enter the friendly name of one of the AWS accounts that you configured on the Configuration page or the name of the automatically discovered EC2 IAM role.
aws_iam_role	Assume Role	The IAM role to assume, see "Add and manage IAM roles" in the Manage accounts for the Splunk Add-on for AWS topic.
aws_s3_region	AWS Region (Optional)	The AWS region that contains your bucket. In inputs.conf, enter the region ID. Provide an AWS Region only if you want to use specific regional endpoints instead of public endpoints for data collection. See the AWS service endpoints topic in the AWS General Reference manual for more information.
bucket_name	S3 Bucket	The S3 bucket that is configured to hold billing reports.
monthly_report_type	Monthly report	The monthly report type that the Splunk platform collects from your AWS account. Enter one of the following values: None Monthly report Monthly cost allocation report
detail_report_type	Detailed report	The detailed report type that the Splunk platform collects from your AWS account. Enter one of the following values: None Detailed billing report Detailed billing report with resource and tags
Splunk-related configuration
initial_scan_datetime	Start Date/Time (UTC)	This add-on starts to collect data later than this time. If you leave this field empty, the default value is 90 days before the input is configured. Note: Once the input is created, you cannot change this value.
sourcetype	Source type	A source type for the events. Specify a value if you want to override the default of aws:billing. Event extraction relies on the default value of source type. If you change the default value, you must update props.conf as well.
index	Index	The index name where the Splunk platform puts the billing data. The default is main.
Advanced settings
interval	Interval	Enter the number of seconds to wait before the Splunk platform runs the command again, or enter a valid cron schedule. The default is 86400 seconds (one day). This interval applies differently for Monthly report types and Detailed report types. For Monthly report types, the interval indicates how often to run the data collection for the current month's monthly report and how often to check the previous month's monthly report's etag to determine if changes were made. If the etag does not match an already-downloaded version of the monthly report, it downloads that report to get the latest data. For Detailed report types, the interval indicates how often to check the previous month's detailed report etag to determine if changes were made. If the etag does not match a report already downloaded, it downloads that report to get the latest data. The present month is never collected until the month has ended. Because AWS Billing Reports are usually not finalized until several days after the last day of the month, you can use the cron expression 0 0 8-31 * * to skip data collection for the first seven days of every month to avoid collecting multiple copies of not-yet-finalized reports for the just-finished month.
report_file_match_reg	Regex for report selection	A regular expression that the Splunk platform uses to match reports in AWS. This expression overrides values in the monthly_report_type and detail_report_type arguments. If you want to collect both Monthly and Detailed billing reports, but you want to use regex to specify the report collection period, configure two separate billing inputs so that the regex you specify here applies only to one of the report types that you want to collect. Use this regex to limit the report collection to a certain time period to avoid collecting data that you do not need. This is particularly important for the first time that you enable the input. By default, the add-on collects all available reports for all previous months. If you collect Detailed reports, which are large in size, the add-on results in collecting a very large amount of data. You might want to limit how many months of past data that you collect. For example, you can use the expression \d+-aws-billing-detailed-line-items-201[56789]-\d+.csv.zip to collect only Detailed reports from January 2015 and later, or the expression \d+-aws-billing-detailed-line-items-with-resources-and-tags-2015-((0[4-9])|(10)|(11)|(12).csv.zip)|(\d+-aws-billing-detailed-line-items-with-resources-and-tags-201[6789]-\d+.csv.zip) to collect only the Detailed billing reports with resources and tags for April 2015 and later.
temp_folder	Temp Folder	Full path to a non-default folder with sufficient space for temporarily storing downloaded detailed billing report .zip files. Take into account the estimated size of uncompressed detailed billing report files, which can be much larger than that of zipped files. If you do not specify a temp folder, the add-on will use the system temp folder by default.

Configure a Billing input using a configuration file

To configure inputs in inputs.conf, create a stanza using the following template and add it to $SPLUNK_HOME/etc/apps/Splunk_TA_aws/local/inputs.conf. If the file or path does not exist, create it.

[aws_billing://<name>]
aws_account = <value>
aws_iam_role=<value>
aws_s3_region = <value>
interval = <value>
initial_scan_datetime = <value>
bucket_name = <value>
detail_report_type = <value>
monthly_report_type = <value>
report_file_match_reg = <value>
sourcetype = <value>
index = <value>
host_name = s3.amazonaws.com

Some of these settings have default values that can be found in $SPLUNK_HOME/etc/apps/Splunk_TA_aws/default/inputs.conf:

[aws_billing]
bucket_name =
aws_account =
monthly_report_type = Monthly cost allocation report
detail_report_type = Detailed billing report with resources and tags
report_file_match_reg =
interval = 86400
sourcetype = aws:billing
host_name = s3.amazonaws.com

The previous values correspond to the default values in Splunk Web. If you choose to copy this stanza to the /local directory and use it as a starting point to configure your inputs.conf manually, change the stanza title from aws_billing to aws_billing://<name>.