Configure Billing inputs for the Splunk Add-on for AWS
Configure Billing inputs to collect Billing data (source type:
Configure a Billing input on the data collection node using one of the following ways:
- Configure a Billing input using Splunk Web (recommended)
- Configure a Billing input using configuration file
If you want to collect both a Monthly report and a Detailed report, you should configure two billing inputs: one for the Monthly report and another for the Detailed report. This way, you can configure the
interval and the
report_file_match_regex for a specific report type rather than having the values you enter there apply to both report types.
Input configuration overview
You can use the Splunk Add-on for AWS to collect data from AWS. For each supported data type, one or more input types are provided for data collection.
Follow these steps to plan and perform your AWS input configuration:
Users adding new inputs must have the
admin_all_objects role enabled.
- Click input type to go to the input configuration details.
- Follow the steps described in the input configuration details to complete the configuration.
Configure a Billing input using Splunk Web
To configure inputs using Splunk Web, click on Splunk Add-on for AWS in the left navigation bar on Splunk Web home, then click Create New Input > Billing.
|Argument in configuration file||Field in Splunk Web||Description|
|AWS Input Configuration|
||AWS account|| The AWS account or EC2 IAM role the Splunk platform uses to access your Billing data. In Splunk Web, select an account from the drop-down list. In |
||Assume Role||The IAM role to assume, see Manage IAM roles|
||S3 Bucket||The S3 bucket that is configured to hold billing reports.|
||Monthly report|| The monthly report type that the Splunk platform collects from your AWS account. Enter one of the following values:
||Detailed report|| The detailed report type that the Splunk platform collects from your AWS account. Enter one of the following values:
||Start Date/Time (UTC)||This add-on starts to collect data later than this time. If you leave this field empty, the default value is 90 days before the input is configured. |
Note: Once the input is created, this value cannot be changed.
||Source type|| A source type for the events. Specify a value if you want to override the default of |
||Index||The index name where the Splunk platform puts the billing data. The default is main.|
||Interval|| Enter the number of seconds to wait before the Splunk platform runs the command again, or a valid cron schedule. Default is 86400 seconds (one day). Note that this interval applies differently for monthly report types and detailed report types. For monthly report types, the interval indicates how often to run the data collection for the current month's monthly report AND how often to check the previous month's monthly report's etag to determine if changes were made. If the etag does not match an already-downloaded version of the monthly report, it will download that report to get the latest data. For detailed report types, the interval indicates how often to check the previous month's detailed report etag to determine if changes were made. If the etag does not match a report already downloaded, it will download that report to get the latest data -- the present month is never collected until the month has ended. |
Because AWS billing reports are usually not finalized until several days after the last day of the month, you can use the cron expression
||Regex for report selection|| A regular expression that the Splunk platform uses to match reports in AWS. This expression overrides values in the |
Use this regex to limit the report collection to a certain time period to avoid collecting data that you do not need. This is particularly important for the first time that you enable the input. By default, the add-on collects all available reports for all previous months. If you collect Detailed reports, which are large in size, this can result in a very large amount of data collection. You may wish to limit how many months of past data that you collect. For example, you can use the expression
||Temp Folder||Full path to a non-default folder with sufficient space for temporarily storing downloaded detailed billing report .zip files. Take into account the estimated size of uncompressed detailed billing report files, which can be much larger than that of zipped files. If you do not specify a temp folder, the add-on will use the system temp folder by default.|
Configure a Billing input using configuration file
To configure inputs in
inputs.conf, create a stanza using the following template and add it to
$SPLUNK_HOME/etc/apps/Splunk_TA_aws/local/inputs.conf. If the file or path does not exist, create it.
[aws_billing://<name>] aws_account = <value> aws_iam_role=<value> interval = <value> initial_scan_datetime = <value> bucket_name = <value> detail_report_type = <value> monthly_report_type = <value> report_file_match_reg = <value> sourcetype = <value> index = <value> host_name = s3.amazonaws.com
Some of these settings have default values that can be found in
[aws_billing] bucket_name = aws_account = monthly_report_type = Monthly cost allocation report detail_report_type = Detailed billing report with resources and tags report_file_match_reg = interval = 86400 sourcetype = aws:billing host_name = s3.amazonaws.com
The values above correspond to the default values in Splunk Web. If you choose to copy this stanza to
/local and use it as a starting point to configure your
inputs.conf manually, change the stanza title from
Configure data collection on your Splunk Enterprise instance
Validate your data
This documentation applies to the following versions of Splunk® Enterprise: 7.2.0, 7.2.1, 7.2.2, 7.2.3, 7.2.4, 7.2.5, 7.2.6, 7.2.7, 7.2.8, 7.3.0, 7.3.1