Splunk® Supported Add-ons

Splunk Add-on for AWS

Download manual as PDF

Download topic as PDF

Release history for the Splunk Add-on for AWS

Latest release

The latest version of the Splunk Add-on for Amazon Web Services is version 4.6.0. See Release notes for the Splunk Add-on for AWS for the release notes of this latest version.

Version 4.5.0

Version 4.5.0 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms.

Splunk platform versions 6.5 and later
CIM 4.3 and later
Supported OS for data collection Platform independent
Vendor products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, Inspector, Kinesis, S3, VPC Flow Logs, Billing services, SQS, and SNS.

New features

Version 4.5.0 of the Splunk Add-on for AWS version contains the following new and changed features:

  • Support for the configuration of billing inputs to collect Cost and Usage Report data (sourcetype: aws:billing:cur).

Fixed issues

Version 4.5.0 of the Splunk Add-on for Amazon Web Services fixes the following issues.

Date resolved Issue number Description
2018-01-22 ADDON-15918 AWS TA is unable to validate role ARNs with "/" in path
2018-01-22 ADDON-16435 AWS - Getting error trying to connect to CloudTrail using SQS Based S3 - EU-WEST-1

Known issues

Version 4.5.0 of the Splunk Add-on for Amazon Web Services has the following known issues.

Date filed Issue number Description
2018-08-22 ADDON-19171 Cannot add regions when configuring Inspector inputs for the TA for AWS
2018-05-17 ADDON-18087, SII-1746 Invalid AWS credentials can be added and interacted with as valid AWS credentials
2018-05-09 ADDON-18031 Small page size causing LimitExceededException error during Kinesis ListStreams operations
2018-03-28 ADDON-17571 AWS TA and *nix TA lack spec files for eventgen.conf, which causes cluster bundle validation errors, and breaks Manage Indexes page in clustered Splunk Cloud

Workaround:
Splunk Cloud customers who cannot create indexes on their own due to this bug should file a support case when they need new indexes created.
2018-02-27 ADDON-17277 Line/event breaking enforcement for ELB/S3 Access Logs
2018-02-19 ADDON-17158 The style of multi-input text box is not correct
2018-02-19 ADDON-17157 The header view of customized page is inconsistent with the default NightLight style
2018-02-13 ADDON-17135 Placeholder tooltip is missing for dropdown
2018-02-13 ADDON-17132 Create/edit input page layout is broken
2018-01-05 ADDON-16518 When kinesis and cloudwatch inputs send large volumes of data over HEC, HEC can block the ingest pipeline, which breaks non-HEC inputs.

Workaround:
Set use_hec=false in [global_settings] stanza of aws_kinesis.conf and/or aws_cloudwatch.conf
2017-09-03 ADDON-15718 Duplicate cloudfront data in description when there are more than 1 regions
2017-09-01 ADDON-15712 It stops pulling Kinesis stream data when the Kinesis stream is resharded
2017-08-22 ADDON-15603 Users can delete an account in use.
2016-12-22 ADDON-12867, ADDON-11894 S3 input: large key numbers lead to excessively large checkpoint files

Third-party software attributions

Version 4.5.0 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Version 4.4.0

Version 4.4.0 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms.

Splunk platform versions 6.5 and later
CIM 4.3 and later
Platforms Platform independent
Vendor Products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, Inspector, Kinesis, S3, VPC Flow Logs, Billing services, SQS, and SNS.

New features

Version 4.4.0 of the Splunk Add-on for AWS version contains the following new and changed features:

  • Splunk Add-on for AWS 4.4.0 is only compatible with Splunk App for AWS 5.1.0. Previous versions of Splunk App for AWS are not supported.
  • Optimized Web UI for better usability and more streamlined configuration workflow
    • The Create New Input menu has been redesigned with all the menu options organized by the type of data to collect.
    • Two separate configuration pages are now available for Generic S3 and Incremental S3 input types respectively. Previously, the two different input types were configured in one configuration page.
    • Input configuration fields are now grouped into AWS Input Configuration, Splunk-related Configuration, and Advanced Settings sections on the Web UI.
    • Redesigned input configuration UIs for CloudWatch and Config input types let you create multiple inputs all at once.
  • Added a new Temp Folder setting to the Billing input type configuration, which lets you specify a non-default folder for temporarily storing downloaded detailed billing report .zip files when the system default temp folder does not provide sufficient space.
  • You can now configure SQS-based S3 inputs to index non-AWS custom logs in plain text in addition to its supported AWS log types.
  • SQS-based S3 input type now supports CloudTrail and Config SQS notifications.
  • Assume Role is now supported in SQS, Config Rule, and Inspector input types.
  • The Description input type now supports the iam_users service.

Upgrade

To upgrade from versions 4.3 and below, AWS users must be given permission to use the ec2:RunInstances API action, and depending on deployment, the following API actions:

API Action Description
ec2:DescribeImages Allows users to view and select an AMI.
ec2:DescribeVpcs Allows users to view the available EC2-Classic and virtual private clouds (VPCs) network options. This API action is required even if you are not launching into a VPC.
ec2:DescribeSubnets Allows users to view all available subnets for the chosen VPC, when launching into a VPC.
ec2:DescribeSecurityGroups Allows users to view the security groups page in the wizard. Users can select an existing security group.
ec2:DescribeKeyPairs or ec2:CreateKeyPair Allows users to select an existing key pair, or create a new key pair.

See Configure Description permissions for more information on how to configure AWS permissions.

See the AWS documentation for more information on the DescribeImages function. https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeImages.html.

Fixed issues

Version 4.4.0 of the Splunk Add-on for Amazon Web Services fixes the following issues.

Date resolved Issue number Description
2017-08-03 ADDON-14890 Add-on truncates Kinesis stream dropdown to 20 items.
2017-07-27 ADDON-12700 Pagination issue in Account page.
2017-07-11 ADDON-11974 Cannot get CloudWatch data using some default configuration in Add-on
2017-05-25 ADDON-13282 Cannot change Description interval in UI more than once

Known issues

Version 4.4.0 of the Splunk Add-on for Amazon Web Services has the following known issues.

Date filed Issue number Description
2018-08-22 ADDON-19171 Cannot add regions when configuring Inspector inputs for the TA for AWS
2018-05-17 ADDON-18087, SII-1746 Invalid AWS credentials can be added and interacted with as valid AWS credentials
2018-03-28 ADDON-17571 AWS TA and *nix TA lack spec files for eventgen.conf, which causes cluster bundle validation errors, and breaks Manage Indexes page in clustered Splunk Cloud

Workaround:
Splunk Cloud customers who cannot create indexes on their own due to this bug should file a support case when they need new indexes created.
2018-02-27 ADDON-17277 Line/event breaking enforcement for ELB/S3 Access Logs
2018-01-05 ADDON-16518 When kinesis and cloudwatch inputs send large volumes of data over HEC, HEC can block the ingest pipeline, which breaks non-HEC inputs.

Workaround:
Set use_hec=false in [global_settings] stanza of aws_kinesis.conf and/or aws_cloudwatch.conf
2017-12-20 ADDON-16435 AWS - Getting error trying to connect to CloudTrail using SQS Based S3 - EU-WEST-1
2017-09-21 ADDON-15918 AWS TA is unable to validate role ARNs with "/" in path
2017-09-03 ADDON-15718 Duplicate cloudfront data in description when there are more than 1 regions
2017-09-01 ADDON-15712 It stops pulling Kinesis stream data when the Kinesis stream is resharded
2017-08-22 ADDON-15603 Users can delete an account in use.
2017-08-19 ADDON-15578 On Windows, fails to rotate CloudWatch and Incremental S3 logs when indexing speed cannot catch up with data collection
2017-07-25 ADDON-15371 Add-on should support non-UTF fields in access logs.
2017-03-29 ADDON-14287 After you replace an IAM role attached to an EC2 instance, the inputs that use the old IAM role stop collecting data.
2016-12-22 ADDON-12867, ADDON-11894 S3 input: large key numbers lead to excessively large checkpoint files

Third-party software attributions

Version 4.4.0 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Version 4.3.0

Version 4.3.0 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms.

Splunk platform versions 6.4 and later
CIM 4.3 and later
Platforms Platform independent
Vendor Products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, Inspector, Kinesis, S3, VPC Flow Logs, Billing services, SQS, and SNS.

New features

Version 4.3.0 of the Splunk Add-on for AWS contains the following new and changed features:

  • SQS-based S3 input type
    A multi-purpose input type that collects several types of logs in response to messages polled from SQS queues. A scalable and higher-performing alternative to the generic S3 and incremental S3 input types. See Multi-purpose input types.
  • Heath Check dashboards
    Health Overview and S3 Health dashboards to help you troubleshoot data collection errors and performance issues. See Health Check dashboards.
  • Optimized logging. See Internal logs.

Fixed issues

Version 4.3.0 of the Splunk Add-on for Amazon Web Services fixes the following issues.

Date resolved Issue number Description
2017-06-09 ADDON-13860 Configuring more AWS accounts increases CPU usage and lowers throughput performance due to increased API calls
2017-06-07 ADDON-13865 Cannot disable/enable inputs under sc_admin role in Splunk Cloud
2017-05-10 ADDON-14039 Incremental S3 input fails to decode non-utf8 encoded files
2017-05-10 ADDON-13651 Describe EC2 is blocked by API throttling of get EBS snapshot data
2017-03-23 ADDON-13492, ADDON-13015, ADDON-13855 Ingesting a continuous stream of large files (e.g., 20MB) from a single incremental S3 data input may cause out-of-memory error
2017-03-06 ADDON-11846, SPL-138046 Logging breaks on rotation when multiple inputs write to the same log. If > 6 inputs, some inputs cannot log
2017-02-28 ADDON-13867 Major performance issue for incremental S3 data inputs when ingesting large plain text files (max throughput only around 4MB/s for files of size 20MB)

Known issues

Version 4.3.0 of the Splunk Add-on for Amazon Web Services has the following known issues.

Date filed Issue number Description
2018-01-05 ADDON-16518 When kinesis and cloudwatch inputs send large volumes of data over HEC, HEC can block the ingest pipeline, which breaks non-HEC inputs.

Workaround:
Set use_hec=false in [global_settings] stanza of aws_kinesis.conf and/or aws_cloudwatch.conf
2017-09-21 ADDON-15918 AWS TA is unable to validate role ARNs with "/" in path
2017-09-03 ADDON-15718 Duplicate cloudfront data in description when there are more than 1 regions
2017-09-01 ADDON-15712 It stops pulling Kinesis stream data when the Kinesis stream is resharded
2017-07-25 ADDON-15371 Add-on should support non-UTF fields in access logs.
2017-06-29 ADDON-15188 Too long input name lead to modular input failure
2017-03-29 ADDON-14287 After you replace an IAM role attached to an EC2 instance, the inputs that use the old IAM role stop collecting data.
2016-12-22 ADDON-12867, ADDON-11894 S3 input: large key numbers lead to excessively large checkpoint files
2016-12-14 ADDON-12700 Pagination issue in Account page.

Third-party software attributions

Version 4.3.0 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Version 4.2.3

Version 4.2.3 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms.

Splunk platform versions 6.4 and later
CIM 4.3 and later
Platforms Platform independent
Vendor Products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, Inspector, Kinesis, S3, VPC Flow Log, Billing services, SQS, and SNS.

New features

Version 4.2.3 of the Splunk Add-on for AWS does not contain any new features.

Fixed issues

Version 4.2.3 of the Splunk Add-on for Amazon Web Services fixes the following issues.

Date resolved Issue number Description
2017-04-26 ADDON-13891 The S3 incremental input fails to skip the Glacier storage type keys
2017-04-16 ADDON-11326 Unexpected timestamp format blocks data ingestion
2017-04-06 ADDON-13768 Upgrading the add-on causes the EC2 configuration in the Splunk App for AWS to fail with IAM Role

Known issues

Version 4.2.3 of the Splunk Add-on for Amazon Web Services has the following known issues.

Date filed Issue number Description
2017-09-03 ADDON-15718 Duplicate cloudfront data in description when there are more than 1 regions
2017-09-01 ADDON-15712 It stops pulling Kinesis stream data when the Kinesis stream is resharded
2017-06-22 ADDON-15124 Logging breaks on rotation for Billing & AWS Config when multiple inputs write to the same log.
2017-05-24 ADDON-14890 Add-on truncates Kinesis stream dropdown to 20 items.
2017-03-29 ADDON-14287 After you replace an IAM role attached to an EC2 instance, the inputs that use the old IAM role stop collecting data.
2017-03-09 ADDON-14038 Orphan process issue after master process been force killed
2017-03-09 ADDON-14039 Incremental S3 input fails to decode non-utf8 encoded files
2017-02-27 ADDON-13865 Cannot disable/enable inputs under sc_admin role in Splunk Cloud
2017-02-27 ADDON-13867 Major performance issue for incremental S3 data inputs when ingesting large plain text files (max throughput only around 4MB/s for files of size 20MB)
2017-02-27 ADDON-13879 Regional Reserve Instance is missing in description data
2017-02-26 ADDON-13860 Configuring more AWS accounts increases CPU usage and lowers throughput performance due to increased API calls

Workaround:
Consolidate AWS accounts when configuring the Splunk Add-on for AWS.
2017-02-19 ADDON-13651 Describe EC2 is blocked by API throttling of get EBS snapshot data
2017-02-06 ADDON-13492, ADDON-13015, ADDON-13855 Ingesting a continuous stream of large files (e.g., 20MB) from a single incremental S3 data input may cause out-of-memory error
2017-01-13 ADDON-13282 Cannot change Description interval in UI more than once
2016-12-28 ADDON-12983 S3 dead loop when processing extremely large S3 files
2016-12-22 ADDON-12867, ADDON-11894 S3 input: large key numbers lead to excessively large checkpoint files
2016-12-14 ADDON-12700 Pagination issue in Account page.
2016-11-21 ADDON-12267 Disabling an active incremental s3 data input may cause duplicate data

Third-party software attributions

Version 4.2.3 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Version 4.2.2

Version 4.2.2 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms.

Splunk platform versions 6.3 and later
CIM 4.3 and later
Platforms Platform independent
Vendor Products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, Inspector, Kinesis, S3, VPC Flow Log, Billing services, SQS, and SNS.

New features

Version 4.2.2 of the Splunk Add-on for AWS does not contain any new features.

Fixed issues

Version 4.2.2 of the Splunk Add-on for Amazon Web Services fixes the following issues.

Date resolved Issue number Description
2017-01-21 ADDON-13369 Failed to list S3 buckets and Kinesis streams in GUI in proxy mode

Known issues

Version 4.2.2 of the Splunk Add-on for Amazon Web Services has the following known issues.

Date filed Issue number Description
2017-03-29 ADDON-14287 After you replace an IAM role attached to an EC2 instance, the inputs that use the old IAM role stop collecting data.
2017-03-09 ADDON-14038 Orphan process issue after master process been force killed
2017-03-09 ADDON-14039 Incremental S3 input fails to decode non-utf8 encoded files
2017-02-28 ADDON-13891 The S3 incremental input fails to skip the Glacier storage type keys
2017-02-27 ADDON-13865 Cannot disable/enable inputs under sc_admin role in Splunk Cloud
2017-02-27 ADDON-13867 Major performance issue for incremental S3 data inputs when ingesting large plain text files (max throughput only around 4MB/s for files of size 20MB)
2017-02-26 ADDON-13860 Configuring more AWS accounts increases CPU usage and lowers throughput performance due to increased API calls

Workaround:
Consolidate AWS accounts when configuring the Splunk Add-on for AWS.
2017-02-19 ADDON-13651 Describe EC2 is blocked by API throttling of get EBS snapshot data
2017-02-06 ADDON-13492, ADDON-13015, ADDON-13855 Ingesting a continuous stream of large files (e.g., 20MB) from a single incremental S3 data input may cause out-of-memory error
2017-01-13 ADDON-13282 Cannot change Description interval in UI more than once
2016-12-28 ADDON-12983 S3 dead loop when processing extremely large S3 files
2016-12-22 ADDON-12867, ADDON-11894 S3 input: large key numbers lead to excessively large checkpoint files
2016-11-21 ADDON-12267 Disabling an active incremental s3 data input may cause duplicate data

Third-party software attributions

Version 4.2.2 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Version 4.2.1

Version 4.2.1 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms.

Splunk platform versions 6.3 and later
CIM 4.3 and later
Platforms Platform independent
Vendor Products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, Inspector, Kinesis, S3, VPC Flow Log, Billing services, SQS, and SNS.

New features

Added support for two new AWS regions: EU (London) and Canada (Central).

Fixed issues

Version 4.2.1 of the Splunk Add-on for Amazon Web Services fixes the following issues.

Date resolved Issue number Description
2017-01-12 ADDON-13260 Error message during restart Splunk on EC2 instance
2017-01-11 ADDON-13209 Unexpected SQS message increases the size of the checkpoint file in SQS-based CloudTrail input and causes performance drop
2017-01-09 ADDON-11838 Cloudtrail event username mismatch between AWS console and app
2017-01-06 ADDON-12874 ExpiredToken error when calling the ListObjects operation may terminate the process

Known issues

Version 4.2.1 of the Splunk Add-on for Amazon Web Services has the following known issues.

Date filed Issue number Description
2017-02-27 ADDON-13865 Cannot disable/enable inputs under sc_admin role in Splunk Cloud
2017-02-27 ADDON-13867 Major performance issue for incremental S3 data inputs when ingesting large plain text files (max throughput only around 4MB/s for files of size 20MB)
2017-02-26 ADDON-13860 Configuring more AWS accounts increases CPU usage and lowers throughput performance due to increased API calls

Workaround:
Consolidate AWS accounts when configuring the Splunk Add-on for AWS.
2017-02-19 ADDON-13651 Describe EC2 is blocked by API throttling of get EBS snapshot data
2017-02-09 ADDON-13768 Upgrading the add-on causes the EC2 configuration in the Splunk App for AWS to fail with IAM Role
2017-02-06 ADDON-13492, ADDON-13015, ADDON-13855 Ingesting a continuous stream of large files (e.g., 20MB) from a single incremental S3 data input may cause out-of-memory error
2017-01-19 ADDON-13369 Failed to list S3 buckets and Kinesis streams in GUI in proxy mode
2017-01-13 ADDON-13282 Cannot change Description interval in UI more than once
2016-12-28 ADDON-12983 S3 dead loop when processing extremely large S3 files
2016-12-22 ADDON-12867, ADDON-11894 S3 input: large key numbers lead to excessively large checkpoint files
2016-11-21 ADDON-12267 Disabling an active incremental s3 data input may cause duplicate data

Third-party software attributions

Version 4.2.1 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Version 4.2.0

Version 4.2.0 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms.

Splunk platform versions 6.3 and later
CIM 4.3 and later
Platforms Platform independent
Vendor Products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, Inspector, Kinesis, S3, VPC Flow Log, Billing services, SQS, and SNS.

New features

Version 4.2.0 of the Splunk Add-on for Amazon Web Services supports the AWS Security Token Service (AWS STS) AssumeRole API action that lets you use IAM roles to delegate permissions to IAM users to access these AWS resources. You can configure accounts to use AssumeRole in these data inputs: S3 (general and incremental), Billing, Description, CloudWatch, Kinesis.

Fixed issues

Version 4.2.0 of the Splunk Add-on for Amazon Web Services fixes the following issues.

Date resolved Issue number Description
2016-12-27 ADDON-12918 API throttling error occurs during ingestion of ELB description input data and blocks ELB data collection
2016-12-12 ADDON-12600 Incorrect file name format blocks data ingestion
2016-12-12 ADDON-12660 Failed to retrieve cloudfront_distributions through proxy
2016-12-07 ADDON-12342 Poor list bucket performance in collecting S3 data
2016-12-07 ADDON-12344 Unwarranted config changed message in the S3 incremental input log
2016-12-06 ADDON-12236 Force killing splunkd leaves input orphan processes, which will be killed after splunkd restarts
2016-12-06 ADDON-12123, ADDON-12485 Race condition after checkpoint files are replaced
2016-12-06 ADDON-12397 One invalid Kinesis input blocks all other Kinesis inputs
2016-12-06 ADDON-12340 ReadTimeoutError - S3 data collection failed
2016-11-28 ADDON-11855, ADDON-11852 Performance degradation of AWS add-on modular input data collection in Splunk Platform 6.5.0
2016-11-27 ADDON-11894, ADDON-12867 S3-generic input ckpt file is too large

Known issues

Version 4.2.0 of the Splunk Add-on for Amazon Web Services has the following known issues.

Date filed Issue number Description
2018-01-05 ADDON-16518 When kinesis and cloudwatch inputs send large volumes of data over HEC, HEC can block the ingest pipeline, which breaks non-HEC inputs.

Workaround:
Set use_hec=false in [global_settings] stanza of aws_kinesis.conf and/or aws_cloudwatch.conf
2017-02-27 ADDON-13865 Cannot disable/enable inputs under sc_admin role in Splunk Cloud
2017-02-26 ADDON-13860 Configuring more AWS accounts increases CPU usage and lowers throughput performance due to increased API calls

Workaround:
Consolidate AWS accounts when configuring the Splunk Add-on for AWS.
2017-02-19 ADDON-13651 Describe EC2 is blocked by API throttling of get EBS snapshot data
2017-02-06 ADDON-13492, ADDON-13015, ADDON-13855 Ingesting a continuous stream of large files (e.g., 20MB) from a single incremental S3 data input may cause out-of-memory error
2017-01-19 ADDON-13369 Failed to list S3 buckets and Kinesis streams in GUI in proxy mode
2017-01-13 ADDON-13282 Cannot change Description interval in UI more than once
2017-01-11 ADDON-13260 Error message during restart Splunk on EC2 instance
2017-01-05 ADDON-13209 Unexpected SQS message increases the size of the checkpoint file in SQS-based CloudTrail input and causes performance drop
2017-01-02 ADDON-13041 s3 indexing latency introduced by assumerole feature (even account do not have assumerole)
2016-12-28 ADDON-12983 S3 dead loop when processing extremely large S3 files
2016-12-27 ADDON-12931 Upgrading from version 4.0.0 to 4.2.0 causes the Start Date/Time field value to be displayed incorrectly on the UI
2016-12-24 ADDON-12874 ExpiredToken error when calling the ListObjects operation may terminate the process
2016-12-22 ADDON-12867, ADDON-11894 S3 input: large key numbers lead to excessively large checkpoint files
2016-12-14 ADDON-12700 Pagination issue in Account page.
2016-11-21 ADDON-12267 Disabling an active incremental s3 data input may cause duplicate data
2016-10-27 ADDON-11838 Cloudtrail event username mismatch between AWS console and app
2016-09-08 ADDON-11225 Fails to download Billing files due to "Operation timed out" error
2015-09-09 ADDON-12762 Selecting all regions and all services in CloudWatch input results in some invalid tasks.

Third-party software attributions

Version 4.2.0 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Version 4.1.2

Version 4.1.2 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms.

Splunk platform versions 6.3, 6.4, 6.5
CIM 4.3 or later
Platforms Platform independent
Vendor Products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, Inspector, Kinesis, S3, VPC Flow Log, Billing services, SQS, and SNS.

New features

Version 4.1.2 of the Splunk Add-on for Amazon Web Services contains no new features.

Fixed issues

Version 4.1.2 of the Splunk Add-on for Amazon Web Services fixes the following issues.

Date resolved Issue number Description
2016-11-16 ADDON-12078 S3 incremental orphan process issue
2016-11-08 ADDON-11960 App menu display issue in Splunk Light

Known issues

Version 4.1.2 of the Splunk Add-on for Amazon Web Services has the following known issues.

Date filed Issue number Description
2018-01-05 ADDON-16518 When kinesis and cloudwatch inputs send large volumes of data over HEC, HEC can block the ingest pipeline, which breaks non-HEC inputs.

Workaround:
Set use_hec=false in [global_settings] stanza of aws_kinesis.conf and/or aws_cloudwatch.conf
2017-02-27 ADDON-13865 Cannot disable/enable inputs under sc_admin role in Splunk Cloud
2017-02-19 ADDON-13651 Describe EC2 is blocked by API throttling of get EBS snapshot data
2017-02-06 ADDON-13492, ADDON-13015, ADDON-13855 Ingesting a continuous stream of large files (e.g., 20MB) from a single incremental S3 data input may cause out-of-memory error
2017-01-13 ADDON-13282 Cannot change Description interval in UI more than once
2017-01-05 ADDON-13209 Unexpected SQS message increases the size of the checkpoint file in SQS-based CloudTrail input and causes performance drop
2016-12-26 ADDON-12918 API throttling error occurs during ingestion of ELB description input data and blocks ELB data collection
2016-12-12 ADDON-12660 Failed to retrieve cloudfront_distributions through proxy
2016-11-25 ADDON-12395 File descriptor leaking in generic S3 due to boto2 defects
2016-11-23 ADDON-12342 Poor list bucket performance in collecting S3 data
2016-11-23 ADDON-12340 ReadTimeoutError - S3 data collection failed
2016-11-21 ADDON-12267 Disabling an active incremental s3 data input may cause duplicate data
2016-11-18 ADDON-12236 Force killing splunkd leaves input orphan processes, which will be killed after splunkd restarts
2016-11-08 ADDON-11974 Cannot get CloudWatch data using some default configuration in Add-on
2016-10-28 ADDON-11846, SPL-138046 Logging breaks on rotation when multiple inputs write to the same log. If > 6 inputs, some inputs cannot log
2016-09-08 ADDON-11225 Fails to download Billing files due to "Operation timed out" error

Third-party software attributions

Version 4.1.2 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Version 4.1.1

Version 4.1.1 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms.

Splunk platform versions 6.3, 6.4 and 6.5
CIM 4.3 or later
Platforms Platform independent
Vendor Products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, Inspector, Kinesis, S3, VPC Flow Log, Billing services, SQS, and SNS.

New features

Version 4.1.1 of the Splunk Add-on for Amazon Web Services contains no new features.

Fixed issues

Version 4.1.1 of the Splunk Add-on for Amazon Web Services fixes the following issues.

Resolved date Issue number Description
2016-10-12 ADDON-11604 Incremental S3 fails to collect data using the IAM role.
2016-09-30 ADDON-11470 The inputs page cannot display more than 30 inputs (S3 as input).
2016-10-11 ADDON-11498, ADDON-11488 Ingesting data from aws:cloudwatchlogs results in invalid JSON format with extraneous trailing angle brackets.
2016-10-04 ADDON-11482 Cloudtrail/SQS fails to collect data using the IAM role.

Known issues

Version 4.1.1 of the Splunk Add-on for Amazon Web Services has the following known issues.

Date filed Issue number Description
2016-12-26 ADDON-12918 API throttling error occurs during ingestion of ELB description input data and blocks ELB data collection
2016-11-23 ADDON-12342 Poor list bucket performance in collecting S3 data
2016-11-17 ADDON-12232 Get S3 error during upgrade from 4.0.0
2016-11-14 ADDON-12123, ADDON-12485 Race condition after checkpoint files are replaced
2016-11-10 ADDON-12072 Do not support Splunk global proxy. Update it in add-on configuration if needed.
2016-11-10 ADDON-12078 S3 incremental orphan process issue
2016-11-08 ADDON-11974 Cannot get CloudWatch data using some default configuration in Add-on
2016-11-07 ADDON-11960 App menu display issue in Splunk Light
2016-11-02 ADDON-11893 IO exception in S3 input
2016-11-02 ADDON-11894, ADDON-12867 S3-generic input ckpt file is too large
2016-10-30 ADDON-11855, ADDON-11852 Performance degradation of AWS add-on modular input data collection in Splunk Platform 6.5.0
2016-10-28 ADDON-11847 s3 input zombie processes

Workaround:
Update the symbolic link so that /bin/sh targets /bin/bash.

$ debconf-set-selections <<< "dash dash/sh string false" $ dpkg-reconfigure -f noninteractive dash

2016-10-28 ADDON-11846, SPL-138046 Logging breaks on rotation when multiple inputs write to the same log. If > 6 inputs, some inputs cannot log
2016-09-22 ADDON-11415 The input name is case sensitive lead to failure on Windows platform
2016-09-13 ADDON-11295 Cloudtrail still delete SQS message even if failed to get S3 file
2016-09-08 ADDON-11225 Fails to download Billing files due to "Operation timed out" error
2016-08-18 ADDON-10957 Log level set to ERROR but still found INFO logs
2016-06-20 ADDON-10286 CloudWatch modular input generates duplicate events when the Splunk platform is restarted

Workaround:
dedup based on the _time field
2016-05-30 ADDON-9753 Proxy password does not support the special characters '|', ':' or '@'
2016-05-12 ADDON-9435 Wrong number of inputs listed on Account page.
2016-05-12 ADDON-9422 CloudWatch input can have data loss when empty results are returned twice in succession and then Splunk platform restarts before the input next collects data.
2016-05-11 ADDON-9408 Detailed Billing is not indexed using UTC timezone
2016-05-11 ADDON-9409 Checkpoints file will not be removed when deleting Config Rules
2016-05-07 ADDON-9332 fails to get latest cloudwatch data sometimes
2016-04-29 ADDON-9148 Updating directly from v2.0.0 to v4.0.0 makes existing accounts unavailable
2016-04-28 ADDON-9133 CloudWatch default configuration may not work in cases where there are millions of dimensions
2016-04-28 ADDON-9145 Error message shown on input creation screen has logic issues and is not as specific as we could be
2016-01-13 ADDON-7448 In the Description data input, the port range defaults to null in vpc_network_acls if no range is specified, which is confusing, because it actually has a range of "all".
2015-12-29 ADDON-7239 Using "/" in data input name causes exceptions. UI does not accept this character in the input names, but if you configure your input using conf files, you will find exceptions in logs.
2015-12-22 ADDON-7159 After removing all search peers, add-on still shows performance warnings.

Workaround:
Restart a Splunk platform instance after changing its role.
2015-12-16 ADDON-7035 Add-on ingests the header line of the CloudFront access log, but it should be skipped.
2015-11-26 ADDON-6701 EC2, RDS, ELB, and EC2 APIs do not consider pagination.
2015-10-14 ADDON-6056 S3 logging errors on Windows.
2015-10-13 ADDON-6043 SQS message mistakenly deleted when the add-on throws an error retrieving data from an S3 bucket.
2015-09-11 ADDON-5500 Preconfigured reports for billing data cannot handle reports that have a mix of different currencies. The report will use the first currency found and apply that to all costs.
2015-09-11 ADDON-5499 CloudWatch: Previous selected Metric namespace always exists in the list regardless of the region change
2015-09-10 ADDON-5471 Deleting a CloudWatch data input takes too long.
2015-09-10 ADDON-5481 The add-on configuration UI does not handle insufficient Splunk user permissions gracefully.
2015-09-07 ADDON-5355 Different error message for same error when creating duplicated data inputs.
2015-09-06 ADDON-5354 Using keyboard to delete selections from configuration dropdown multi-select field causes drop-down list to appear in corner of screen.
2015-09-01 ADDON-5309 UI default value is not read from default input config file
2015-09-01 ADDON-5295 Description inconsistent in the GUI for CloudTrail service and CloudTrail from S3 service blacklist behavior.
2015-04-02 ADDON-3578 S3: uppercase bucket names cause an error
2014-09-28 ADDON-2135 The list of regions shown in inputs configuration in Splunk Web shows all Amazon regions regardless of the permissions associated with the selected AWS account.
2014-09-26 ADDON-2118 Data inputs continue to work after user deletes the account used for that input.

Workaround:
Restart the Splunk platform after deleting or modifying an AWS account.
2014-09-25 ADDON-2113 The app.conf file includes a stanza for a proxy server configuration with a hashed password even if the user has not configured a proxy or password.

Workaround:
This behavior is expected because Splunk Enterprise automatically sets the proxy field to 0 and saves an encrypted entry in app.conf.
2014-09-16 ADDON-2029 In saved search "Monthly Cost till *" _time is displayed per day rather than per month.

Third-party software attributions

Version 4.1.1 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.



Version 4.1.0

Version 4.1.0 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms.

Splunk platform versions 6.3, 6.4
CIM 4.3 or later
Platforms Platform independent
Vendor Products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, Inspector, Kinesis, S3, VPC Flow Log, Billing services, SQS, and SNS.

New features

Version 4.1.0 of the Splunk Add-on for Amazon Web Services has the following new features.

Date Issue number Description
2016-09-22 ADDON-6145 Add AWS SQS modular input for Splunk add-on for AWS.
2016-09-22 ADDON-6146 Add custom alert to AWS SNS for Splunk add-on for AWS.
2016-09-22 ADDON-10952 Performance enhancement for AWS Cloudtrail modular input.
2016-09-22 ADDON-11149 Add Record Format field for AWS Kinesis modular input.
2016-09-22 ADDON-10917 Mapping to ITSI IaaS data module.
2016-09-22 ADDON-10941 Add new incremental data collection for S3 modular input.
2016-09-22 ADDON-10414 Checkpoint and performance enhancement for S3 modular input.
2016-09-22 ADDON-10906 Performance and API call enhancement for Cloudwatch modular input.

Fixed issues

Version 4.1.0 of the Splunk Add-on for Amazon Web Services fixes the following issues.

Resolved date Issue number Description
2016-09-20 ADDON-11251 There will be data loss of ASW S3 input if the network connection is bad.
2016-09-20 ADDON-11196 If there is a blank space at the beginning or the end of the input name (or both). The input name displays on the UI is not the consistent with the one saved in the configuration file.
2016-09-20 ADDON-11056 In the AWS Region list, it displays ap-northeast-2 instead of Seoul.
2016-09-20 ADDON-10980 Line breaker error for AWS S3 input.
2016-09-14 ADDON-10186 AWS Config fails to fetch S3 object in AWS GovCloud (US) region.
2016-09-09 ADDON-11009 Vanguard: Not getting data from 1 of 3 S3 inputs. This is considered critical for the customer as they have PS on site.
2016-08-18 ADDON-10137 If the number of the AWS input exceeds 30, some of the inputs cannot run successfully.
2016-09-14 ADDON-9778 There are some errors of AWS Kinesis modular input if the request from HEC exceeds its max limit.
2016-09-05 ADDON-9732 Failed to get proxy credentials when password includes # character.
2016-08-28 ADDON-9533 The default Dimension Name is empty square brackets for Autoscaling and EBS namespaces.
2016-08-08 ADDON-9328 CloudWatch data input encounters API rate limit for large metrics.
2016-09-09 ADDON-8758 Mixing log types or gzip with plain text in the same stream causes knowledge extraction to fail for CloudWatch Logs data collected through Kinesis

Known issues

Version 4.1.0 of the Splunk Add-on for Amazon Web Services has the following known issues.

Date filed Issue number Description
2016-10-30 ADDON-11855, ADDON-11852 Performance degradation of AWS add-on modular input data collection in Splunk Platform 6.5.0
2016-10-28 ADDON-11846, SPL-138046 Logging breaks on rotation when multiple inputs write to the same log. If > 6 inputs, some inputs cannot log
2016-10-12 ADDON-11611 Fails to publish search result to SNS using the IAM role.
2016-10-11 ADDON-11604 S3 incremental failed to fetch data using IAM role
2016-10-05 ADDON-11498 Trailing angle bracket and invalid JSON in aws:cloudwatchlogs
2016-10-04 ADDON-11488 aws_cloudwatch_logs_data_loader.py#L88
2016-10-03 ADDON-11482 Cloudtrail input error after upgrade to 4.1
2016-09-28 ADDON-11470 Inputs page doesn't show more than 30 inputs (S3 as input)
2016-09-22 ADDON-11415 The input name is case sensitive lead to failure on Windows platform
2016-09-19 ADDON-11326 Unexpected timestamp format blocks data ingestion
2016-09-13 ADDON-11295 Cloudtrail still delete SQS message even if failed to get S3 file
2016-09-08 ADDON-11225 Fails to download Billing files due to "Operation timed out" error
2016-08-18 ADDON-10957 Log level set to ERROR but still found INFO logs
2016-06-20 ADDON-10286 CloudWatch modular input generates duplicate events when the Splunk platform is restarted

Workaround:
dedup based on the _time field
2016-05-30 ADDON-9753 Proxy password does not support the special characters '|', ':' or '@'
2016-05-30 ADDON-9745 Add-on does not support proxy accounts that do not require passwords
2016-05-12 ADDON-9435 Wrong number of inputs listed on Account page.
2016-05-12 ADDON-9422 CloudWatch input can have data loss when empty results are returned twice in succession and then Splunk platform restarts before the input next collects data.
2016-05-11 ADDON-9408 Detailed Billing is not indexed using UTC timezone
2016-05-11 ADDON-9409 Checkpoints file will not be removed when deleting Config Rules
2016-05-07 ADDON-9332 fails to get latest cloudwatch data sometimes
2016-04-29 ADDON-9148 Updating directly from v2.0.0 to v4.0.0 makes existing accounts unavailable
2016-04-28 ADDON-9133 CloudWatch default configuration may not work in cases where there are millions of dimensions
2016-04-28 ADDON-9145 Error message shown on input creation screen has logic issues and is not as specific as we could be
2016-01-13 ADDON-7448 In the Description data input, the port range defaults to null in vpc_network_acls if no range is specified, which is confusing, because it actually has a range of "all".
2015-12-29 ADDON-7239 Using "/" in data input name causes exceptions. UI does not accept this character in the input names, but if you configure your input using conf files, you will find exceptions in logs.
2015-12-22 ADDON-7159 After removing all search peers, add-on still shows performance warnings.

Workaround:
Restart a Splunk platform instance after changing its role.
2015-12-16 ADDON-7035 Add-on ingests the header line of the CloudFront access log, but it should be skipped.
2015-11-26 ADDON-6701 EC2, RDS, ELB, and EC2 APIs do not consider pagination.
2015-10-14 ADDON-6056 S3 logging errors on Windows.
2015-10-13 ADDON-6043 SQS message mistakenly deleted when the add-on throws an error retrieving data from an S3 bucket.
2015-09-11 ADDON-5500 Preconfigured reports for billing data cannot handle reports that have a mix of different currencies. The report will use the first currency found and apply that to all costs.
2015-09-11 ADDON-5499 CloudWatch: Previous selected Metric namespace always exists in the list regardless of the region change
2015-09-10 ADDON-5471 Deleting a CloudWatch data input takes too long.
2015-09-10 ADDON-5481 The add-on configuration UI does not handle insufficient Splunk user permissions gracefully.
2015-09-07 ADDON-5355 Different error message for same error when creating duplicated data inputs.
2015-09-06 ADDON-5354 Using keyboard to delete selections from configuration dropdown multi-select field causes drop-down list to appear in corner of screen.
2015-09-01 ADDON-5309 UI default value is not read from default input config file
2015-09-01 ADDON-5295 Description inconsistent in the GUI for CloudTrail service and CloudTrail from S3 service blacklist behavior.
2015-04-02 ADDON-3578 S3: uppercase bucket names cause an error
2014-09-28 ADDON-2135 The list of regions shown in inputs configuration in Splunk Web shows all Amazon regions regardless of the permissions associated with the selected AWS account.
2014-09-26 ADDON-2118 Data inputs continue to work after user deletes the account used for that input.

Workaround:
Restart the Splunk platform after deleting or modifying an AWS account.
2014-09-25 ADDON-2113 The app.conf file includes a stanza for a proxy server configuration with a hashed password even if the user has not configured a proxy or password.

Workaround:
This behavior is expected because Splunk Enterprise automatically sets the proxy field to 0 and saves an encrypted entry in app.conf.
2014-09-16 ADDON-2029 In saved search "Monthly Cost till *" _time is displayed per day rather than per month.

Third-party software attributions

Version 4.1.0 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Version 4.0.0

Version 4.0.0 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms.

Splunk platform versions 6.2.X and later
CIM 4.0 and later
Platforms Platform independent
Vendor Products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, Inspector, Kinesis, S3, VPC Flow Log, and Billing services

Upgrade Guide

If you are upgrading from a previous version of the Splunk Add-on for AWS, be aware of the following changes which may require some actions to preserve the functionality of your existing accounts and inputs:

  • This release includes three new inputs that each require new IAM permissions. Be sure to adjust the IAM permissions of your existing accounts if you want to use them to collect these new data sources. See Configure AWS permissions for the Splunk Add-on for AWS for details.
  • If you are upgrading directly from version 2.0.0 or earlier of this add-on to the 4.0.0 version, you need to open and resave the AWS accounts using the Splunk Add-on for AWS account UI.
  • In this version, the CloudWatch input is rearchitected for better performance and improved stability. One result of this new architecture is that the input has a built in four minute delay after a polling period has ended for any given metric before the actual data collection occurs. This change ensures that there is no data loss due to latency on the AWS side.
  • This version requires a single selection for the Region Category for each AWS account. If you added accounts before region category selection was required, or if you added accounts and selected more than one region category for a single account, the upgrade to version 4.0.0 will put these accounts into an error state until you edit them to select a single region category. On your data collection node, open the add-on and check your Configuration tab to see if any of your existing accounts are missing a region category. If they are, edit the account to add the region category. Any inputs using accounts that were determined to be in error stop collecting data until the account has a region category assigned. Once the account error is resolved, the affected inputs start collecting data again automatically starting from the point when data collection stopped.

New Features

Version 4.0.0 of the Splunk Add-on for Amazon Web Services has the following new features.

Resolved date Issue number Description
2016-04-29 ADDON-7042 CloudWatch input configuration UI now provides auto-filled correct default JSON for metrics and dimensions in each namespace.
2016-04-08 ADDON-7587 Support for AWS Signature V.4 managed keys for S3 related data collection.
2016-04-05 ADDON-7818 New input and CIM mapping for Amazon Inspector data.
2016-04-05 ADDON-7817 New input and CIM mapping for AWS Config rules data.
2016-04-05 ADDON-5391 New input for data from Kinesis streams, including high volume VPC flow log data.
2016-03-31 ADDON-6811 Support for using an EC2 IAM role instead of an AWS account when the add-on's collection node is on your own managed AWS instance.
2016-03-23 ADDON-7872 Support for the Seoul region.
2016-01-08 ADDON-7311 Support for setting an initial scan time in the Billing input if configuring using the conf files.

Fixed issues

Version 4.0.0 of the Splunk Add-on for Amazon Web Services fixes the following issues.

Resolved date Defect number Description
2016-05-04 ADDON-9169
Monthly Billing is not indexed using the UTC timezone
2016-04-19 ADDON-8801
Billing initial scan time should not use last modified time of S3 key
2016-04-15 ADDON-8721
Sourcetype="aws:cloudwatchlogs:vpcflow" handles src and dest incorrectly
2016-04-11 ADDON-8686
S3 input UI cannot display custom source types when user edits the input.
2016-04-03 ADDON-8547
S3 modular input loses data if new keys are generated during the key listing process
2016-04-02 ADDON-8546
S3 logging is unclear, should include indication of which input stanza is involved.
2016-03-31 ADDON-8548
CloudWatch collection failing with "Failed to get proxy information Empty"
2016-03-15 ADDON-8299
S3 input cannot progress if keys are deleted during the data collection.
2016-02-29 ADDON-8705
Add-on throws "is not JSON serializable" error when calling AWS API for ELB information
2016-02-25 ADDON-7969
CloudWatch has performance problems in large AWS accounts.
2016-02-24 ADDON-7957
Unnecessary tag expansion slows performance.
2016-02-24 ADDON-7926
Default value of max_file_size_csv_zip_in_bytes is too small to handle large detailed billing reports
2016-02-22 ADDON-7897
s3util.py list_cloudwatch_namespaces has performance issue
2016-02-19 ADDON-7877
Upon upgrade from version 2.0.X, S3 inputs experience two problems.
Workaround: 1. Inputs with a S3 key prefix specified stop collecting data from AWS.

Workaround: Stop splunkd and go to $SPLUNK_HOME/var/lib/modinputs/aws_s3/, find the checkpoint file for that data input (ls -lh to list and find the large files), open the file, and note the last_modified_time in the file. Remove the checkpoint file and update the data input in inputs.conf using the last_modified_time value that you observed in the checkpoint file for the initial_scan_time in the new input. Reboot splunkd. 2. The polling_interval does not persist automatically. Workaround: In Splunk Web, open the input configuration, go to Settings, set an interval value, then click Update. Or, in local/inputs.conf, set the polling_interval to a value that matches your needs, then save the file.

2016-02-14 ADDON-7777
Not all fields are parsed for CloudFront
2016-02-13 ADDON-7778
Cannot create new input when Splunk does not have a user named "admin"
2016-02-13 ADDON-7776
CloudFront logs should be urldecoded
2016-01-25 ADDON-7573
CloudWatch input requests too many data points in long time windows.
2016-01-18 ADDON-7701
CloudWatch fails to gather data when no metrics appear in a namespace for more than 12 hours.
2015-09-11 ADDON-5498
Unclear error: Unexpected error "<class 'socket.error'>" from python handler: " Connection refused" when user specifies all regions in CloudWatch for one namespace, saves the configuration, and reloads it.
2015-09-10 ADDON-5469
Missing or improper default value for un-required fields.

Known issues

Version 4.0.0 of the Splunk Add-on for Amazon Web Services has the following known issues.

Date filed Issue number Description
2016-10-28 ADDON-11847 s3 input zombie processes

Workaround:
Update the symbolic link so that /bin/sh targets /bin/bash.

$ debconf-set-selections <<< "dash dash/sh string false" $ dpkg-reconfigure -f noninteractive dash

2016-10-28 ADDON-11846, SPL-138046 Logging breaks on rotation when multiple inputs write to the same log. If > 6 inputs, some inputs cannot log
2016-10-05 ADDON-11498 Trailing angle bracket and invalid JSON in aws:cloudwatchlogs
2016-09-22 ADDON-11415 The input name is case sensitive lead to failure on Windows platform
2016-09-19 ADDON-11326 Unexpected timestamp format blocks data ingestion
2016-09-12 ADDON-11266 Chrome failed to create account
2016-09-10 ADDON-11251 Data loss when creating multi inputs to ingesting data
2016-09-08 ADDON-11225 Fails to download Billing files due to "Operation timed out" error
2016-09-06 ADDON-11196 Strip blank space in input name
2016-08-29 ADDON-11056 Region shows "ap-northeast-2" but not Seoul
2016-08-24 ADDON-11009 Vanguard: Not getting data from 1 of 3 S3 inputs. This is considered critical for the customer as they have PS on site
2016-08-22 ADDON-10978 S3 data loss after disable/enable
2016-08-22 ADDON-10980 S3 line breaker error
2016-08-18 ADDON-10957 Log level set to ERROR but still found INFO logs
2016-07-20 ADDON-10643 Rest handler splunk_ta_aws_settings_account_region is missing
2016-07-17 ADDON-10574 Log level for can't find checkpoint should not be ERROR
2016-07-06 ADDON-10450 REST handler s3buckets still returns status 200 while connection failed
2016-06-20 ADDON-10286 CloudWatch modular input generates duplicate events when the Splunk platform is restarted

Workaround:
dedup based on the _time field
2016-06-13 ADDON-10186 AWS Config fails to fetch S3 object in AWS GovCloud (US) region
2016-05-31 ADDON-9778 HEC max limit needs to take padding into account to avoid 413 "Content-Length of <value> too large" errors
2016-05-30 ADDON-9745 Add-on does not support proxy accounts that do not require passwords
2016-05-30 ADDON-9753 Proxy password does not support the special characters '|', ':' or '@'
2016-05-27 ADDON-9732 failed to get proxy credentials when password includes # sign
2016-05-18 ADDON-9533 Dimensions default to empty square brackets for Autoscaling and EBS namespaces
2016-05-16 ADDON-9451 Monthly billing date is displayed as next month for some timezones
2016-05-12 ADDON-9435 Wrong number of inputs listed on Account page.
2016-05-12 ADDON-9431 further save the cost with more efficient API call
2016-05-12 ADDON-9422 CloudWatch input can have data loss when empty results are returned twice in succession and then Splunk platform restarts before the input next collects data.
2016-05-12 ADDON-9434, ADDON-10137 Rest Handler Of List Data Inputs Truncates Result.

Workaround:
1) Navigate to /opt/splunk/etc/apps/Splunk_TA_aws/bin/splunktalib/rest.py

2) Change line 44 of this script from: resp, content = http.request(splunkd_uri, method=method, to resp, content = http.request(splunkd_uri + "?count=-1", method=method, 3) Save and exit

2016-05-11 ADDON-9408 Detailed Billing is not indexed using UTC timezone
2016-05-11 ADDON-9409 Checkpoints file will not be removed when deleting Config Rules
2016-05-07 ADDON-9332 fails to get latest cloudwatch data sometimes
2016-05-06 ADDON-9328 CloudWatch data input encounters API rate limit for large metrics

Workaround:
Increase your granularity and polling interval in order to make fewer API calls, or contact AWS to increase your allowed number of API calls per month.
2016-04-29 ADDON-9148 Updating directly from v2.0.0 to v4.0.0 makes existing accounts unavailable
2016-04-28 ADDON-9145 Error message shown on input creation screen has logic issues and is not as specific as we could be
2016-04-28 ADDON-9133 CloudWatch default configuration may not work in cases where there are millions of dimensions
2016-04-27 ADDON-9117 Using EC2 IAM role for data collection does not work in China or GovCloud regions.
2016-04-20 ADDON-8905 Add-on throws "connection refused" error when Splunk platform restarts
2016-04-19 ADDON-8758 Mixing log types or gzip with plain text in the same stream causes knowledge extraction to fail for CloudWatch Logs data collected through Kinesis
2016-03-01 ADDON-8113 Excessive S3 API calls
2016-01-13 ADDON-7448 In the Description data input, the port range defaults to null in vpc_network_acls if no range is specified, which is confusing, because it actually has a range of "all".
2015-12-29 ADDON-7239 Using "/" in data input name causes exceptions. UI does not accept this character in the input names, but if you configure your input using conf files, you will find exceptions in logs.
2015-12-22 ADDON-7159 After removing all search peers, add-on still shows performance warnings.

Workaround:
Restart a Splunk platform instance after changing its role.
2015-12-16 ADDON-7035 Add-on ingests the header line of the CloudFront access log, but it should be skipped.
2015-11-26 ADDON-6701 EC2, RDS, ELB, and EC2 APIs do not consider pagination.
2015-10-14 ADDON-6056 S3 logging errors on Windows.
2015-10-13 ADDON-6043 SQS message mistakenly deleted when the add-on throws an error retrieving data from an S3 bucket.
2015-09-11 ADDON-5500 Preconfigured reports for billing data cannot handle reports that have a mix of different currencies. The report will use the first currency found and apply that to all costs.
2015-09-11 ADDON-5499 CloudWatch: Previous selected Metric namespace always exists in the list regardless of the region change
2015-09-10 ADDON-5471 Deleting a CloudWatch data input takes too long.
2015-09-10 ADDON-5481 The add-on configuration UI does not handle insufficient Splunk user permissions gracefully.
2015-09-07 ADDON-5355 Different error message for same error when creating duplicated data inputs.
2015-09-06 ADDON-5354 Using keyboard to delete selections from configuration dropdown multi-select field causes drop-down list to appear in corner of screen.
2015-09-01 ADDON-5309 UI default value is not read from default input config file
2015-09-01 ADDON-5295 Description inconsistent in the GUI for CloudTrail service and CloudTrail from S3 service blacklist behavior.
2015-07-06 ADDON-6177 When tmp file system runs out of space, aws_billing.py fails with IOError: No space left on device.
2015-04-02 ADDON-3578 S3: uppercase bucket names cause an error
2015-03-25 ADDON-3460 On OSs (like Debian and Ubuntu) that use dash for shell scripts, aws_cloudwatch.py spawns zombie processes.

Workaround:
Kill the processes and restart. Use bash to prevent re-occurrence.
2014-09-28 ADDON-2135 The list of regions shown in inputs configuration in Splunk Web shows all Amazon regions regardless of the permissions associated with the selected AWS account.
2014-09-26 ADDON-2118 Data inputs continue to work after user deletes the account used for that input.

Workaround:
Restart the Splunk platform after deleting or modifying an AWS account.
2014-09-25 ADDON-2113 The app.conf file includes a stanza for a proxy server configuration with a hashed password even if the user has not configured a proxy or password.

Workaround:
This behavior is expected because Splunk Enterprise automatically sets the proxy field to 0 and saves an encrypted entry in app.conf.
2014-09-16 ADDON-2029 In saved search "Monthly Cost till *" _time is displayed per day rather than per month.

Third-party software attributions

Version 4.0.0 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Version 3.0.0

Version 3.0.0 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms.

Splunk platform versions 6.2.X and later
CIM 4.0 and later
Platforms Platform independent
Vendor Products AWS CloudTrail, CloudWatch, CloudWatch Logs, Config, Billing, S3

Upgrade guide

This release includes some changes to the S3 input configuration that break backwards compatibility. If you are upgrading from a previous version and had previously used any of the following parameters, review the new behavior noted here and make any necessary changes in your existing S3 inputs:

  • interval now refers to how long splunkd should wait before checking the health of the modular input and restarting it if it has crashed. The new argument polling_interval, still shown as Interval in the UI, handles the data collection interval. If you had a custom value configured, the 3.0.0 version of the add-on copies your custom setting to the polling_interval value so that your data collection behavior does not change. However, you may wish to tune the interval value to enable splunkd to check for input crashes more frequently.
  • is_secure is deprecated and removed, but the parameter is retained in default/inputs.conf to avoid spec file violations. All traffic is over https. If you have this parameter in your local/inputs.conf, it will have no effect.
  • max_items is deprecated and removed, but the parameter is retained in default/inputs.conf to avoid spec file violations. It is set to 100000 items. If you have this parameter in your local/inputs.conf, it will have no effect.
  • queueSize is deprecated and removed. If you have this parameter in your local/inputs.conf, remove it to avoid potential data loss.
  • persistentQueueSize is deprecated and removed. If you have this parameter in your local/inputs.conf, remove it to avoid potential data loss.
  • recursion_depth is deprecated and removed, but the parameter is retained in default/inputs.conf to avoid spec file violations. The input recursively scans all subdirectories. If you have this parameter in your local/inputs.conf, it will have no effect.
  • ct_excluded_events_index is deprecated and removed, but the parameter is retained in default/inputs.conf to avoid spec file violations. Excluded events will be discarded. If you have this parameter in your local/inputs.conf, it will have no effect.

New features

Version 3.0.0 of the Splunk Add-on for Amazon Web Services has the following new features.

Resolved date Issue number Description
2015-11-16 ADDON-6690 Add-on configuration screen serves a warning message when you access it on a Splunk search head to remind you to configure it on heavy forwarders as a best practice.
2015-12-23 ADDON-6870 Support for GovCloud and China regions in the configuration UI.
2015-12-22 ADDON-6862 Support in the configuration UI and backend for new source types: aws:s3:accesslogs,

aws:cloudfront:accesslogs, aws:elb:accesslogs

2015-12-17 ADDON-6190 CloudWatch input refreshes the resource ID list every few hours so as to include additional resources to a wildcarded statement.
2015-12-17 ADDON-6187 CloudWatch collects S3 key count and total size of all keys in buckets.
2015-12-15 ADDON-6864 S3 modular input backend automatically detects the region, thus supporting bucket names with dots in them without user's needing to specify a region-specific endpoint.
2015-12-15 ADDON-6854 Deprecation of character_set parameter for S3 input. Input supports auto-detection among UTF-8 with/without BOM, UTF-16LE/BE with BOM, UTF-32BE/LE with BOM. Other character sets are not supported.
2015-12-15 ADDON-6189 Support for collecting ELB access logs using the aws:elb:accesslogs.
2015-12-14 ADDON-6869 Support for S3 buckets in the Frankfurt region with V4 signature only.
2016-12-14 ADDON-6866 Improved auditing information for log enrichment.
2015-12-14 ADDON-6859 S3 input blacklist has improved performance.
2015-12-14 ADDON-6857 S3 input whitelist has improved performance.
2015-12-14 ADDON-6860 Improved handling of process failures without duplication or loss of data.
2015-12-14 ADDON-6861 Support for checkpoint deletion behavior for the S3 input to avoid running into collection limits.
2015-12-14 ADDON-6865 Support for initial scan time in the S3 input, as well as in the new aws:s3:accesslogs, aws:cloudfront:accesslogs, and aws:elb:accesslogs source types.
2015-12-14 ADDON-6863 Improved collection behavior in the S3 input: if the key is updated without content changes, the add-on indexes the key again. If the key is changed during data collection, the add-on starts over with the data collection.
2015-12-14 ADDON-6868 The S3 input supports standard server-side KMS encrypted objects.
2015-12-14 ADDON-6855 The S3 input supports bin files.
2015-12-14 ADDON-6852 Improved performance for S3 input. Approximately 300% performance enhancement against 2.0.1 release. Over 8000% performance improvement for small files. See Performance reference for the S3 input in the Splunk Add-on for AWS for details.
2015-12-14 ADDON-6434 UI support for configuring alternate source types within the S3 input.
2015-12-14 ADDON-6196 Support for collecting CloudFront access logs with the aws:cloudfront:accesslogs source type.
2015-12-14 ADDON-6526 S3 input recognizes and skips S3 buckets with contents that have been moved to Glacier.
2015-12-14 ADDON-6188 New source type for S3 access logs: aws:s3:accesslogs.
2015-12-03 ADDON-6433 Improvements to the Description input's API and interval configuration UI.
2015-12-01 ADDON-6519 Improved timeout behavior in the configuration UI.
2015-11-26 ADDON-6194 Improvements to field aliasing for AWS regions.
2015-11-26 ADDON-6207 Gather metadata through the Description input for EBS, VPC, Security Group, Subnet, Network ACL, Key Pairs, ELB, CloudFront, RDS.

Fixed issues

Version 3.0.0 of the Splunk Add-on for Amazon Web Services fixes the following issues.

Resolved date Defect number Description
2016-01-14 ADDON-7291 S3 data input only shows 30 entries at maximum.
2016-01-03 ADDON-7258 Configuration screen needs to show better error message when user may be trying to use an invalid AWS account.
2015-12-31 ADDON-7253 Default initial_scan_datetime should be ISO8601 instead of the current default of current time minus 7 days.
2015-12-16 ADDON-7031 UI errors when using the base URL via reverse proxy.
2015-12-15 ADDON-6754 Typo in aws_cloudtrail.py script throws critical error in aws_cloudtrail.log with "NameError: global name 'taaw' is not defined".
2015-12-15 ADDON-7008 Add-on is not indexing ELB data through Description input.
2015-12-14 ADDON-6308 S3 input should validate key name does not include invalid characters such as leading or trailing whitespace.
2015-11-26 ADDON-6698 AWS Billing account ID should be payer's account ID instead of linked account ID.
2015-12-22 ADDON-5491 The add-on configuration UI displays all regions instead of those within the selected account's permission scope.
2015-12-20 ADDON-6958 / ADDON-5474 No detailed error shown while getting S3 buckets via REST endpoint with wrong proxy or account settings.
2015-01-22 ADDON-3050/
SPL-96729/
SPL-64904
S3 input is breaking lines incorrectly and inconsistently indexing only partial events due to use of persistentQueueSize.
2014-08-14 ADDON-1827 Checkpoints are not cleared after data inputs are removed or the add-on is uninstalled, thus if you create a new input with the same name as the deleted one, the add-on uses the checkpoint from the old input.

Known issues

Version 3.0.0 of the Splunk Add-on for Amazon Web Services has the following known issues.

Date filed Defect number Description
2016-05-04 ADDON-9169
Monthly Billing is not indexed by using UTC timezone
2016-04-28 ADDON-9145
Error message shown on input creation screen has logic issues and is not as specific as we could be
2016-04-19 ADDON-8801
Billing initial scan time should not use last modified time of S3 key
2016-04-15 ADDON-8721
Sourcetype="aws:cloudwatchlogs:vpcflow" handles src and dest incorrectly
2016-04-11 ADDON-8686
S3 input UI cannot display custom source types when user edits the input.
2016-04-03 ADDON-8547
S3 modular input loses data if new keys are generated during the key listing process
2016-04-02 ADDON-8546
S3 logging is unclear, should include indication of which input stanza is involved.
2016-03-31 ADDON-8548
Cloudwatch Collection failing with Failed to get proxy information Empty
2016-03-15 ADDON-8299
S3 input cannot progress if keys are deleted during the data collection.
2016-02-29 ADDON-8705
Add-on throws "is not JSON serializable" error when calling AWS API for ELB information
2016-02-25 ADDON-7969
CloudWatch has performance problems in large AWS accounts.
2016-02-24 ADDON-7957
Unnecessary tag expansion slows performance.
2016-02-24 ADDON-7926
Default value of max_file_size_csv_zip_in_bytes is too small to handle large detailed billing reports
2016-02-22 ADDON-7897
s3util.py list_cloudwatch_namespaces has performance issue
2016-02-19 ADDON-7877
Upon upgrade from version 2.0.X, S3 inputs experience two problems.
Workaround: 1. Inputs with a S3 key prefix specified stop collecting data from AWS.

Workaround: Stop splunkd and go to $SPLUNK_HOME/var/lib/modinputs/aws_s3/, find the checkpoint file for that data input (ls -lh to list and find the large files), open the file, and note the last_modified_time in the file. Remove the checkpoint file and update the data input in inputs.conf using the last_modified_time value that you observed in the checkpoint file for the initial_scan_time in the new input. Reboot splunkd. 2. The polling_interval does not persist automatically. Workaround: In Splunk Web, open the input configuration, go to Settings, set an interval value, then click Update. Or, in local/inputs.conf, set the polling_interval to a value that matches your needs, then save the file.

2016-02-14 ADDON-7777
Not all fields are parsed for CloudFront
2016-02-13 ADDON-7778
Cannot create new input when Splunk does not have a user named "admin"
2016-02-13 ADDON-7776
CloudFront logs should be urldecoded
2016-02-11 ADDON-7764
FIPS mode is not supported by this add-on.
2016-01-25 ADDON-7573
CloudWatch input requests too many data points in long time windows.
2016-01-18 ADDON-7701
CloudWatch fails to gather data when no metrics appear in a namespace for more than 12 hours.
2016-01-13 ADDON-7448
In the Description data input, the port range defaults to null in vpc_network_acls if no range is specified, which is confusing, because it actually has a range of "all".
2015-12-29 ADDON-7239
Using "/" in data input name causes exceptions. UI does not accept this character in the input names, but if you configure your input using conf files, you will find exceptions in logs.
2015-12-22 ADDON-7160
Add-on throws a timeout error in the UI when user attempts to create a new S3 input, but successfully creates the input in the backend, causing errors if the user tries to create the same input again.
2015-12-22 ADDON-7159
After removing all search peers, add-on still shows performance warnings.
Workaround: Restart a Splunk platform instance after changing its role.
2015-12-21 ADDON-7077
Infrequent Access storage type not supported
2015-12-16 ADDON-7035
Add-on ingests the header line of the CloudFront access log, but it should be skipped.
2015-11-26 ADDON-6701
EC2, RDS, ELB, and EC2 APIs do not consider pagination.
2015-10-14 ADDON-6056
S3 logging errors on Windows.
2015-10-13 ADDON-6043
SQS message mistakenly deleted when the add-on throws an error retrieving data from an S3 bucket.
2015-09-11 ADDON-5500
Preconfigured reports for billing data cannot handle reports that have a mix of different currencies. The report will use the first currency found and apply that to all costs.
2015-09-11 ADDON-5499
CloudWatch: Previous selected Metric namespace always exists in the list regardless of the region change
2015-09-11 ADDON-5498
Unclear error: Unexpected error "<class 'socket.error'>" from python handler: " Connection refused" when user specifies all regions in CloudWatch for one namespace, saves the configuration, and reloads it.
2015-09-10 ADDON-5481
The add-on configuration UI does not handle insufficient Splunk user permissions gracefully.
2015-09-10 ADDON-5471
Deleting a CloudWatch data input takes too long.
2015-09-10 ADDON-5469
Missing or improper default value for un-required fields.
2015-09-07 ADDON-5355
Different error message for same error when creating duplicated data inputs.
2015-09-06 ADDON-5354
Using keyboard to delete selections from configuration dropdown multi-select field causes drop-down list to appear in corner of screen.
2015-09-01 ADDON-5309
UI default value is not read from default input config file
2015-09-01 ADDON-5295
Description inconsistent in the GUI for CloudTrail service and CloudTrail from S3 service blacklist behavior.
2015-08-31 ADDON-5212
Chrome highlights "misspelling" of configuration text in the GUI.
2015-07-06 ADDON-6177
When tmp file system runs out of space, aws_billing.py fails with IOError: No space left on device.
2015-04-02 ADDON-3578
S3: uppercase bucket names cause an error
2015-03-25 ADDON-3460
On OSs (like Debian and Ubuntu) that use dash for shell scripts, aws_cloudwatch.py spawns zombie processes.
Workaround: Kill the processes and restart. Use bash to prevent re-occurrence.
2014-09-28 ADDON-2135
The list of regions shown in inputs configuration in Splunk Web shows all Amazon regions regardless of the permissions associated with the selected AWS account.
2014-09-26 ADDON-2118
Data inputs continue to work after user deletes the account used for that input.
Workaround: Restart the Splunk platform after deleting or modifying an AWS account.
2014-09-25 ADDON-2113
The app.conf file includes a stanza for a proxy server configuration with a hashed password even if the user has not configured a proxy or password.
Workaround: This behavior is expected because Splunk Enterprise automatically sets the proxy field to 0 and saves an encrypted entry in app.conf.
2014-09-16 ADDON-2029
In saved search "Monthly Cost till *" _time is displayed per day rather than per month.

Third-party software attributions

Version 3.0.0 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Version 2.0.1

Version 2.0.1 of the Splunk Add-on for Amazon Web Services has the same compatibility specifications as version 3.0.0.

Fixed issues

Version 2.0.1 of the Splunk Add-on for Amazon Web Services fixes the following issues.

Resolved date Defect number Description
2015-11-04 ADDON-5813 S3 input cannot handle bucket names with "." in them. See "Add an S3 input for the Splunk Add-on for AWS" for details of the solution.
2015-10-28 ADDON-6125 Add-on makes too many unnecessary get_log_event API calls, causing inefficiencies in environments with many spot instances.
2015-10-26 ADDON-5785 Corrupt VPC Flow checkpointer file in race condition.
2015-10-20 ADDON-5612 When CloudTrail userName is null, add-on coalesces the userName to "root" instead of "unknown".
2015-10-15 ADDON-6004 Add-on GUI does not allow user to select an index that is only defined on the indexers.
2015-10-11 ADDON-6003 Incorrect regions shown in region drop-down list.
2015-10-11 ADDON-6001 Config fails to fetch events from an S3 bucket in a different region.
2015-10-09 ADDON-5833 AWS CloudWatch log formatting exception.
2015-10-09 ADDON-4505 Cloudwatchlog deadlocks due to throttling exceptions when an input task includes a large number of log groups.
2015-10-09 ADDON-5782 A corrupted checkpointer file for VPC Flow blocks other logstreams.

Known issues

Version 2.0.1 of the Splunk Add-on for Amazon Web Services has the following known issues.

Date filed Defect number Description
2015-12-15 ADDON-7930 Data collection for Cloudwatch S3 metrics does not support wildcard in BucketName or array length > 1.
2015-11-09 ADDON-6371 In some cases, Splunk Cloud does not save the AWS account credentials after they are correctly entered. Workaround: File a support request to redeploy the add-on and restart the instance.
2015-10-14 ADDON-6056 S3 logging errors on Windows.
2015-10-13 ADDON-6043 SQS message mistakenly deleted when the add-on throws an error retrieving data from an S3 bucket.
2015-09-11 ADDON-5500 Preconfigured reports for billing data cannot handle reports that have a mix of different currencies. The report will use the first currency found and apply that to all costs.
2015-09-11 ADDON-5499 CloudWatch: Previous selected Metric namespace always exists in the list regardless of the region change.
2015-09-11 ADDON-5498 Unclear error message: Failed to load options for Metric namespace. Detailed Error: Unexpected error "<class 'socket.error'>" from python handler: "[Errno 111] Connection refused" when user specifies all regions in CloudWatch for one namespace, saves the configuration, and reloads it.
2015-09-10 ADDON-5481 The add-on configuration UI does not handle insufficient Splunk user permissions gracefully.
2015-09-10 ADDON-5474 No detailed error shown while getting S3 buckets via REST endpoint with wrong proxy or account settings.
2015-09-10 ADDON-5471 Deleting a CloudWatch data input takes too long.
2015-09-10 ADDON-5469 Missing or improper default value for un-required fields.
2015-09-10 ADDON-5491 The add-on configuration UI displays all regions instead of those within the selected account's permission scope.
2015-09-07 ADDON-5355 Different error message for same error when creating duplicated data inputs.
2015-09-06 ADDON-5354 Using keyboard to delete selections from configuration dropdown multi-select field causes drop-down list to appear in corner of screen.
2015-09-01 ADDON-5309 UI default value is not read from default input config file.
2015-09-01 ADDON-5295 Description inconsistent in the GUI for CloudTrail service and CloudTrail from S3 service blacklist behavior.
2015-08-31 ADDON-5212 Chrome highlights "misspelling" of configuration text in the GUI.
2015-07-09 ADDON-3460 /
CO-4749 /
SPL-55904
On OSs (like Debian and Ubuntu) that use dash for shell scripts, aws_cloudwatch.py spawns zombie processes. Workaround: Kill the processes and restart. Use bash to prevent re-occurrence.
2015-07-06 ADDON-6177 aws_billing.py fails with IOError: [Errno 28] No space left on device.
2015-04-03 ADDON-3578 Uppercase bucket name causes errors.
2015-01-22 ADDON-3050/
SPL-96729/
SPL-64904
S3 input is breaking lines incorrectly and inconsistently indexing only partial events.
Workaround: Disable the persistent queue for the S3 input by changing persistentQueueSize = 24MB to persistentQueueSize = 0 in local/inputs.conf.
2015-01-25 ADDON-3070 The add-on does not index the Configuration.State.Code change from SQS that is reported to users on the AWS Config UI. Splunk Enterprise only indexes configuration snapshots from S3 as new events, and only after a "ConfigurationHistoryDeliveryCompleted" notification is recieved by SQS.
2014-09-26 ADDON-2118 Data inputs continue to work after user deletes the account used for that input. Workaround: Restart Splunk Enterprise after deleting or modifying an AWS account.
2014-09-28 ADDON-2135 The list of regions shown in inputs configuration in Splunk Web shows all Amazon regions regardless of the permissions associated with the selected AWS account.
2014-09-26 ADDON-2116/
SPL-91709
On Windows 2012, Splunk Web shows a timeout error when a user attempts to add or delete an AWS account on the setup page. Workaround: Refresh the page.
2014-09-25 ADDON-2113 The app.conf file includes a stanza for a proxy server configuration with a hashed password even if the user has not configured a proxy or password. This behavior is expected because Splunk Enterprise automatically sets the proxy field to 0 and saves an encrypted entry in app.conf.
2014-09-16 ADDON-2029 In saved search "Monthly Cost till *" _time is displayed per day rather than per month.
2014-09-09 ADDON-1983 /
ADDON-1938 /
SPL-81771
Errors can occur in checkpointing if modular input stdout is prematurely closed during termination. Checkpoint and retry time do not log correctly when Splunkd stops.
2014-08-26 ADDON-1919 If a user changes the configuration to use a different AWS account, Splunk Web continues to list buckets for the previously configured account.
2014-08/17 ADDON-1854 After initial configuration, adjusting Max trackable items might cause data loss.
2014-08-14 ADDON-1827 Checkpoints are not cleared after data inputs are removed or the add-on is uninstalled, thus if you create a new input with the same name as the deleted one, the add-on uses the checkpoint from the old input. Workaround: create unique input names to avoid picking up old checkpoint files.

Third-party software attributions

Version 2.0.1 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Version 2.0.0

Version 2.0.0 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms.

Splunk platform versions 6.3, 6.2
CIM 4.0 and above
Platforms Platform independent
Vendor Products AWS CloudTrail, CloudWatch, CloudWatch Logs, Config, Billing, S3

New features

Version 2.0.0 of the Splunk Add-on for Amazon Web Services has the following new features.

Resolved date Defect number Description
2015-09-08 ADDON-1671 New configuration UI.
2015-09-08 ADDON-2126 / ADDON-5466 Ability to manually enter S3 bucket names, SQS queue names, and metric namespaces in Splunk Web fields, in case connection to AWS is poor or user account lacks permissions to list buckets.
2015-07-14 ADDON-4543 Added unified field for AWS account ID across all data inputs: aws_account_id.
2015-07-06 ADDON-3189 Currency field added to AWS billing report data, allowing users to more accurately judge financial impact.
2015-07-03 ADDON-4260 / ADDON-1665 Support for data ingestion from AWS CloudWatch Logs service, including VPC Flow Logs.
2015-07-03 ADDON-4259 CIM mapping for VPC Flow Logs data.
2015-06-30 ADDON-4158 Support for Config snapshot collection.
2015-06-29 ADDON-2364 Support for collecting archives of CloudTrail data via S3 buckets by configuring the sourcetype aws:cloudtrail in an S3 input.
2015-06-29 ADDON-4413 Support for multiple regions in a single CloudWatch input.
2015-06-29 ADDON-3235 Support for disabling SSL proxies using the is_secure parameter in local/aws_global_settings.conf to alter the behavior of connections to AWS.
2015-06-29 ADDON-4180 Support for inventory metadata collection from AWS.

Fixed issues

Version 2.0.0 of the Splunk Add-on for Amazon Web Services fixes the following issues.

Resolved date Defect number Description
2015-09-14 ADDON-5158 CloudTrail data missing some CIM tagging.
2015-08-31 ADDON-5200 CloudWatch input calls AWS API inefficiently, using separate API call for each instance-metric combination.
2015-08-31 ADDON-2006 Unfriendly error message when user specifies invalid account.
2015-08-31 ADDON-1932 Unfriendly error message when configuring proxy incorrectly.
2015-08-31 ADDON-1926 Splunk Web allows you to update and delete an AWS account for the add-on simultaneously.
2015-09-09 ADDON-4822 / CO-4912 Some instances of Splunk Cloud show blank screens for all data input pages. Workaround: Set up a heavy forwarder on-prem to handle data inputs and forward the data to Splunk Cloud.

Known issues

Version 2.0.0 of the Splunk Add-on for Amazon Web Services has the following known issues.

Date filed Defect number Description
2015-10-14 ADDON-6056 S3 logging errors on Windows.
2015-10-13 ADDON-6043 SQS message mistakenly deleted when the add-on throws an error retrieving data from an S3 bucket.
2015-10-09 ADDON-6004 Add-on GUI does not allow user to select an index that is only defined on the indexers.
2015-10-09 ADDON-6003 Incorrect regions shown in region drop-down list.
2015-10-09 ADDON-6001 Confg fails to fetch events from an S3 bucket in a different region.
2015-10-03 ADDON-5833 AWS CloudWatch log formatting exception.
2015-09-28 ADDON-5813 S3 input cannot handle bucket names with "." in them.
2015-09-24 ADDON-5785 Corrupt VPC Flow checkpointer file in race condition.
2015-09-24 ADDON-5782 A corrupted checkpointer file for VPC Flow blocks other logstreams.
2015-09-17 ADDON-5612 When CloudTrail userName is null, add-on coalesces the userName to "root" instead of "unknown".
2015-09-11 ADDON-5500 Preconfigured reports for billing data cannot handle reports that have a mix of different currencies. The report will use the first currency found and apply that to all costs.
2015-09-11 ADDON-5499 CloudWatch: Previous selected Metric namespace always exists in the list regardless of the region change.
2015-09-11 ADDON-5498 Unclear error message: Failed to load options for Metric namespace. Detailed Error: Unexpected error "<class 'socket.error'>" from python handler: "[Errno 111] Connection refused" when user specifies all regions in CloudWatch for one namespace, saves the configuration, and reloads it.
2015-09-10 ADDON-5481 The add-on configuration UI does not handle insufficient Splunk user permissions gracefully.
2015-09-10 ADDON-5491 The add-on configuration UI displays all regions instead of those within the selected account's permission scope.
2015-09-10 ADDON-5474 No detailed error shown while getting S3 buckets via REST endpoint with wrong proxy or account settings.
2015-09-10 ADDON-5471 Deleting a CloudWatch data input takes too long.
2015-09-10 ADDON-5469 Missing or improper default value for un-required fields.
2015-09-07 ADDON-5355 Different error message for same error when creating duplicated data inputs.
2015-09-06 ADDON-5354 Using keyboard to delete selections from configuration dropdown multi-select field causes drop-down list to appear in corner of screen.
2015-09-01 ADDON-5309 UI default value is not read from default input config file.
2015-09-01 ADDON-5295 Description inconsistent in the GUI for CloudTrail service and CloudTrail from S3 service blacklist behavior.
2015-08-31 ADDON-5212 Chrome highlights "misspelling" of configuration text in the GUI.
2015-07-10 ADDON-4505 Cloudwatchlog deadlocks due to throttling exceptions when an input task includes a large number of log groups.
2015-07-09 ADDON-3460 / CO-4749 / SPL-55904 On OSs (like Debian and Ubuntu) that use dash for shell scripts, aws_cloudwatch.py spawns zombie processes. Workaround: Kill the processes and restart. Use bash to prevent re-occurrence.
2015-07-06 ADDON-6177 aws_billing.py fails with IOError: [Errno 28] No space left on device.
2015-04-03 ADDON-3578 Uppercase bucket name causes errors.
2015-01-22 ADDON-3050/
SPL-96729/
SPL-64904
S3 input is breaking lines incorrectly and inconsistently indexing only partial events.
Workaround: Disable the persistent queue for the S3 input by changing persistentQueueSize = 24MB to persistentQueueSize = 0 in local/inputs.conf.
2015-01-25 ADDON-3070 The add-on does not index the Configuration.State.Code change from SQS that is reported to users on the AWS Config UI. Splunk Enterprise only indexes configuration snapshots from S3 as new events, and only after a "ConfigurationHistoryDeliveryCompleted" notification is recieved by SQS.
2014-09-26 ADDON-2118 Data inputs continue to work after user deletes the account used for that input. Workaround: Restart Splunk Enterprise after deleting or modifying an AWS account.
2014-09-28 ADDON-2135 The list of regions shown in inputs configuration in Splunk Web shows all Amazon regions regardless of the permissions associated with the selected AWS account.
2014-09-26 ADDON-2116/
SPL-91709
On Windows 2012, Splunk Web shows a timeout error when a user attempts to add or delete an AWS account on the setup page. Workaround: Refresh the page.
2014-09-25 ADDON-2113 The app.conf file includes a stanza for a proxy server configuration with a hashed password even if the user has not configured a proxy or password. This behavior is expected because Splunk Enterprise automatically sets the proxy field to 0 and saves an encrypted entry in app.conf.
2014-09-16 ADDON-2029 In saved search "Monthly Cost till *" _time is displayed per day rather than per month.
2014-09-09 ADDON-1983 / ADDON-1938 / SPL-81771 Errors can occur in checkpointing if modular input stdout is prematurely closed during termination. Checkpoint and retry time do not log correctly when Splunkd stops.
2014-08-26 ADDON-1919 If a user changes the configuration to use a different AWS account, Splunk Web continues to list buckets for the previously configured account.
2014-08/17 ADDON-1854 After initial configuration, adjusting Max trackable items might cause data loss.
2014-08-14 ADDON-1827 Checkpoints are not cleared after data inputs are removed or the add-on is uninstalled, thus if you create a new input with the same name as the deleted one, the add-on uses the checkpoint from the old input. Workaround: create unique input names to avoid picking up old checkpoint files.

Third-party software attributions

Version 2.0.0 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Version 1.1.1

Version 1.1.1 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms.

Splunk Enterprise versions 6.2, 6.1
CIM 4.2, 4.1, 4.0
Platforms Platform independent
Vendor Products AWS Billing, CloudTrail, CloudWatch, Config, S3

Fixed issues

Version 1.1.1 of the Splunk Add-on for Amazon Web Services fixes the following issues.

Resolved date Defect number Description
04/24/15 ADDON-3512 Timeout error on new account definition. Users can now set splunkdConnectionTimeout = 3000 in $SPLUNK_HOME/etc/system/local/web.conf to avoid setup timeout problems.
04/21/15 ADDON-3612 Add-on cannot parse multi-account message format from SQS and CloudTrail.
04/21/15 ADDON-3577 Input configuration timeout on retrieving bucket/key list from S3.
03/01/15 ADDON-3119 Add-on fails to collect payloads from GovCloud region.

Known issues

Version 1.1.1 of the Splunk Add-on for Amazon Web Services has the following known issues.

Date Defect number Description
08/27/15 ADDON-5158 CloudTrail data missing some CIM tagging.
08/06/15 ADDON-4822 / CO-4581 Some instances of Splunk Cloud show blank screens for all data input pages. Workaround: Set up a heavy forwarder on-prem to handle data inputs and forward the data to Splunk Cloud.
04/10/15 ADDON-3652 Billing reports are not performant.
04/03/15 ADDON-3578 Uppercase bucket name causes errors.
01/22/15 ADDON-3050/
SPL-96729/
SPL-64904
S3 input is breaking lines incorrectly and inconsistently indexing only partial events.
Workaround: Disable the persistent queue for the S3 input by changing persistentQueueSize = 24MB to persistentQueueSize = 0 in local/inputs.conf.
01/25/15 ADDON-3070 The add-on does not index the Configuration.State.Code change from SQS that is reported to users on the AWS Config UI. Splunk Enterprise only indexes configuration snapshots from S3 as new events, and only after a "ConfigurationHistoryDeliveryCompleted" notification is recieved by SQS.
01/06/15 ADDON-2910 Splunk Cloud customers cannot access props.conf to configure line breaking on S3 events.
10/10/14 ADDON-2154 Billing input data has a non-ISO-8601 timestamp appended to the source field of each event. Workaround: Add a new field named "source2" in the suggested format:

..... | rex field=source "(?<source_file>s3://[^:])" | rex field=source "(?<source_date>(csv|zip):.$)" | eval source2=strftime(strptime(substr(source_date, 9),"%d %b %Y %H:%M:%S"),"%Y-%m-%dT%H:%M:%S%z") | eval source2=source_file+":"+source2.

09/26/14 ADDON-2118 Data inputs continue to work after user deletes the account used for that input. Workaround: Restart Splunk Enterprise after deleting or modifying an AWS account.
09/28/14 ADDON-2135 The list of regions shown in inputs configuration in Splunk Web shows all Amazon regions regardless of the permissions associated with the selected AWS account.
09/26/14 ADDON-2116/
SPL-86716
On Windows 2012, Splunk Web shows a timeout error when a user attempts to add or delete an AWS account on the setup page. Workaround: Refresh the page.
09/26/14 ADDON-2115 If user does not provide a friendly name when configuring an AWS account in the setup screen, account is not configured but no error message appears
09/25/14 ADDON-2113 The app.conf file includes a stanza for a proxy server configuration with a hashed password even if the user has not configured a proxy or password. This behavior is expected because Splunk Enterprise automatically sets the proxy field to 0 and saves an encrypted entry in app.conf.
09/25/14 ADDON-2110 In Splunk 6.2, when network is unstable, some input configuration fields fail to display in Splunk Web and no error message is shown.
09/16/14 ADDON-2029 In saved search "Monthly Cost till *" _time is displayed per day rather than per month.
09/11/14 ADDON-2006 Unfriendly error message when user specifies invalid account.
09/09/14 ADDON-1983 If Splunk Enterprise restarts while indexing S3 data, data duplication might occur. Workaround: Use AWS command line tools.
08/28/14 ADDON-1938 Checkpoint and retry time do not log correctly when Splunkd stops.
08/28/14 ADDON-1932 Unfriendly error message when configuring proxy incorrectly.
08/26/14 ADDON-1926 Splunk Web allows you to update and delete an AWS account for the add-on simultaneously.
08/26/14 ADDON-1919 If a user changes the configuration to use a different AWS account, Splunk Web continues to list buckets for the previously configured account.
08/24/14 ADDON-1895 If user tries to update a billing report manually using Microsoft Excel, the add-on cannot process the modified file and throws "failed to parse key" error.
08/21/14 ADDON-1885 Splunk Enterprise does not validate Amazon Web Services credentials during add-on setup.
08/17/14 ADDON-1854 After initial configuration, adjusting Max trackable items might cause data loss.
08/14/14 ADDON-1827 Checkpoints are not cleared after data inputs are removed or the add-on is uninstalled, thus if you create a new input with the same name as the deleted one, the add-on uses the checkpoint from the old input. Workaround: create unique input names to avoid picking up old checkpoint files.
03/12/14 SPL-81771 Errors can occur in checkpointing if modular input stdout is prematurely closed during termination.

Third-party software attributions

Version 1.1.1 of the Splunk Add-on for Amazon Web Services incorporates boto - AWS for Python.


Version 1.1.0

Version 1.1.0 had the same compatibility specifications as Version 1.1.1.

New features

Version 1.1.0 of the Splunk Add-on for Amazon Web Services has the following new features.

Date Issue number Description
02/12/15 ADDON-3148 Support for the SNS Subscription attributes for Raw Message Delivery for AWS Config and CloudTrail.
02/09/15 ADDON-1644 Pre-built panels for CloudWatch, CloudTrail, and Billing data.
12/18/14 ADDON-2678 Allow users to configure the log level.
11/12/14 ADDON-2202 New modular input for AWS Config data.

Fixed issues

Version 1.1.0 of the Splunk Add-on for Amazon Web Services fixes the following issues.

Resolved date Defect number Description
02/11/15 ADDON-2533 Internal logs are source typed as "this-too-small".
02/10/15 ADDON-2679 Process for fetching logs runs in a loop.
02/09/15 ADDON-3154 Support AssumedRole user name for CloudTrail.

Known issues

Version 1.1.0 of the Splunk Add-on for Amazon Web Services has the following known issues.

Date Defect number Description
01/22/15 ADDON-3050 S3 input is breaking lines incorrectly.
01/25/15 ADDON-3070 The add-on does not index the Configuration.State.Code change from SQS that is reported to users on the AWS Config UI. Splunk Enterprise only indexes configuration snapshots from S3 as new events, and only after a "ConfigurationHistoryDeliveryCompleted" notification is recieved by SQS.
01/06/15 ADDON-2910 Splunk Cloud customers cannot access props.conf to configure line breaking on S3 events.
09/28/14 ADDON-2135 The list of regions shown in inputs configuration in Splunk Web shows all Amazon regions regardless of the permissions associated with the selected AWS account.
09/26/14 ADDON-2116 On Windows 2012, Splunk Web shows a timeout error when a user attempts to add or delete an AWS account on the setup page. Workaround: Refresh the page.
09/26/14 ADDON-2115 If user does not provide a friendly name when configuring an AWS account in the setup screen, account is not configured but no error message appears
09/25/14 ADDON-2113 The app.conf file includes a stanza for a proxy server configuration with a hashed password even if the user has not configured a proxy or password. This behavior is expected because Splunk Enterprise automatically sets the proxy field to 0 and saves an encrypted entry in app.conf.
09/25/14 ADDON-2110 In Splunk 6.2, when network is unstable, some input configuration fields fail to display in Splunk Web and no error message is shown.
09/16/14 ADDON-2029 In saved search "Monthly Cost till *" _time is displayed per day rather than per month.
09/11/14 ADDON-2006 Unfriendly error message when user specifies invalid account.
09/09/14 ADDON-1983 If Splunk Enterprise restarts while indexing S3 data, data duplication might occur. Workaround: Use AWS command line tools.
08/28/14 ADDON-1938 Checkpoint and retry time do not log correctly when Splunkd stops.
08/28/14 ADDON-1932 Unfriendly error message when configuring proxy incorrectly.
08/26/14 ADDON-1926 Splunk Web allows you to update and delete an AWS account for the add-on simultaneously.
08/26/14 ADDON-1919 If a user changes the configuration to use a different AWS account, Splunk Web continues to list buckets for the previously configured account.
08/24/14 ADDON-1895 If user tries to update a billing report manually using Microsoft Excel, the add-on cannot process the modified file and throws "failed to parse key" error.
08/21/14 ADDON-1885 Splunk Enterprise does not validate Amazon Web Services credentials during add-on setup.
08/17/14 ADDON-1854 After initial configuration, adjusting Max trackable items might cause data loss.
08/14/14 ADDON-1827 Checkpoints are not cleared after data inputs are removed or the add-on is uninstalled, thus if you create a new input with the same name as the deleted one, the add-on uses the checkpoint from the old input. Workaround: create unique input names to avoid picking up old checkpoint files.
03/12/14 SPL-81771 Errors can occur in checkpointing if modular input stdout is prematurely closed during termination.

Third-party software attributions

Version 1.1.0 of the Splunk Add-on for Amazon Web Services incorporates boto - AWS for Python.


Version 1.0.1

Version 1.0.1 of the Splunk Add-on for Amazon Web Services was compatible with the following software, CIM versions, and platforms.

Splunk Enterprise versions 6.2, 6.1
CIM 4.1, 4.0, 3.0
Platforms Platform independent
Vendor Products AWS Billing, CloudTrail, CloudWatch, S3

Fixed issues

Version 1.0.1 of the Splunk Add-on for Amazon Web Services fixed the following issues.

Resolved date Defect number Description
12/16/14 ADDON-2530 New version of boto library required to support eu-central-1 region.
12/11/14 ADDON-2359 Unexpected SQS messages can block inputs.

Known issues

Version 1.0.1 of the Splunk Add-on for Amazon Web Services has the following known issues.

  • Internal log files are incorrectly sourcetyped as N-too-small. (ADDON-2533)
  • Errors can occur in checkpointing if modular input stdout is prematurely closed during termination. (SPL-81771)
  • After initial configuration, adjusting Max trackable items might cause data loss. (ADDON-1854)
  • Splunk Enterprise does not validate Amazon Web Services credentials during add-on setup. (ADDON-1885)
  • If user tries to update a billing report manually using Microsoft Excel, the add-on cannot process the modified file and throws "failed to parse key" error. (ADDON-1895)
  • If a user changes the configuration to use a different AWS account, Splunk Web continues to list buckets for the previously configured account. (ADDON-1919)
  • Splunk Web allows you to update and delete an AWS account for the add-on simultaneously. (ADDON-1926)
  • Setup and configuration pages in Splunk Web give unfriendly error messages when given invalid inputs (ADDON-1932, ADDON-2006)
  • If Splunk Enterprise restarts while indexing S3 data, data duplication might occur. Workaround: Use AWS command line tools. (ADDON-1983 and ADDON-1938)
  • In saved search "Monthly Cost till *" _time is displayed per day rather than per month. (ADDON-2029)
  • The app.conf file includes a stanza for a proxy server configuration with a hashed password even if the user has not configured a proxy or password. This behavior is expected because Splunk Enterprise automatically sets the proxy field to 0 and saves an encrypted entry in app.conf. (ADDON-2113)
  • If user does not provide a friendly name when configuring an AWS account in the setup screen, account is not configured but no error message appears (ADDON-2115)
  • On Windows 2012, Splunk Web shows a timeout error when a user attempts to add or delete an AWS account on the setup page. Workaround: Refresh the page. (ADDON-2116)
  • The list of regions shown in inputs configuration in Splunk Web shows all Amazon regions regardless of the permissions associated with the selected AWS account. (ADDON-2135)
  • In Splunk 6.2, when network is unstable, some input configuration fields fail to display in Splunk Web and no error message is shown. (ADDON-2110)

Third-party software attributions

Version 1.0.1 of the Splunk Add-on for Amazon Web Services incorporated boto - AWS for Python.

Version 1.0.0

Version 1.0.0 of the Splunk Add-on for Amazon Web Services had the same compatibility specifications as version 1.0.1.

Known issues

Version 1.0.0 of the Splunk Add-on for Amazon Web Services had the following known issues:

  • Errors can occur in checkpointing if modular input stdout is prematurely closed during termination. (SPL-81771)
  • After initial configuration, adjusting Max trackable items might cause data loss. (ADDON-1854)
  • Splunk Enterprise does not validate Amazon Web Services credentials during add-on setup. (ADDON-1885)
  • If user tries to update a billing report manually using Microsoft Excel, the add-on cannot process the modified file and throws "failed to parse key" error. (ADDON-1895)
  • If a user changes the configuration to use a different AWS account, Splunk Web continues to list buckets for the previously configured account. (ADDON-1919)
  • Splunk Web allows you to update and delete an AWS account for the add-on simultaneously. (ADDON-1926)
  • Setup and configuration pages in Splunk Web give unfriendly error messages when given invalid inputs (ADDON-1932, ADDON-2006)
  • If Splunk Enterprise restarts while indexing S3 data, data duplication might occur. Workaround: Use AWS command line tools. (ADDON-1983 and ADDON-1938)
  • In saved search "Monthly Cost till *" _time is displayed per day rather than per month. (ADDON-2029)
  • The app.conf file includes a stanza for a proxy server configuration with a hashed password even if the user has not configured a proxy or password. This behavior is expected because Splunk Enterprise automatically sets the proxy field to 0 and saves an encrypted entry in app.conf. (ADDON-2113)
  • If user does not provide a friendly name when configuring an AWS account in the setup screen, account is not configured but no error message appears (ADDON-2115)
  • On Windows 2012, Splunk Web shows a timeout error when a user attempts to add or delete an AWS account on the setup page. Workaround: Refresh the page. (ADDON-2116)
  • The list of regions shown in inputs configuration in Splunk Web shows all Amazon regions regardless of the permissions associated with the selected AWS account. (ADDON-2135)
  • In Splunk 6.2, when network is unstable, some input configuration fields fail to display in Splunk Web and no error message is shown. (ADDON-2110)

Third-party software attributions

Version 1.0.0 of the Splunk Add-on for Amazon Web Services incorporated boto - AWS for Python.

PREVIOUS
Release notes for the Splunk Add-on for AWS
  NEXT
Installation and configuration overview for the Splunk Add-on for AWS

This documentation applies to the following versions of Splunk® Supported Add-ons: released


Was this documentation topic helpful?

Enter your email address, and someone from the documentation team will respond to you:

Please provide your comments here. Ask a question or make a suggestion.

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters