Splunk® Supported Add-ons

Splunk Add-on for AWS

Acrobat logo Download manual as PDF


Acrobat logo Download topic as PDF

Release history for the Splunk Add-on for AWS

Latest release

The latest version of the Splunk Add-on for Amazon Web Services is version 7.4.1. See Release notes for the Splunk Add-on for AWS for the release notes of this latest version.

Version 7.4.0

Version 7.4.0 of the Splunk Add-on for Amazon Web Services was released on December 21, 2023.

Starting in version 7.1.0 of the Splunk Add-on for AWS, the file based checkpoint mechanism was migrated to the Splunk KV Store for Billing Cost and Usage Report, CloudWatch Metrics, and Incremental S3 inputs. The inputs must be disabled whenever the Splunk software is restarted. Otherwise, it will result in data duplication against your already configured inputs.

Version 7.0.0 of the Splunk Add-on for AWS includes a merge of all the capabilities of the Splunk Add-on for Amazon Security Lake. Configure the Splunk Add-on for AWS to ingest across all AWS data sources for ingesting AWS data into your Splunk platform deployment.

If you use both the Splunk Add-on for Amazon Security Lake as well as the Splunk Add-on for AWS on the same Splunk instance, then you must uninstall the Splunk Add-on for Amazon Security Lake before upgrading the Splunk Add-on for AWS to version 7.0.0 or later in order to avoid any data duplication and discrepancy issues.


Version 6.0.0 of the Splunk Add-on for AWS includes a merge of all the capabilities of the Splunk Add-on for Amazon Kinesis Firehose. Configure the Splunk Add-on for AWS to ingest across all AWS data sources for ingesting AWS data into your Splunk platform deployment.

If you use both the Splunk Add-on for Amazon Kinesis Firehose as well as the Splunk Add-on for AWS on the same Splunk instance, then you must uninstall the Splunk Add-on for Amazon Kinesis Firehose after upgrading the Splunk Add-on for AWS to version 6.0.0 or later in order to avoid any data duplication and discrepancy issues.

Data that you previously onboarded through the Splunk Add-on for Amazon Kinesis Firehose will still be searchable, and your existing searches will be compatible with version 6.0.0 of the Splunk Add-on for AWS.

If you are not currently using the Splunk Add-on for Amazon Kinesis Firehose, but plan to use it in the future, then the best practice is to download and configure version 6.0.0 or later of the Splunk Add-on for AWS, instead of the Splunk Add-on for Amazon Kinesis Firehose.

Compatibility

Version 7.4.0 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms:

Splunk platform versions 8.2.x, 9.0.x
CIM 5.1.1 and later
Supported OS for data collection Platform independent
Vendor products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, EventBridge (CloudWatch API), Inspector Classic, Inspector, Kinesis, S3, VPC Flow Logs, Billing services, Metadata, SQS, SNS, AWS Identity and Access Management (IAM) Access Analyzer, AWS Security Hub findings, and Amazon Security Lake events

The field alias functionality is compatible with the current version of this add-on. The current version of this add-on does not support older field alias configurations.

For more information about the field alias configuration change, refer to the Splunk Enterprise Release Notes.

New features

Version 7.4.0 of the Splunk Add-on for AWS version contains the following new and changed features:

  • Added decoding support for parsing Kinesis Firehose error data. This significant improvement eliminates the necessity of employing a Lambda function for decoding purposes. Consequently, the use of Lambda for the re-ingestion of failed events from S3 is no longer required, streamlining the process and reducing complexity.
  • Enhanced UI experience features.
  • Minor bug fixes.


Fixed issues

Version 7.4.0 of the Splunk Add-on for Amazon Web Services fixes the following, if any, issues:

Date resolved Issue number Description
2023-11-28 ADDON-65023 Extractions not working for sourcetype aws:s3:accesslogs
2023-11-21 ADDON-63652 Incremental S3 input not working on windows

Known issues

Version 7.4.0 of the Splunk Add-on for Amazon Web Services has the following, if any, known issues.

Date filed Issue number Description
2024-01-29 ADDON-68124 Fix Selected API loading issue for metadata input

Third-party software attributions

Version 7.4.0 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Third-party software attributions for the Splunk Add-on for Amazon Web Services

Version 7.3.0

Version 7.3.0 of the Splunk Add-on for Amazon Web Services was released on November 10, 2023.

Starting in version 7.1.0 of the Splunk Add-on for AWS, the file based checkpoint mechanism was migrated to the Splunk KV Store for Billing Cost and Usage Report, CloudWatch Metrics, and Incremental S3 inputs. The inputs must be disabled whenever the Splunk software is restarted. Otherwise, it will result in data duplication against your already configured inputs.

Version 7.0.0 of the Splunk Add-on for AWS includes a merge of all the capabilities of the Splunk Add-on for Amazon Security Lake. Configure the Splunk Add-on for AWS to ingest across all AWS data sources for ingesting AWS data into your Splunk platform deployment.

If you use both the Splunk Add-on for Amazon Security Lake as well as the Splunk Add-on for AWS on the same Splunk instance, then you must uninstall the Splunk Add-on for Amazon Security Lake before upgrading the Splunk Add-on for AWS to version 7.0.0 or later in order to avoid any data duplication and discrepancy issues.


Version 6.0.0 of the Splunk Add-on for AWS includes a merge of all the capabilities of the Splunk Add-on for Amazon Kinesis Firehose. Configure the Splunk Add-on for AWS to ingest across all AWS data sources for ingesting AWS data into your Splunk platform deployment.

If you use both the Splunk Add-on for Amazon Kinesis Firehose as well as the Splunk Add-on for AWS on the same Splunk instance, then you must uninstall the Splunk Add-on for Amazon Kinesis Firehose after upgrading the Splunk Add-on for AWS to version 6.0.0 or later in order to avoid any data duplication and discrepancy issues.

Data that you previously onboarded through the Splunk Add-on for Amazon Kinesis Firehose will still be searchable, and your existing searches will be compatible with version 6.0.0 of the Splunk Add-on for AWS.

If you are not currently using the Splunk Add-on for Amazon Kinesis Firehose, but plan to use it in the future, then the best practice is to download and configure version 6.0.0 or later of the Splunk Add-on for AWS, instead of the Splunk Add-on for Amazon Kinesis Firehose.

Compatibility

Version 7.3.0 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms:

Splunk platform versions 8.2.x, 9.0.x
CIM 5.1.1 and later
Supported OS for data collection Platform independent
Vendor products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, EventBridge (CloudWatch API), Inspector Classic, Inspector, Kinesis, S3, VPC Flow Logs, Billing services, Metadata, SQS, SNS, AWS Identity and Access Management (IAM) Access Analyzer, AWS Security Hub findings, and Amazon Security Lake events

The field alias functionality is compatible with the current version of this add-on. The current version of this add-on does not support older field alias configurations.

For more information about the field alias configuration change, refer to the Splunk Enterprise Release Notes.

New features

Version 7.3.0 of the Splunk Add-on for AWS version contains the following new and changed features:

  • The file checkpoint mechanism was migrated to the Splunk KV store for Inspector, InspectorV2, ConfigRule, CloudwatchLogs and Kinesis inputs.
  • Updated existing Health Check Dashboards in order to enhance troubleshooting and performance monitoring. See AWS Health Check Dashboards for more information.


Fixed issues

Version 7.3.0 of the Splunk Add-on for Amazon Web Services fixes the following, if any, issues:


Known issues

Version 7.3.0 of the Splunk Add-on for Amazon Web Services has the following, if any, known issues.

Date filed Issue number Description
2023-07-31 ADDON-63652 Incremental S3 input not working on windows

Third-party software attributions

Version 7.3.0 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Third-party software attributions for the Splunk Add-on for Amazon Web Services

Version 7.2.0

Version 7.2.0 of the Splunk Add-on for Amazon Web Services was released on October 17, 2023.

Compatibility

Version 7.2.0 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms:

Splunk platform versions 8.2.x, 9.0.x
CIM 5.1.1 and later
Supported OS for data collection Platform independent
Vendor products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, EventBridge (CloudWatch API), Inspector Classic, Inspector, Kinesis, S3, VPC Flow Logs, Billing services, Metadata, SQS, SNS, AWS Identity and Access Management (IAM) Access Analyzer, AWS Security Hub findings, and Amazon Security Lake events

The field alias functionality is compatible with the current version of this add-on. The current version of this add-on does not support older field alias configurations.

For more information about the field alias configuration change, refer to the Splunk Enterprise Release Notes.

New features

Version 7.2.0 of the Splunk Add-on for AWS version contains the following new and changed features:

  • Removed the AWS Lambda function dependency for the aws:cloudwatchlogs:vpcflow sourcetype.
  • Added support to fetch logs from AWS Organization level directory structures using the CloudTrail Incremental S3 input.
  • Enhanced throttle support for the Metadata input in order to mitigate throttling and limit errors.
  • Added the SNS message Max Age parameter to the SQS-based S3 input. This can be used to improve the efficiency of your data collection of messages within specified age limits.


Fixed issues

Version 7.2.0 of the Splunk Add-on for Amazon Web Services fixes the following, if any, issues:


Known issues

Version 7.2.0 of the Splunk Add-on for Amazon Web Services has the following, if any, known issues.

Date filed Issue number Description
2023-07-31 ADDON-63652 Incremental S3 input not working on windows

Third-party software attributions

Version 7.2.0 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Third-party software attributions for the Splunk Add-on for Amazon Web Services

Version 7.1.0

Version 7.1.0 of the Splunk Add-on for Amazon Web Services was released on July 25, 2023.

Starting in version 7.1.0 of the Splunk Add-on for AWS, the file based checkpoint mechanism was migrated to the Splunk KV Store for Billing Cost and Usage Report, CloudWatch Metrics, and Incremental S3 inputs. The inputs must be disabled whenever the Splunk software is restarted. Otherwise, it will result in data duplication against your already configured inputs.

Version 7.0.0 of the Splunk Add-on for AWS includes a merge of all the capabilities of the Splunk Add-on for Amazon Security Lake. Configure the Splunk Add-on for AWS to ingest across all AWS data sources for ingesting AWS data into your Splunk platform deployment.

If you use both the Splunk Add-on for Amazon Security Lake as well as the Splunk Add-on for AWS on the same Splunk instance, then you must uninstall the Splunk Add-on for Amazon Security Lake before upgrading the Splunk Add-on for AWS to version 7.0.0 or later in order to avoid any data duplication and discrepancy issues.


Version 6.0.0 of the Splunk Add-on for AWS includes a merge of all the capabilities of the Splunk Add-on for Amazon Kinesis Firehose. Configure the Splunk Add-on for AWS to ingest across all AWS data sources for ingesting AWS data into your Splunk platform deployment.

If you use both the Splunk Add-on for Amazon Kinesis Firehose as well as the Splunk Add-on for AWS on the same Splunk instance, then you must uninstall the Splunk Add-on for Amazon Kinesis Firehose after upgrading the Splunk Add-on for AWS to version 6.0.0 or later in order to avoid any data duplication and discrepancy issues.

Data that you previously onboarded through the Splunk Add-on for Amazon Kinesis Firehose will still be searchable, and your existing searches will be compatible with version 6.0.0 of the Splunk Add-on for AWS.

If you are not currently using the Splunk Add-on for Amazon Kinesis Firehose, but plan to use it in the future, then the best practice is to download and configure version 6.0.0 or later of the Splunk Add-on for AWS, instead of the Splunk Add-on for Amazon Kinesis Firehose.

Compatibility

Version 7.1.0 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms:

Splunk platform versions 8.2.x, 9.0.x
CIM 5.1.1 and later
Supported OS for data collection Platform independent
Vendor products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, EventBridge (CloudWatch API), Inspector Classic, Inspector, Kinesis, S3, VPC Flow Logs, Billing services, Metadata, SQS, SNS, AWS Identity and Access Management (IAM) Access Analyzer, AWS Security Hub findings, and Amazon Security Lake events

The field alias functionality is compatible with the current version of this add-on. The current version of this add-on does not support older field alias configurations.

For more information about the field alias configuration change, refer to the Splunk Enterprise Release Notes.

New features

Version 7.1.0 of the Splunk Add-on for AWS version contains the following new and changed features:

  • Added support for the following services for the AWS metadata input:
    • EKS
    • ElasticCache
    • EMR
    • GuardDuty
    • Network Firewall
    • Route 53
    • WAF
    • WAF v2
  • Enhancements have been made to the AWS metadata input for below mentioned services:
    • CloudFront
    • EC2
    • ELB
    • IAM
    • Kinesis Data Firehose
    • VPC
  • The file checkpoint mechanism was migrated to the Splunk KV store for Billing Cost and Usage Report, CloudWatch Metrics, and Incremental S3 inputs.

Fixed issues

Version 7.1.0 of the Splunk Add-on for Amazon Web Services fixes the following, if any, issues:

Date resolved Issue number Description
2023-07-20 ADDON-62779 tmp files under aws_sqs_based_s3 in modinputs directory not being deleted, taking up too much disk space
2023-07-06 ADDON-62721 Fixed JSON format Issue for AWS Security Hub data

Known issues

Version 7.1.0 of the Splunk Add-on for Amazon Web Services has the following, if any, known issues.

Date filed Issue number Description
2023-07-31 ADDON-63652 Incremental S3 input not working on windows

Third-party software attributions

Version 7.1.0 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Third-party software attributions for the Splunk Add-on for Amazon Web Services

Version 7.0.0

Version 7.0.0 of the Splunk Add-on for Amazon Web Services was released on May 18th, 2023.

Version 7.0.0 of the Splunk Add-on for AWS includes a merge of all the capabilities of the Splunk Add-on for Amazon Security Lake. Configure the Splunk Add-on for AWS to ingest across all AWS data sources for ingesting AWS data into your Splunk platform deployment.

If you use both the Splunk Add-on for Amazon Security Lake as well as the Splunk Add-on for AWS on the same Splunk instance, then you must uninstall the Splunk Add-on for Amazon Security Lake before upgrading the Splunk Add-on for AWS to version 7.0.0 or later in order to avoid any data duplication and discrepancy issues.


Version 6.0.0 of the Splunk Add-on for AWS includes a merge of all the capabilities of the Splunk Add-on for Amazon Kinesis Firehose. Configure the Splunk Add-on for AWS to ingest across all AWS data sources for ingesting AWS data into your Splunk platform deployment.

If you use both the Splunk Add-on for Amazon Kinesis Firehose as well as the Splunk Add-on for AWS on the same Splunk instance, then you must uninstall the Splunk Add-on for Amazon Kinesis Firehose after upgrading the Splunk Add-on for AWS to version 6.0.0 or later in order to avoid any data duplication and discrepancy issues.

Data that you previously onboarded through the Splunk Add-on for Amazon Kinesis Firehose will still be searchable, and your existing searches will be compatible with version 6.0.0 of the Splunk Add-on for AWS.

If you are not currently using the Splunk Add-on for Amazon Kinesis Firehose, but plan to use it in the future, then the best practice is to download and configure version 6.0.0 or later of the Splunk Add-on for AWS, instead of the Splunk Add-on for Amazon Kinesis Firehose.

Compatibility

Version 7.0.0 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms:

Splunk platform versions 8.2.x, 9.0.x
CIM 5.1.1 and later
Supported OS for data collection Platform independent
Vendor products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, EventBridge (CloudWatch API), Inspector Classic, Inspector, Kinesis, S3, VPC Flow Logs, Billing services, Metadata, SQS, SNS, AWS Identity and Access Management (IAM) Access Analyzer, AWS Security Hub findings, and Amazon Security Lake events

The field alias functionality is compatible with the current version of this add-on. The current version of this add-on does not support older field alias configurations.

For more information about the field alias configuration change, refer to the Splunk Enterprise Release Notes.

New features

Version 7.0.0 of the Splunk Add-on for AWS version contains the following new and changed features:

  • Data input support for the Amazon Security Lake service. Users will now be able to ingest security events from Amazon Security Lake, normalized to the Open Cybersecurity Schema Framework (OCSF) schema. The Amazon Security Lake service makes AWS security events available as multi-event Apache Parquet objects in an S3 bucket. Each object has a corresponding SQS notification, once ready for download. Open Cybersecurity Schema Framework (OCSF) is an open-source project, delivering an extensible framework for developing schemas, along with a vendor-agnostic core security schema.


Fixed issues

Version 7.0.0 of the Splunk Add-on for Amazon Web Services fixes the following, if any, issues:


Known issues

Version 7.0.0 of the Splunk Add-on for Amazon Web Services has the following, if any, known issues.

Date filed Issue number Description
2023-09-18 ADDON-65023 Extractions not working for sourcetype aws:s3:accesslogs
2023-08-04 ADDON-63911, ADDON-63910, ADDON-63912 AWS Metadata input process does not exit if there are more number of inputs.
2023-08-04 ADDON-63910, ADDON-62105, ADDON-63911 Connect timeout on endpoint URL error if Private DNS Names is disabled for S3 VPC endpoint
2023-06-12 ADDON-62779 tmp files under aws_sqs_based_s3 in modinputs directory not being deleted, taking up too much disk space
2023-06-05 ADDON-62721 Fixed JSON format Issue for AWS Security Hub data
2023-05-08 ADDON-62105, ADDON-63910 Disabling and re-enabling a Security Lake input can lead to data duplication.

Third-party software attributions

Version 7.0.0 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Third-party software attributions for the Splunk Add-on for Amazon Web Services

Version 6.4.0

Version 6.4.0 of the Splunk Add-on for Amazon Web Services was released on April 19th, 2023.

Version 6.0.0 of the Splunk Add-on for AWS includes a merge of all the capabilities of the Splunk Add-on for Amazon Kinesis Firehose. Configure the Splunk Add-on for AWS to ingest across all AWS data sources for ingesting AWS data into Splunk.

If you use both the Splunk Add-on for Amazon Kinesis Firehose as well as the Splunk Add-on for AWS on the same Splunk instance, then you must uninstall the Splunk Add-on for Amazon Kinesis Firehose after upgrading the Splunk Add-on for AWS to version 6.0.0 or later in order to avoid any data duplication and discrepancy issues.

Data that you previously onboarded through the Splunk Add-on for Amazon Kinesis Firehose will still be searchable, and your existing searches will be compatible with version 6.0.0 of the Splunk Add-on for AWS.

If you are not currently using the Splunk Add-on for Amazon Kinesis Firehose, but plan to use it in the future, then the best practice is to download and configure version 6.0.0 or later of the Splunk Add-on for AWS, instead of the Splunk Add-on for Amazon Kinesis Firehose.

Compatibility

Version 6.4.0 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms:

Splunk platform versions 8.1.x, 8.2.x, 9.0.x
CIM 5.1.1 and later
Supported OS for data collection Platform independent
Vendor products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, Inspector Classic, Inspector, Kinesis, S3, VPC Flow Logs, Billing services, Metadata, SQS, SNS, AWS Identity and Access Management (IAM) Access Analyzer, and AWS Security Hub findings events

The field alias functionality is compatible with the current version of this add-on. The current version of this add-on does not support older field alias configurations.

For more information about the field alias configuration change, refer to the Splunk Enterprise Release Notes.

New features

Version 6.4.0 of the Splunk Add-on for AWS version contains the following new and changed features:

  • Enhanced CIM support of aws:securityhub:findings source type in order to support the new event format. (Consolidated controls feature)
  • Fixed CIM extractions for the app and user fields and added extractions for user_name in aws:securityhub:findings source type.

If you use both the Splunk Add-on for Amazon Kinesis Firehose as well as the Splunk Add-on for AWS on the same Splunk instance, then you must uninstall the Splunk Add-on for Amazon Kinesis Firehose after upgrading the Splunk Add-on for AWS to version 6.0.0 or later in order to avoid any data duplication and discrepancy issues.
Data that you previously onboarded through the Splunk Add-on for Amazon Kinesis Firehose will still be searchable, and your existing searches will be compatible with version 6.0.0 of the Splunk Add-on for AWS.
If you are not currently using the Splunk Add-on for Amazon Kinesis Firehose, but plan to use it in the future, then the best practice is to download and configure version 6.0.0 or later of the Splunk Add-on for AWS, instead of the Splunk Add-on for Amazon Kinesis Firehose.

Fixed issues

Version 6.4.0 of the Splunk Add-on for Amazon Web Services fixes the following, if any, issues:

Date resolved Issue number Description
2023-04-10 ADDON-59257 : JSONDecodeError in Inspector v1 and Inspector v2 inputs
2023-04-10 ADDON-58897, ADDON-61758 ELB logs - fields not getting extracted after upgrade
2023-04-09 ADDON-61182 Unable to clone inputs that were created in previous versions

Known issues

Version 6.4.0 of the Splunk Add-on for Amazon Web Services has the following, if any, known issues.


Third-party software attributions

Version 6.4.0 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Third-party software attributions for the Splunk Add-on for Amazon Web Services

Version 6.3.2

Version 6.3.2 of the Splunk Add-on for Amazon Web Services was released on February 23, 2023.

Version 6.0.0 of the Splunk Add-on for AWS includes a merge of all the capabilities of the Splunk Add-on for Amazon Kinesis Firehose. Configure the Splunk Add-on for AWS to ingest across all AWS data sources for ingesting AWS data into Splunk.

If you use both the Splunk Add-on for Amazon Kinesis Firehose as well as the Splunk Add-on for AWS on the same Splunk instance, then you must uninstall the Splunk Add-on for Amazon Kinesis Firehose after upgrading the Splunk Add-on for AWS to version 6.0.0 or later in order to avoid any data duplication and discrepancy issues.

Data that you previously onboarded through the Splunk Add-on for Amazon Kinesis Firehose will still be searchable, and your existing searches will be compatible with version 6.0.0 of the Splunk Add-on for AWS.

If you are not currently using the Splunk Add-on for Amazon Kinesis Firehose, but plan to use it in the future, then the best practice is to download and configure version 6.0.0 or later of the Splunk Add-on for AWS, instead of the Splunk Add-on for Amazon Kinesis Firehose.

Compatibility

Version 6.3.2 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms:

Splunk platform versions 8.1.x, 8.2.x, 9.0.x
CIM 4.20 and later
Supported OS for data collection Platform independent
Vendor products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, Inspector Classic, Inspector, Kinesis, S3, VPC Flow Logs, Billing services, Metadata, SQS, SNS, AWS Identity and Access Management (IAM) Access Analyzer, and AWS Security Hub findings events

The field alias functionality is compatible with the current version of this add-on. The current version of this add-on does not support older field alias configurations.

For more information about the field alias configuration change, refer to the Splunk Enterprise Release Notes.

New features

Version 6.3.2 of the Splunk Add-on for AWS version contains the following new and changed features:

  • Security related bug fixes. No new features added.

If you use both the Splunk Add-on for Amazon Kinesis Firehose as well as the Splunk Add-on for AWS on the same Splunk instance, then you must uninstall the Splunk Add-on for Amazon Kinesis Firehose after upgrading the Splunk Add-on for AWS to version 6.0.0 or later in order to avoid any data duplication and discrepancy issues.
Data that you previously onboarded through the Splunk Add-on for Amazon Kinesis Firehose will still be searchable, and your existing searches will be compatible with version 6.0.0 of the Splunk Add-on for AWS.
If you are not currently using the Splunk Add-on for Amazon Kinesis Firehose, but plan to use it in the future, then the best practice is to download and configure version 6.0.0 or later of the Splunk Add-on for AWS, instead of the Splunk Add-on for Amazon Kinesis Firehose.

Fixed issues

Version 6.3.2 of the Splunk Add-on for Amazon Web Services fixes the following, if any, issues:

Date resolved Issue number Description
2023-01-11 ADDON-58978 Incorrect extraction issue with sourcetype

Known issues

Version 6.3.2 of the Splunk Add-on for Amazon Web Services has the following, if any, known issues.

Date filed Issue number Description
2023-03-30 ADDON-61589 After upgrade the Splunk Add-on for AWS to ver 6.3.2 extraction field became unknown

Workaround:
As part of the workaround, the customer can add props.conf file with the below content to the local folder of the add-on and restart Splunk.

(path: $SPLUNK_HOME/etc/apps/Splunk_TA_aws/local)

[aws:cloudwatchlogs:vpcflow] EXTRACT-vpcflowlog=^\s*(\d{4}-\d{2}-\d{2}.\d{2}:\d{2}:\d{2}[.\d\w]*)?\s*^(?<version>\d+)\s+(?<account_id>[^\s]{7,12})\s+(?<interface_id>[^\s]+)\s+(?<src_ip>[^\s]+)\s+(?<dest_ip>[^\s]+)\s+(?<src_port>[^\s]+)\s+(?<dest_port>[^\s]+)\s+(?<protocol_code>[^\s]+)\s+(?P<packets>[^\s]+)\s+(?<bytes>[^\s]+)\s+(?<start_time>[^\s]+)\s+(?<end_time>[^\s]+)\s+(?<vpcflow_action>[^\s]+)\s+(?<log_status>[^\s]+)

Note: This workaround is specific to the default log format of VPC flow log. The regex needs to be changed based on the log format.

2023-03-03 ADDON-61182 Unable to clone inputs that were created in previous versions

Workaround:
None known - yet
2023-02-28 ADDON-61160 Unable to clone inputs that were created in previous versions

Workaround:
Workaround:

This issue can be fixed by checking the "Parse all files as CSV" once and again unchecking to save the input for a non-CSV parsing use case. Note: the "CSV file delimiter" field cannot remain empty while editing or cloning any input.

Third-party software attributions

Version 6.3.2 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Third-party software attributions for the Splunk Add-on for Amazon Web Services

Version 6.3.1

Version 6.3.1 of the Splunk Add-on for Amazon Web Services was released on January 23, 2022.

Version 6.0.0 of the Splunk Add-on for AWS includes a merge of all the capabilities of the Splunk Add-on for Amazon Kinesis Firehose. Configure the Splunk Add-on for AWS to ingest across all AWS data sources for ingesting AWS data into Splunk.

If you use both the Splunk Add-on for Amazon Kinesis Firehose as well as the Splunk Add-on for AWS on the same Splunk instance, then you must uninstall the Splunk Add-on for Amazon Kinesis Firehose after upgrading the Splunk Add-on for AWS to version 6.0.0 or later in order to avoid any data duplication and discrepancy issues.

Data that you previously onboarded through the Splunk Add-on for Amazon Kinesis Firehose will still be searchable, and your existing searches will be compatible with version 6.0.0 of the Splunk Add-on for AWS.

If you are not currently using the Splunk Add-on for Amazon Kinesis Firehose, but plan to use it in the future, then the best practice is to download and configure version 6.0.0 or later of the Splunk Add-on for AWS, instead of the Splunk Add-on for Amazon Kinesis Firehose.

Compatibility

Version 6.3.1 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms:

Splunk platform versions 8.1.x, 8.2.x, 9.0.x
CIM 4.20 and later
Supported OS for data collection Platform independent
Vendor products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, Inspector Classic, Inspector, Kinesis, S3, VPC Flow Logs, Billing services, Metadata, SQS, SNS, AWS Identity and Access Management (IAM) Access Analyzer, and AWS Security Hub findings events

The field alias functionality is compatible with the current version of this add-on. The current version of this add-on does not support older field alias configurations.

For more information about the field alias configuration change, refer to the Splunk Enterprise Release Notes.

New features

Version 6.3.1 of the Splunk Add-on for AWS version contains the following new and changed features:

  • Returned support for the AWS VPC default log format (v1-v2 fields only)
  • Fix for generic S3 upgrade issue


If you use both the Splunk Add-on for Amazon Kinesis Firehose as well as the Splunk Add-on for AWS on the same Splunk instance, then you must uninstall the Splunk Add-on for Amazon Kinesis Firehose after upgrading the Splunk Add-on for AWS to version 6.0.0 or later in order to avoid any data duplication and discrepancy issues.
Data that you previously onboarded through the Splunk Add-on for Amazon Kinesis Firehose will still be searchable, and your existing searches will be compatible with version 6.0.0 of the Splunk Add-on for AWS.
If you are not currently using the Splunk Add-on for Amazon Kinesis Firehose, but plan to use it in the future, then the best practice is to download and configure version 6.0.0 or later of the Splunk Add-on for AWS, instead of the Splunk Add-on for Amazon Kinesis Firehose.

Fixed issues

Version 6.3.1 of the Splunk Add-on for Amazon Web Services fixes the following, if any, issues:

Date resolved Issue number Description
2023-01-18 ADDON-59785 Splunk Add-on for AWS - Working inputs break after upgrading to 6.3.0
2023-01-18 ADDON-59825 AWS v6.3.0 - support for vpcflowlogs v1-v2 log format is broken

Known issues

Version 6.3.1 of the Splunk Add-on for Amazon Web Services has the following, if any, known issues.

Date filed Issue number Description
2023-03-03 ADDON-61182 Unable to clone inputs that were created in previous versions

Workaround:
None known - yet
2023-02-28 ADDON-61160 Unable to clone inputs that were created in previous versions

Workaround:
Workaround:

This issue can be fixed by checking the "Parse all files as CSV" once and again unchecking to save the input for a non-CSV parsing use case. Note: the "CSV file delimiter" field cannot remain empty while editing or cloning any input.

2022-06-16 ADDON-52954 AWS addon: Generic S3 input does not parse/index multiple files in tar without losing events
2022-02-04 ADDON-47713 Sorting of table rows for Input Page is not working as expected

Third-party software attributions

Version 6.3.1 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Third-party software attributions for the Splunk Add-on for Amazon Web Services

Version 6.3.0

Version 6.3.0 of the Splunk Add-on for Amazon Web Services was released on December 12, 2022.

Version 6.0.0 of the Splunk Add-on for AWS includes a merge of all the capabilities of the Splunk Add-on for Amazon Kinesis Firehose. Configure the Splunk Add-on for AWS to ingest across all AWS data sources for ingesting AWS data into Splunk.

If you use both the Splunk Add-on for Amazon Kinesis Firehose as well as the Splunk Add-on for AWS on the same Splunk instance, then you must uninstall the Splunk Add-on for Amazon Kinesis Firehose after upgrading the Splunk Add-on for AWS to version 6.0.0 or later in order to avoid any data duplication and discrepancy issues.

Data that you previously onboarded through the Splunk Add-on for Amazon Kinesis Firehose will still be searchable, and your existing searches will be compatible with version 6.0.0 of the Splunk Add-on for AWS.

If you are not currently using the Splunk Add-on for Amazon Kinesis Firehose, but plan to use it in the future, then the best practice is to download and configure version 6.0.0 or later of the Splunk Add-on for AWS, instead of the Splunk Add-on for Amazon Kinesis Firehose.

Compatibility

Version 6.3.0 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms:

Splunk platform versions 8.1.x, 8.2.x, 9.0.x
CIM 4.20 and later
Supported OS for data collection Platform independent
Vendor products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, Inspector Classic, Inspector, Kinesis, S3, VPC Flow Logs, Billing services, Metadata, SQS, SNS, AWS Identity and Access Management (IAM) Access Analyzer, and AWS Security Hub findings events

The field alias functionality is compatible with the current version of this add-on. The current version of this add-on does not support older field alias configurations.

For more information about the field alias configuration change, refer to the Splunk Enterprise Release Notes.

New features

Version 6.3.0 of the Splunk Add-on for AWS version contains the following new and changed features:

Starting in version 6.3.0 of the Splunk Add-on for AWS, the VPC Flow log extraction format has been updated to include v3-v5 fields. Before upgrading to versions 6.3.0 and higher of the Splunk Add-on for AWS, Splunk platform deployments ingesting AWS VPC Flow Logs must update the log format in AWS VPC to include v3-v5 fields in order to ensure successful field extractions.
For more information on updating the log format in AWS VPC, see the Configure VPC Flow Logs inputs for the Splunk Add-on for AWS topic in this manual.

  • Expanded support for VPC FlowLogs, sourcetype aws:cloudwatchlogs:vpcflow :
    • Ingestion of VPC flow logs via SQS-Based S3.
    • Support for the parsing of v3-v5 fields defined by AWS for VPC flow logs for both the Splunk defined custom log format and the select all log format.
    • Validation of the native delivery of VPC Flow Logs through Kinesis Firehose.
  • The addition of an iam_list_policy API to the Metadata input to fetch data related to:
    • Fetch all policies related to IAM using iam:ListPolicy.
    • Fetch permissions data using iam:GetPolicyVersion.
    • To link the users with policy, the following policies iam:ListUserPolicies and iam:ListAttachedUserPolicies were added to Iam_users data.
  • Support for the ingestion of OversizedChangeNotification events via the AWS Config > Config input.
  • Expanded support for Network Load Balancer (NLB) access logs. The new field elb_type was created to distinguish between ELB, ALB, and NLB access logs.
  • UI input page support to enable/disable CSV parsing and custom delimiter definition for Generic S3 & SQS-based S3.

Fields added and fields removed

See the following list of fields added and fields removed between the Splunk Add-on for AWS 6.2.0 and 6.3.0:

Source-type app Fields added Fields removed
[u'aws:elb:accesslogs'] AWS ELB alpn_client_preference_list, destination_ip, connection_time, tls_named_group, log_version, chosen_cert_arn, alpn_be_protocol, domain_name, listener, tls_cipher, chosen_cert_serial, tls_handshake_time, elb_type, tls_protocol_version, destination_port, type, alpn_fe_protocol, incoming_tls_alert
Source-type action Fields added Fields removed
[u'aws:cloudwatchlogs:vpcflow'] unknown tcp_flags, flow_direction, pkt_dstaddr, subnet_id, instance_id, traffic_path, pkt_srcaddr, sublocation_type, pkt_dst_aws_service, sublocation_id, vpc_id, type, az_id, pkt_src_aws_service timestamp
[u'aws:cloudwatchlogs:vpcflow'] blocked, allowed tcp_flags, flow_direction, pkt_dstaddr, subnet_id, instance_id, traffic_path, pkt_srcaddr, sublocation_type, pkt_dst_aws_service, sublocation_id, vpc_id, type, az_id, pkt_src_aws_service

If you use both the Splunk Add-on for Amazon Kinesis Firehose as well as the Splunk Add-on for AWS on the same Splunk instance, then you must uninstall the Splunk Add-on for Amazon Kinesis Firehose after upgrading the Splunk Add-on for AWS to version 6.0.0 or later in order to avoid any data duplication and discrepancy issues.
Data that you previously onboarded through the Splunk Add-on for Amazon Kinesis Firehose will still be searchable, and your existing searches will be compatible with version 6.0.0 of the Splunk Add-on for AWS.
If you are not currently using the Splunk Add-on for Amazon Kinesis Firehose, but plan to use it in the future, then the best practice is to download and configure version 6.0.0 or later of the Splunk Add-on for AWS, instead of the Splunk Add-on for Amazon Kinesis Firehose.

Fixed issues

Version 6.3.0 of the Splunk Add-on for Amazon Web Services fixes the following, if any, issues:

Date resolved Issue number Description
2022-12-22 ADDON-54804 Generic S3 and SQS-based S3 inputs have field extraction issues for CSV files without a header
2022-12-13 ADDON-54134 Repeated errors in our Splunk Add-On for AWS for Cloudwatch inputs.
2022-12-05 ADDON-54678 Upgrade to AWS TA v6.1 broke Umbrella DNS and Proxy Log ingestion
2022-11-01 ADDON-47714 Dependent Input Fields are not getting reset when the Parent Input Field is reset.
2022-10-25 ADDON-55398 S3 SQS Log ingestion for Custom CSV logs is causing log format corruption since upgrading to version 6.2.0 from version 5.2.1
2022-10-17 ADDON-56641 Issue with parsing csv while using Generic S3 input type
2022-10-12 ADDON-56513 Unable to save input with valid region due to invalid region loading issue.
2022-10-12 ADDON-55728 Getting an error while user try to create input using region like Jakarta, cape town, Hongkong, or Bahrain as these regions are disabled by default
2022-10-12 ADDON-56514 Not getting UI validation message
2022-10-10 ADDON-56013 Customer was unable to configure a CloudTrail input on splunk addon for AWS.
2022-10-05 ADDON-56144 Incorrect parsing of CSV files which have double-quotes (") as a delimiter
2022-09-30 ADDON-55763 Splunk Add-on for AWS fails with TypeError: cannot unpack non-iterable NoneType object
2022-09-29 ADDON-55762 Syntax Error in python file for SNS alert
2022-09-22 ADDON-55677 Unable to create input with "Custom Data Type > SQS" when using cross-account configuration
2022-09-20 ADDON-55810 For SQS and Config Rule input some of the fields are not pre-filled for cloning functionality
2022-08-10 ADDON-53858 All AWS inputs are showing error. "Index out of range"
2022-08-09 ADDON-53520 AWS Add-on for Splunk v6.0.0 cannot download SQS-Based-S3 Data File Generated by SentinelOne
2022-08-09 ADDON-52241 Documentation link provided on input page is not working
2022-08-05 ADDON-54130 IMDS (Instance Metadata Service) in AWS, is insecure

Known issues

Version 6.3.0 of the Splunk Add-on for Amazon Web Services has the following, if any, known issues.

Date filed Issue number Description
2023-03-08 ADDON-61231 Splunk_TA_aws failing to add regions in Config rule inputs
2023-01-18 ADDON-59825 AWS v6.3.0 - support for vpcflowlogs v1-v2 log format is broken
2023-01-16 ADDON-59785 Splunk Add-on for AWS - Working inputs break after upgrading to 6.3.0
2022-12-08 ADDON-58978 Incorrect extraction issue with sourcetype
2022-06-16 ADDON-52954 AWS addon: Generic S3 input does not parse/index multiple files in tar without losing events
2022-02-04 ADDON-47713 Sorting of table rows for Input Page is not working as expected

Third-party software attributions

Version 6.3.0 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Third-party software attributions for the Splunk Add-on for Amazon Web Services



Version 6.2.0

Version 6.2.0 of the Splunk Add-on for Amazon Web Services was released on July 28th, 2022.

Version 6.0.0 of the Splunk Add-on for AWS includes a merge of all the capabilities of the Splunk Add-on for Amazon Kinesis Firehose. Configure the Splunk Add-on for AWS to ingest across all AWS data sources for ingesting AWS data into Splunk.

If you use both the Splunk Add-on for Amazon Kinesis Firehose as well as the Splunk Add-on for AWS on the same Splunk instance, then you must uninstall the Splunk Add-on for Amazon Kinesis Firehose after upgrading the Splunk Add-on for AWS to version 6.0.0 or later in order to avoid any data duplication and discrepancy issues.

Data that you previously onboarded through the Splunk Add-on for Amazon Kinesis Firehose will still be searchable, and your existing searches will be compatible with version 6.0.0 of the Splunk Add-on for AWS.

If you are not currently using the Splunk Add-on for Amazon Kinesis Firehose, but plan to use it in the future, then the best practice is to download and configure version 6.0.0 or later of the Splunk Add-on for AWS, instead of the Splunk Add-on for Amazon Kinesis Firehose.

Compatibility

Version 6.2.0 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms:

Splunk platform versions 8.0 and later
CIM 4.20 and later
Supported OS for data collection Platform independent
Vendor products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, Inspector Classic, Inspector, Kinesis, S3, VPC Flow Logs, Billing services, Metadata, SQS, SNS, AWS Identity and Access Management (IAM) Access Analyzer, and AWS Security Hub findings events

Versions 5.0.0 and above of the Splunk Add-on for AWS are Python 3 releases, and only compatible with Splunk platform versions 8.0.0 and later. To use version 5.0.0 or later of this add-on, upgrade your Splunk platform deployment to version 8.0.0 or later. For users of Splunk platforms 6.x.x and Splunk 7.x.x, the Splunk Add-on for Amazon Web Services version 4.6.1 is supported. Do not upgrade to Splunk Add-on for AWS 5.0.0 or above on these versions of the Splunk platform.

The field alias functionality is compatible with the current version of this add-on. The current version of this add-on does not support older field alias configurations.

For more information about the field alias configuration change, refer to the Splunk Enterprise Release Notes.

New features

Version 6.2.0 of the Splunk Add-on for AWS version contains the following new and changed features:

  • Support for the Inspector v2 API ingestion method.
  • Added Common Information Model (CIM) mappings for Inspector v2.
  • Deprecation of the Description Input
  • Added UI warning message and warning logs for Generic S3 inputs.
  • Bug fixes.

If you use both the Splunk Add-on for Amazon Kinesis Firehose as well as the Splunk Add-on for AWS on the same Splunk instance, then you must uninstall the Splunk Add-on for Amazon Kinesis Firehose after upgrading the Splunk Add-on for AWS to version 6.0.0 or later in order to avoid any data duplication and discrepancy issues.
Data that you previously onboarded through the Splunk Add-on for Amazon Kinesis Firehose will still be searchable, and your existing searches will be compatible with version 6.0.0 of the Splunk Add-on for AWS.
If you are not currently using the Splunk Add-on for Amazon Kinesis Firehose, but plan to use it in the future, then the best practice is to download and configure version 6.0.0 or later of the Splunk Add-on for AWS, instead of the Splunk Add-on for Amazon Kinesis Firehose.

Fixed issues

Version 6.2.0.of the Splunk Add-on for Amazon Web Services fixes the following, if any, issues.

Date resolved Issue number Description
2022-05-17 ADDON-46742 Log ingestion has stopped from S3 buckets using the Splunk Add for AWS 5.2.0

Known issues

Version 6.2.0.of the Splunk Add-on for Amazon Web Services has the following, if any, known issues.

Date filed Issue number Description
2023-01-23 ADDON-59953 AWS Metadata Inputs stop collecting data when add-on permissions are set to App instead of Global

Workaround:
For workaround, please use the Global level permission for the AWS Addon.
2022-12-22 ADDON-59257 : JSONDecodeError in Inspector v1 and Inspector v2 inputs
2022-12-08 ADDON-58978 Incorrect extraction issue with sourcetype
2022-12-05 ADDON-58897, ADDON-61758 ELB logs - fields not getting extracted after upgrade
2022-10-14 ADDON-56641 Issue with parsing csv while using Generic S3 input type
2022-10-11 ADDON-56513 Unable to save input with valid region due to invalid region loading issue.
2022-10-11 ADDON-56514 Not getting UI validation message
2022-09-28 ADDON-56144 Incorrect parsing of CSV files which have double-quotes (") as a delimiter
2022-09-22 ADDON-56013 Customer was unable to configure a CloudTrail input on splunk addon for AWS.

Workaround:
The workaround to configure the environment-level proxy was provided to unblock the customer. Reference: https://splunk.atlassian.net/browse/ADDON-51630?focusedCommentId=9182551
2022-09-19 ADDON-55810 For SQS and Config Rule input some of the fields are not pre-filled for cloning functionality
2022-09-15 ADDON-55763 Splunk Add-on for AWS fails with TypeError: cannot unpack non-iterable NoneType object
2022-09-15 ADDON-55762 Syntax Error in python file for SNS alert
2022-09-14 ADDON-55728 Getting an error while user try to create input using region like Jakarta, cape town, Hongkong, or Bahrain as these regions are disabled by default
2022-09-12 ADDON-55677 Unable to create input with "Custom Data Type > SQS" when using cross-account configuration
2022-08-31 ADDON-55398 S3 SQS Log ingestion for Custom CSV logs is causing log format corruption since upgrading to version 6.2.0 from version 5.2.1
2022-08-11 ADDON-54804 Generic S3 and SQS-based S3 inputs have field extraction issues for CSV files without a header
2022-08-09 ADDON-54678 Upgrade to AWS TA v6.1 broke Umbrella DNS and Proxy Log ingestion

Workaround:
Customer has spun up an onprem IDM with the AWS TA installed with v5.2.0 and is using that to forward the logs they need in the meantime to Cloud
2022-06-16 ADDON-52954 AWS addon: Generic S3 input does not parse/index multiple files in tar without losing events
2022-02-04 ADDON-47714 Dependent Input Fields are not getting reset when the Parent Input Field is reset.
2022-02-04 ADDON-47713 Sorting of table rows for Input Page is not working as expected


Third-party software attributions

Version 6.2.0 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Third-party software attributions for the Splunk Add-on for Amazon Web Services

Version 6.1.0

Version 6.1.0 of the Splunk Add-on for Amazon Web Services was released on July 11, 2022

Version 6.0.0 of the Splunk Add-on for AWS includes a merge of all the capabilities of the Splunk Add-on for Amazon Kinesis Firehose. Configure the Splunk Add-on for AWS to ingest across all AWS data sources for ingesting AWS data into Splunk.

If you use both the Splunk Add-on for Amazon Kinesis Firehose as well as the Splunk Add-on for AWS on the same Splunk instance, then you must uninstall the Splunk Add-on for Amazon Kinesis Firehose after upgrading the Splunk Add-on for AWS to version 6.0.0 or later in order to avoid any data duplication and discrepancy issues.

Data that you previously onboarded through the Splunk Add-on for Amazon Kinesis Firehose will still be searchable, and your existing searches will be compatible with version 6.0.0 of the Splunk Add-on for AWS.

If you are not currently using the Splunk Add-on for Amazon Kinesis Firehose, but plan to use it in the future, then the best practice is to download and configure version 6.0.0 or later of the Splunk Add-on for AWS, instead of the Splunk Add-on for Amazon Kinesis Firehose.

Compatibility

Version 6.1.0 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms:

Splunk platform versions 8.0 and later
CIM 4.20 and later
Supported OS for data collection Platform independent
Vendor products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, Inspector, Kinesis, S3, VPC Flow Logs, Billing services, Metadata, SQS, SNS, AWS Identity and Access Management (IAM) Access Analyzer, and AWS Security Hub findings events

Versions 5.0.0 and above of the Splunk Add-on for AWS are Python 3 releases, and only compatible with Splunk platform versions 8.0.0 and later. To use version 5.0.0 or later of this add-on, upgrade your Splunk platform deployment to version 8.0.0 or later. For users of Splunk platforms 6.x.x and Splunk 7.x.x, the Splunk Add-on for Amazon Web Services version 4.6.1 is supported. Do not upgrade to Splunk Add-on for AWS 5.0.0 or above on these versions of the Splunk platform.

The field alias functionality is compatible with the current version of this add-on. The current version of this add-on does not support older field alias configurations.

For more information about the field alias configuration change, refer to the Splunk Enterprise Release Notes.

New features

Version 6.1.0 of the Splunk Add-on for AWS version contains the following new and changed features:

  • Support for the parsing of CSV files from AWS S3 (Generic S3 and SQS-based S3 ingestion methods)
  • Bug fixes.

If you use both the Splunk Add-on for Amazon Kinesis Firehose as well as the Splunk Add-on for AWS on the same Splunk instance, then you must uninstall the Splunk Add-on for Amazon Kinesis Firehose after upgrading the Splunk Add-on for AWS to version 6.0.0 or later in order to avoid any data duplication and discrepancy issues.
Data that you previously onboarded through the Splunk Add-on for Amazon Kinesis Firehose will still be searchable, and your existing searches will be compatible with version 6.0.0 of the Splunk Add-on for AWS.
If you are not currently using the Splunk Add-on for Amazon Kinesis Firehose, but plan to use it in the future, then the best practice is to download and configure version 6.0.0 or later of the Splunk Add-on for AWS, instead of the Splunk Add-on for Amazon Kinesis Firehose.

Fixed issues

Version 6.1.0.of the Splunk Add-on for Amazon Web Services fixes the following, if any, issues.

Date resolved Issue number Description
2022-06-27 ADDON-53189, ADDON-51630 AWS TA - SNS validation not using TAs proxy settings
2022-06-23 ADDON-41472 Splunk_TA_aws Account creation fails for China region because cn sts domain not used
2022-05-17 ADDON-24471 Billing input causes double-ingest of CUR billing files when splunk restarts during ingest
2022-05-12 ADDON-49902 source=*:ec2_ebs_snapshots is importing >30K unwanted EC2 Snapshots

Known issues

Version 6.1.0.of the Splunk Add-on for Amazon Web Services has the following, if any, known issues.

Date filed Issue number Description
2022-08-31 ADDON-55398 S3 SQS Log ingestion for Custom CSV logs is causing log format corruption since upgrading to version 6.2.0 from version 5.2.1
2022-08-11 ADDON-54804 Generic S3 and SQS-based S3 inputs have field extraction issues for CSV files without a header
2022-08-09 ADDON-54678 Upgrade to AWS TA v6.1 broke Umbrella DNS and Proxy Log ingestion

Workaround:
Customer has spun up an onprem IDM with the AWS TA installed with v5.2.0 and is using that to forward the logs they need in the meantime to Cloud
2022-06-16 ADDON-52954 AWS addon: Generic S3 input does not parse/index multiple files in tar without losing events
2022-02-04 ADDON-47714 Dependent Input Fields are not getting reset when the Parent Input Field is reset.
2022-02-04 ADDON-47713 Sorting of table rows for Input Page is not working as expected

Third-party software attributions

Version 6.1.0 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Third-party software attributions for the Splunk Add-on for Amazon Web Services

Version 6.0.0

Version 6.0.0 of the Splunk Add-on for Amazon Web Services was released on May 3, 2022

Version 6.0.0 of the Splunk Add-on for AWS includes a merge of all the capabilities of the Splunk Add-on for Amazon Kinesis Firehose. Configure the Splunk Add-on for AWS to ingest across all AWS data sources for ingesting AWS data into Splunk.

If you use both the Splunk Add-on for Amazon Kinesis Firehose as well as the Splunk Add-on for AWS on the same Splunk instance, then you must uninstall the Splunk Add-on for Amazon Kinesis Firehose after upgrading the Splunk Add-on for AWS to version 6.0.0 or later in order to avoid any data duplication and discrepancy issues.

Data that you previously onboarded through the Splunk Add-on for Amazon Kinesis Firehose will still be searchable, and your existing searches will be compatible with version 6.0.0 of the Splunk Add-on for AWS.

If you are not currently using the Splunk Add-on for Amazon Kinesis Firehose, but plan to use it in the future, then the best practice is to download and configure version 6.0.0 or later of the Splunk Add-on for AWS, instead of the Splunk Add-on for Amazon Kinesis Firehose.

Compatibility

Version 6.0.0 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms:

Splunk platform versions 8.0 and later
CIM 4.20 and later
Supported OS for data collection Platform independent
Vendor products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, Inspector, Kinesis, S3, VPC Flow Logs, Billing services, Metadata, SQS, SNS, AWS Identity and Access Management (IAM) Access Analyzer, and AWS Security Hub findings events

Versions 5.0.0 and above of the Splunk Add-on for AWS are Python 3 releases, and only compatible with Splunk platform versions 8.0.0 and later. To use version 5.0.0 or later of this add-on, upgrade your Splunk platform deployment to version 8.0.0 or later. For users of Splunk platforms 6.x.x and Splunk 7.x.x, the Splunk Add-on for Amazon Web Services version 4.6.1 is supported. Do not upgrade to Splunk Add-on for AWS 5.0.0 or above on these versions of the Splunk platform.

The field alias functionality is compatible with the current version of this add-on. The current version of this add-on does not support older field alias configurations.

For more information about the field alias configuration change, refer to the Splunk Enterprise Release Notes.

New features

Version 6.0.0 of the Splunk Add-on for AWS version contains the following new and changed features:

  • Version 6.0.0 of the Splunk Add-on for AWS includes a merge of all the capabilities of the Splunk Add-on for Amazon Kinesis Firehose:
    • Provided support of all the following vendor products which were supported in the Splunk Add-on for Amazon Kinesis Firehose: AWS Identity and Access Management (IAM) Access Analyzer, and AWS Security Hub findings events.
    • Support for HTTP Event Collector (HEC) data collection for AWS Cloudtrail, AWS VPC Flowlogs, AWS Guardduty, AWS Identity and Access Management (IAM) Access Analyzer and AWS Security Hub findings.
    • Support for the aws:cloudwatch:guardduty Splunk Add-on for Kinesis Firehose sourcetype. Support for the aws:cloudwatchlogs:guardduty sourcetype will be added to a future release of the Splunk Add-on for Amazon Web Services.
  • Improved Common Information Model (CIM) mappings.
  • UI component upgrades for compatibility with future versions of the Splunk software (Fast and intuitive UI with an improved look and feel).
  • Added signature validation for SNS/SQS messages.
  • Added Data Manager banner on the Splunk Add-on for AWS home page.
  • Updated the source for the Metadata data input to match Data Manager functionality.

If you use both the Splunk Add-on for Amazon Kinesis Firehose as well as the Splunk Add-on for AWS on the same Splunk instance, then you must uninstall the Splunk Add-on for Amazon Kinesis Firehose after upgrading the Splunk Add-on for AWS to version 6.0.0 or later in order to avoid any data duplication and discrepancy issues.
Data that you previously onboarded through the Splunk Add-on for Amazon Kinesis Firehose will still be searchable, and your existing searches will be compatible with version 6.0.0 of the Splunk Add-on for AWS.
If you are not currently using the Splunk Add-on for Amazon Kinesis Firehose, but plan to use it in the future, then the best practice is to download and configure version 6.0.0 or later of the Splunk Add-on for AWS, instead of the Splunk Add-on for Amazon Kinesis Firehose.

Fixed issues

Version 6.0.0.of the Splunk Add-on for Amazon Web Services fixes the following, if any, issues.

Date resolved Issue number Description
2022-05-06 ADDON-17910 Rest endpoint /splunk_ta_aws/settings/account should not be exposed to Splunk Web
2022-05-06 ADDON-47321, SPL-217156 The Splunk Add-on for AWS Inputs/Configuration Pages/Tabs Fail to Load (Seeing spinning icon) after upgrade from 5.2.0 to 5.2.1 on NOAH
2022-05-06 ADDON-49879 PreConditioning Failure: AWS TA SQS-based S3 for versioned buckets
2022-05-05 ADDON-47661 AWS config and input page in a constant "loading" state when IMDSV2 enabled on EC2 instance.
2022-05-05 ADDON-46187 5.2.1 SNS Signature verification does not check that the message is actually from SNS
2022-04-29 ADDON-46596 Add-on for AWS can't get logs from AWS
2022-03-21 ADDON-41767 Add T3 burstable instances to the Metrics collection on addon
2022-03-16 ADDON-46852 SQS-based S3 input does not handle space character in S3 object name
2022-03-13 ADDON-44918 AWS TA SQS-based S3 inputs do not handle versioned buckets properly

Known issues

Version 6.0.0.of the Splunk Add-on for Amazon Web Services has the following, if any, known issues.

Date filed Issue number Description
2022-09-19 ADDON-55810 For SQS and Config Rule input some of the fields are not pre-filled for cloning functionality
2022-09-12 ADDON-55677 Unable to create input with "Custom Data Type > SQS" when using cross-account configuration
2022-07-25 ADDON-54130 IMDS (Instance Metadata Service) in AWS, is insecure
2022-07-25 ADDON-54134 Repeated errors in our Splunk Add-On for AWS for Cloudwatch inputs.
2022-07-04 ADDON-53520 AWS Add-on for Splunk v6.0.0 cannot download SQS-Based-S3 Data File Generated by SentinelOne
2022-06-27 ADDON-53189, ADDON-51630 AWS TA - SNS validation not using TAs proxy settings
2022-05-25 ADDON-52241 Documentation link provided on input page is not working
2022-04-21 ADDON-50908 Browser back button doesn't cancel the "Advanced mode" of Cloudwatch input
2022-02-06 ADDON-47727 Getting error while collecting description data with proxy
2022-02-04 ADDON-47714 Dependent Input Fields are not getting reset when the Parent Input Field is reset.
2022-02-04 ADDON-47713 Sorting of table rows for Input Page is not working as expected

Added/Removed Common Information Model Fields

See the following table for a list of fields added/removed CIM fields between Splunk Add-on for Amazon Web Services v5.2.2 and v6.0.0:

Sourcetype eventName Fields added in AWS 5.2.2 Fields removed in AWS 6.0.0
['aws:cloudtrail'] DeleteNetworkInterface object_id, action, status, user, src_user_type, object_attrs, src_user, user_id, object
['aws:cloudtrail'] UpdateUser user_id
Source-type source Fields added in AWS 5.2.2 Fields removed in AWS 6.0.0
['aws:metadata'] All image_id

See the following table for a list of fields added/removed between Splunk Add-on for Amazon Kinesis Firehose v1.3.2 and Splunk Add-on for Amazon Web Services v6.0.0:

Source-type eventName Fields added in Kinesis 1.3.2 Fields removed in AWS 6.0.0
['aws:cloudtrail'] ListAliases object_attrs
Source-type State Fields added in Kinesis 1.3.2 Fields removed in AWS 6.0.0
['aws:metadata'] All availability_zone, instance_tenancy, currency_code, instance_count, duration, fixed_price, end, region, description, vm_os, vendor_region, start, vendor_product, offering_type, state, mem_capacity, vm_size, cpu_cores, usage_price, aws_account_id, vendor_account, id
Source-type source Fields added in Kinesis 1.3.2 Fields removed in AWS 6.0.0
['aws:securityhub:finding'] aws_eventbridgeevents_securityhub instance_extract, vpc_extract, accesskey_extract, volume_extract, security_group_extract, managed_instance_extract, s3bucket_extract


See the following table for a list of fields modified between Splunk Add-on for Amazon Web Services v5.2.2 and v6.0.0:

Sourcetype CIM Field eventName, Resources{}.Type Vendor Field in AWS 5.2.2 Vendor Field in AWS 6.0.0
aws:cloudtrail user eventName: ConsoleLogin userIdentity.principalId,
example: AIDA3HRA7T6MUVTYUHPKV
userIdentity.principalId OR userIdentity.userName,
example: AIDA3HRA7T6MUVTYUHPKV, test_user
user_id eventName: CreateUser, DeleteUser userIdentity.principalId OR userIdentity.accountId OR userIdentity.sessionContext.sessionIssuer.principalId,
example: AIDA3HRA7T6MUVTYUHPKV
userIdentity.principalId OR userIdentity.accountId OR userIdentity.sessionContext.sessionIssuer.principalId OR userIdentity.userName,
example: AIDA3HRA7T6MUVTYUHPKV, test_user
action eventName: DeleteLoginProfile Static Value: deleted,unknown Static Value: modified
object_category eventName: DeleteNetworkInterface Static Value: unknown Static Value: network_interface
user_type eventName: DeleteNetworkInterface userIdentity.type,
example: Assume Role
sessionContext.sessionIssuer.type,
example: Role

See the following table for a list of fields modified between Splunk Add-on for Amazon Kinesis Firehose v1.3.2 and Splunk Add-on for Amazon Web Services v6.0.0:

Sourcetype CIM Field eventName, Resources{}.Type Vendor Field in Kinesis 1.3.2 Vendor Field in AWS 6.0.0
aws:cloudtrail action eventName: DeleteLoginProfile Static Value: deleted, unknown Static Value: modified
status eventName: DeleteNetworkInterface Static Value: failure Static Value: failure, success
aws:cloudwatchlogs:vpcflow dvc All Static Value: VPC Flow interface_id,
example: eni-11302624
aws:metadata account_id All account_id,
example: 906585968227
OwnerId,
example: 404565499102
aws:securityhub:finding dest Resources{}.Type: AwsEc2Instance, AwsEc2Volume, AwsIamAccessKey, AwsS3Bucket, AwsEc2Volume, AwsEc2Vpc Resources.Details.AwsEc2Instance.IpV4Addresses,
example: 127.0.0.1
Resources{}.Id,
i-0259101da3a8675d0
dest_name Resources{}.Type: AwsEc2Instance, AwsEc2Volume, AwsIamAccessKey, AwsS3Bucket, AwsEc2Volume, AwsEc2Vpc Resources{}.Id,
i-0259101da3a8675d0

CIM model changes

Source eventName Previous CIM model in AWS 5.2.2 New CIM model in AWS 6.0.0
Sourcetype State Previous CIM model in Kinesis 1.3.2 New CIM model in AWS 6.0.0
aws:metadata All Inventory.All_Inventory.Virtual_OS.Snapshot


Third-party software attributions

Version 6.0.0 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Third-party software attributions for the Splunk Add-on for Amazon Web Services

Version 5.2.0

Version 5.2.0 of the Splunk Add-on for Amazon Web Services was released on October 4, 2021.

Compatibility

Version 5.2.0 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms:

Splunk platform versions 8.0 and later
CIM 4.20 and later
Supported OS for data collection Platform independent
Vendor products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, Inspector, Kinesis, S3, VPC Flow Logs, Billing services, Metadata, SQS, and SNS.

Versions 5.0.0 and above of the Splunk Add-on for AWS are Python 3 releases, and only compatible with Splunk platform versions 8.0.0 and later. To use version 5.0.0 or later of this add-on, upgrade your Splunk platform deployment to version 8.0.0 or later. For users of Splunk platforms 6.x.x and Splunk 7.x.x, the Splunk Add-on for Amazon Web Services version 4.6.1 is supported. Do not upgrade to Splunk Add-on for AWS 5.0.0 or above on these versions of the Splunk platform.

The field alias functionality is compatible with the current version of this add-on. The current version of this add-on does not support older field alias configurations.

For more information about the field alias configuration change, refer to the Splunk Enterprise Release Notes.

New features

Version 5.2.0 of the Splunk Add-on for AWS version contains the following new and changed features:

  • CIM 4.20 compatibility and enhanced CIM mapping
  • UI component upgrades (jQuery) that are compatible with future versions of the Splunk software.
  • The aws:cloudtrail sourcetype is updated for app field mapping.

See the following tables for information on field changes between 5.1.0 and 5.2.0:

Source-type Fields added Fields removed
aws:cloudfront:accesslogs action, app, bytes, bytes_in, bytes_out, c_port, category, cs_protocol_version, dest, duration, fle_encrypted_fields, fle_status, http_content_type, http_method, http_referrer, http_referrer_domain, http_user_agent, http_user_agent_length, response_time, sc_content_len, sc_content_type, sc_range_end, sc_range_start, src,src_ip, src_port, status, time_to_first_byte, uri_path, url, url_domain, url_length, vendor_product, x_edge_detail_result_type
aws:cloudtrail action, authentication_method, change_type, dest, men_free, object, object_attrs, object_id, rule_action, src_user, src_user_name, src_user_type, status, user_name, vendor_account, vendor_product user_agent, user_id, user_type
aws:cloudwatchlogs:guardduty body, findingType
aws:cloudwatchlogs:vpcflow app, protocol_version, user_id, vendor_product,
aws:config object_id, object_path, result, vendor_account, vendor_product,
aws:config:notification object_attrs, object_path, result, user, vendor_product
aws:description enabled, user_id, family, status, description, time, type, snapshot
aws:elb:accesslogs ActionExecuted, ChosenCertArn, ClientPort, DomainName, ELB, ELBStatusCode, ErrorReason, MatchedRulePriority, ReceivedBytes, RedirectUrl, Request, RequestCreationTime, RequestProcessingTime, RequestTargetIP, RequestTargetPort, RequestType, ResponseProcessingTime, ResponseTime, SSLCipher, SSLProtocol, SentBytes, TargetGroupArn, TargetPort, TargetProcessingTime, TargetStatusCode, TraceId, UserAgent, action, app, bytes, bytes_in, bytes_out, category, dest, dest_port, http_method, http_user_agent, http_user_agent_length, response_time, src, src_ip, src_port, status, url, url_length, vendor_product
aws:metadata enabled, region, snapshot, status, time, user_id, vendor_region
aws:s3 AuthType, BucketCreationTime, BucketName, BucketOwner, BytesSent, CipherSuite, ErrorCode, HTTPMethod, HTTPStatus, HostHeader, HostId, ObjectSize, OperationKey, Referer, RemoteIp, RequestID, RequestKey, RequestURI, RequestURIPath, Requester, SignatureVersion, TLSVersion, TotalTime, TurnAroundTime, UserAgent, VersionId, action, bytes, bytes_out, category, dest, error_code, http_method, http_user_agent, http_user_agent_length, operation,response_time, src, src_ip, status, storage_name, url, url_domain, url_length, user, vendor_product
aws:s3:accesslogs action, category, http_referrer, http_referrer_domain, http_user_agent_length, src_ip,status, storage_name, url, url_length, vendor_product

See the following table for a list of fields modified between 5.1.0 and 5.2.0:

Sourcetype CIM Field eventName, resourceID, resourceType, or source Vendor Field in 5.1.0 Vendor Field in 5.2.0
aws:cloudtrail app eventName: All eventSource,
example: sts.amazonaws.com
eventType,
example: AwsApiCall
user eventName: AssumeRole userIdentity.principalId,
example: AIDA3HRA7T6MUVTYUHPKV
requestParameters.roleArn OR responseElements.assumedRoleUser.arn,
example: Role2WithTags
eventNames: AssumeRoleWithSAML, AssumeRoleWithWebIdentity userIdentity.principalId,
example: AIDA3HRA7T6MUVTYUHPKV
requestParameters.roleArn,
example: arnRoleSession@abc.com
eventNames: AttachVolume, AuthorizeSecurityGroupEgress, AuthorizeSecurityGroupIngress, CheckMfa, ConsoleLogin, CreateAccessKey, CreateBucket, CreateChangeSet, CreateDeliveryStream, CreateFunction20150331, CreateKeyspace, CreateLoadBalancerListeners, CreateLoadBalancerPolicy, CreateLogGroup, CreateLogStream, CreateLoginProfile, CreateNetworkAcl, CreateNetworkAclEntry, CreateNetworkInterface, CreateQueue, CreateSecurityGroup, CreateTable, CreateUser, CreateVirtualMFADevice, CreateVolume, DeleteNetworkAcl, DeleteNetworkAclEntry, DeleteSecurityGroup, DeleteVolume, DetachVolume, GetFederationToken, GetSessionToken, PutBucketAcl, PutBucketPublicAccessBlock, PutObject, RebootInstances, RevokeSecurityGroupEgress, ReplaceNetworkAclAssociation, ReplaceNetworkAclEntry, RevokeSecurityGroupIngress userIdentity.principalId,
example: AIDA3HRA7T6MUVTYUHPKV
userIdentity.userName,
example: test_user
eventNames: GetAccountSummary, GetUser, ListAccessKeys, ListAccountAliases, ListSigningCertificates - Failure Event userIdentity.principalId,
example: AIDA3HRA7T6MUVTYUHPKV
errorMessage,
example: userName
eventNames: GetBucketEncryption, ListAliases, ListRoles userIdentity.principalId,
example: AIDA3HRA7T6MUVTYUHPKV
userIdentity.sessionContext.sessionIssuer.userName,
example: SessionUserName
eventName: PutBucketAcl requestParameters.AccessControlPolicy.AccessControlList.Grant{}.Grantee.DisplayName OR requestParameters.AccessControlPolicy.AccessControlList.Grant{}.Grantee.URI,
example: splunk_aws_dsg_sa
userIdentity.userName,
example: test_user
eventNames: RunInstances, StartInstances, StopInstances, TerminateInstances userIdentity.principalId,
example: AIDA3HRA7T6MUVTYUHPKV
userIdentity.userName OR userIdentity.sessionContext.sessionIssuer.userName,
example: test_user
eventName: UpdateUser requestParameters.userName,
example: OldUserName
requestParameters.newUserName,
example: test_new_user
user_type eventNames: AssumeRole, AssumeRoleWithSAML, AssumeRoleWithWebIdentity userIdentity.type,
example: AWS::IAM::Role
resources{}.type OR responseElements.assumedRoleUser.arn,
example: AWS::IAM::Role
eventNames: ListAliases, ListRoles userIdentity.type,
example: AWS::IAM::Role
userIdentity.sessionContext.sessionIssuer.type,
example: Role
eventName: PutBucketAcl requestParameters.AccessControlPolicy.AccessControlList.Grant{}.Grantee.xsi:type,
example: CanonicalUser
userIdentity.type,
example: AWS::IAM::Role
src_user eventNames: AssumeRole, AssumeRoleWithSAML, AssumeRoleWithWebIdentity userIdentity.principalId,
example: AIDA3HRA7T6MUVTYUHPKV
userIdentity.userName OR requestParameters.sourceIdentity OR userIdentity.sessionContext.sessionIssuer.userName,
example: test_user
eventName: CreateUser userIdentity.principalId,
example: AIDA3HRA7T6MUVTYUHPKV:abc@abc.com
userIdentity.principalId,
example: abc@abc.com
eventNames: DeleteUser, GetUser, PutBucketAcl, UpdateUser userIdentity.principalId,
example: AIDA3HRA7T6MUVTYUHPKV
userIdentity.userName,
example: test_user
src_user_id eventNames: AssumeRole, AssumeRoleWithSAML userIdentity.principalId,
example: AIDA3HRA7T6MUVTYUHPKV:abc@abc.com
userIdentity.principalId OR userIdentity.sessionContext.sessionIssuer.principalId,
example: AIDA3HRA7T6MUVTYUHPKV
user_id AssumeRole, AssumeRoleWithSAML, AssumeRoleWithWebIdentity,
example: responseElements.assumedRoleUser.assumedRoleId
userIdentity.principalId,
example: AIDA3HRA7T6MUVTYUHPKV
responseElements.assumedRoleUser.assumedRoleId
eventNames: AttachVolume, AuthorizeSecurityGroupEgress, AuthorizeSecurityGroupIngress, CreateAccessKey, CreateBucket, CreateChangeSet, CreateDeliveryStream, CreateFunction20150331, CreateNetworkAcl, CreateNetworkAclEntry, CreateSecurityGroup, CreateTable, CreateVirtualMFADevice, DeleteBucket, DeleteNetworkAcl, DeleteSecurityGroup, DeleteVolume, GetAccountSummary, ListSigningCertificates, PutBucketPublicAccessBlock, RebootInstances, ReplaceNetworkAclEntry, RevokeSecurityGroupEgress, RevokeSecurityGroupIngress, RunInstances, StartInstances, StopInstances, TerminateInstances userIdentity.principalId,
example: AIDA3HRA7T6MUVTYUHPKV
userIdentity.userName,
example: test_user
eventName: ConsoleLogin userIdentity.principalId,
example: AIDA3HRA7T6MUVTYUHPKV:abc@abc.com
userIdentity.principalId OR userIdentity.accountId OR userIdentity.sessionContext.sessionIssuer.principalId,
example: AIDA3HRA7T6MUVTYUHPKV
eventNames: ListAliases, ListRoles userIdentity.principalId,
example: AROACKCEVSQ6C2EXAMPLE:Session_Name
userIdentity.sessionContext.sessionIssuer.principalId,
example: AROACKCEVSQ6C2EXAMPLE
object_category eventNames: AttachVolume, DeleteVolume, DetachVolume Static Value: disk Static Value: volume
eventNames: AuthorizeSecurityGroupEgress, AuthorizeSecurityGroupIngress, CreateSecurityGroup, DeleteSecurityGroup, RevokeSecurityGroupEgress, RevokeSecurityGroupIngress Static Value: firewall Static Value: security_group
eventNames: CreateAccessKey, CreateLoginProfile, CreateVirtualMFADevice, GetAccountSummary, GetUser, ListAccessKeys, ListAccountAliases, ListRoles, ListSigningCertificates Static Value: unknown Static Value: user
eventNames: CreateBucket, DeleteBucket, PutBucket, PublicAccessBlock, PutObject Static Value: storage Static Value: bucket
eventName: CreateChangeSet Static Value: unknown Static Value: stack
eventName: CreateDeliveryStream Static Value: unknown Static Value: delivery_stream
eventName: CreateFunction20150331 Static Value: unknown Static Value: function
eventName: CreateKeyspace Static Value: unknown Static Value: keyspace
eventNames: CreateLoadBalancerListeners, CreateLoadBalancerPolicy Static Value: unknown Static Value: load_balancer
eventName: CreateLogGroup Static Value: unknown Static Value: log_group
eventName: CreateLogStream Static Value: unknown Static Value: log_stream
eventNames: CreateNetworkAcl, CreateNetworkAclEntry, DeleteNetworkAcl, DeleteNetworkAclEntry, ReplaceNetworkAclAssociation, ReplaceNetworkAclEntry Static Value: unknown Static Value: ACL
eventName: CreateNetworkInterface Static Value: unknown Static Value: network_interface
eventName: CreateQueue Static Value: unknown Static Value: message_queue
eventName: CreateTable Static Value: unknown Static Value: table
eventNames: GetBucketEncryption, PutBucketAcl Static Value: unknown Static Value: bucket
eventName: ListAliases Static Value: unknown Static Value: alias
user_idchange_type eventNames: AttachVolume, CreateVolume, DeleteVolume, DetachVolume Static Value: EC2 Static Value: storage
eventNames: AuthorizeSecurityGroupEgress, AuthorizeSecurityGroupIngress, CreateNetworkAcl, CreateNetworkAclEntry, CreateNetworkInterface, CreateSecurityGroup, DeleteNetworkAcl, DeleteNetworkAclEntry, DeleteSecurityGroup, ReplaceNetworkAclAssociation, ReplaceNetworkAclEntry, RevokeSecurityGroupEgress, RevokeSecurityGroupIngress Static Value: EC2 Static Value: firewall
eventNames: CreateAccessKey, CreateLoginProfile, CreateUser, CreateVirtualMFADevice, DeleteUser, GetAccountSummary, GetUser, ListAccessKeys, ListAccountAliases, ListRoles, ListSigningCertificates, ListSigningCertificates, UpdateUser Static Value: IAM Static Value: AAA
eventNames: GetFederationToken, GetSessionToken Static Value: STS Static Value: AAA
eventNames: RunInstances, RebootInstances, StartInstances, StopInstances, TerminateInstances Static Value: EC2 Static Value: virtual_server
dest eventName: AttachVolume requestParameters.volumeId,
example: vol-3ox0otf8xaqxrptxi
requestParameters.instanceId,
example: i-3ox0otf8xaqxrptxi
eventNames: AuthorizeSecurityGroupEgress, AuthorizeSecurityGroupIngress, CreateSecurityGroup, RevokeSecurityGroupEgress, RevokeSecurityGroupIngress requestParameters.groupId,
example: sg-gnzeup7yzumo3f40i
eventSource,
example: ec2.amazonaws.com
eventName: ConsoleLogin eventSource,
example: ec2.amazonaws.com
additionalEventData.LoginTo OR eventSource,
example: https://console.aws.amazon.com/console/home
eventNames: CreateBucket, DeleteBucket, GetBucketEncryption, PutBucketAcl, PutBucketPublicAccessBlock, PutObject requestParameters.bucketName,
example: bucket1
requestParameters.Host OR requestParameters.host{},
example: s3-us-east-2.amazonaws.com
eventNames: CreateNetworkAcl, CreateNetworkAclEntry requestParameters.networkAclId OR responseElements.networkAcl.networkAclId,
example: acl-328f8f90a8e21dc7e
eventSource,
example: ec2.amazonaws.com
eventName: CreateUser responseElements.user.userId,
example: UB9BNXNERJHO8APB
eventSource,
example: iam.amazonaws.com
eventNames: CreateVolume, DeleteVolume responseElements.volumeId,
example: vol-pjk4yh53x5xy3kldx
eventSource,
example: ec2.amazonaws.com
eventNames: DeleteUser, UpdateUser requestParameters.userName,
example: test_user
eventSource,
example: iam.amazonaws.com
eventName: DetachVolume responseElements.volumeId,
example: vol-pjk4yh53x5xy3kldx
responseElements.instanceId,
example: i-3ox0otf8xaqxrptxi
eventNames: RunInstances, StartInstances responseElements.instancesSet.items{}.instanceId,
example: i-pjk4yh53x5xy3kldx
responseElements.instancesSet.items{}.instanceId OR eventSource,
example: i-pjk4yh53x5xy3kldx
action eventNames: CreateAccessKey, CreateLoginProfile, CreateNetworkAclEntry, CreateVirtualMFADevice, DeleteNetworkAclEntry Static Value: created Static Value: modified
eventNames: GetAccountSummary, GetUser, ListAccessKeys, ListAccountAliases, ListSigningCertificates Static Value: unknown Static Value: read
protocol eventName: CreateNetworkAclEntry Static Value: TCP Static Value: IP
object_attrs eventName: PutBucketAcl requestParameters.AccessControlPolicy.AccessControlList.Grant{}.Permission,
example: "READ

READ_ACP WRITE FULL_CONTROL"

Static value: AccessControlList
object eventName: RunInstances responseElements.instancesSet.items{}.instanceId,
example: i-pjk4yh53x5xy3kldx
responseElements.instancesSet.items{}.instanceId OR eventSource,
example: i-pjk4yh53x5xy3kldx
eventName: StartInstances requestParameters.instancesSet.items{}.instanceId,
example: i-pjk4yh53x5xy3kldx
requestParameters.instancesSet.items{}.instanceId OR eventSource,
example: ec2.amazonaws.com
eventName: UpdateUser requestParameters.userName,
example: test_user
requestParameters.newUserName,
example: test_new_user
object_id eventName: StartInstances requestParameters.instancesSet.items{}.instanceId, example: i-pjk4yh53x5xy3kldx requestParameters.instancesSet.items{}.instanceId OR eventSource, example: i-pjk4yh53x5xy3kldx
eventName: UpdateUser requestParameters.userName,
example: test_user
requestParameters.newUserName,
example: test_new_user
aws:config object_category resourceIDs: AWS::Redshift::ClusterSnapshot, AWS::Config::ResourceCompliance Static Value: unknown Statc Value: file
object_id resourceIDs: AWS::Redshift::ClusterSnapshot, AWS::EC2::NetworkInterface ARN,
example: arn:aws:redshift:eu-central-2:00000:snapshot:redshift-cluster-1/rs:redshift-cluster-1-2021-10-11-12-32-53
resourceId,
example: rs:redshift-cluster-1-2021-10-11-12-33-00
aws:config:notification object_category resourceTypes: AWS::Config::ResourceCompliance, AWS::Redshift::ClusterSnapshot Static Value: unknown Static Value: file
object_id resourceTypes: All N/A resourceId,
example: rs:redshift-cluster-1-2021-10-11-12-33-00
aws:description user_id source: All UserId,
example: ZWV5FIRT1Q4ZOFCQML63P
UserID,
example: account_Id, ZWV5FIRT1Q4ZOFCQML63P
status source: *ec2_instances status,
example: completed
image.attributes.state OR state OR status,
example: completed, available
aws:cloudwatchlogs:guardduty dest_type N/A Static value from lookup,
example: user
detail.resource.resourceType,
example: AccessKey
user N/A detail.resource.accessKeyDetails.principleId,
example: GeneratedFindingPrincipalId
detail.resource.accessKeyDetails.userName,
example: test_user
severity N/A Static Value: LOW, MEDIUM, HIGH Static Value: low, medium, high
aws:s3:accesslogs bytes N/A bytes,
example: 0
bytes_sent,
example: 470
response_time N/A turn_around_time,
example: 0
total_time,
example: 25

CIM model changes

See the following CIM model changes between 5.1.0 and 5.2.0:

Sourcetype metric_name Previous CIM model New CIM model
aws:cloudwatch FreeableMemory Database:Stats, All_Performance:Memory All_Performance:Memory
Sourcetype eventName Previous CIM model New CIM model
aws:cloudtrail AssumeRole, AssumeRoleWithSAML, AssumeRoleWithWebIdentity, GetFederationToken, GetSessionToken Authentication:Default_Authentication
aws:cloudtrail GetBucketEncryption, PutBucketAcl Change:Account_Management Change:All_Changes
aws:cloudtrail GetBucketEncryption, PutBucketAcl Change:Account_Management Change:All_Changes
aws:cloudtrail ListRoles, ListAliases Change:All_Changes
aws:cloudtrail RunInstances Change:Endpoint_Changes, Change:Instance_Changes Change:Instance_Changes
Sourcetype source Previous CIM model New CIM model
aws:description *:ec2_instances, *:ec2_images All_Inventory All_Inventory:Virtual_OS:Snapshot
aws:description *:ec2_instances All_Inventory All_Inventory:Virtual_OS:Snapshot
aws:inspector *:inspector:assessmentRun All_Inventory:Newtwok, All_Inventory:User, All_Inventory:Virtual_OS:Snapshot
Sourcetype Previous CIM model New CIM model
aws:cloudfront:accesslogs, aws:elb:accesslogs Web
aws:cloudwatchlogs:guardduty Alerts, Malware_Attacks Alerts
aws:config:rule All_Inventory:Network, All_Inventory:Virtual_OS:Snapshot Alerts
aws:s3 Web:Storage

Fixed issues

Version 5.2.0 of the Splunk Add-on for Amazon Web Services fixes the following, if any, issues.

Date resolved Issue number Description
2021-09-21 ADDON-41646 aws:metadata input is populating S3 buckets for AWS accounts where the bucket does not exist.
2021-09-13 ADDON-35220 In Splunk_TA_aws KeyError: 'LaunchConfigurationName' appearing when attempting to ingest cloudwatch data
2021-09-10 ADDON-41009 cloudwatch input timeout issue
2021-09-07 ADDON-39428 On upgrade to 5.1.0 - Cloudwatch Inputs need manual line added in conf - private_endpoint_enabled

Known issues

Version 5.2.0 of the Splunk Add-on for Amazon Web Services has the following, if any, known issues.

Date filed Issue number Description
2022-04-03 ADDON-49902 source=*:ec2_ebs_snapshots is importing >30K unwanted EC2 Snapshots
2022-03-31 ADDON-49879 PreConditioning Failure: AWS TA SQS-based S3 for versioned buckets

Workaround:
https://splunk.atlassian.net/browse/ADDON-44918
2022-02-02 ADDON-47661 AWS config and input page in a constant "loading" state when IMDSV2 enabled on EC2 instance.
2022-01-11 ADDON-46596 Add-on for AWS can't get logs from AWS
2021-11-18 ADDON-44918 AWS TA SQS-based S3 inputs do not handle versioned buckets properly
2021-10-27 ADDON-43991 AWS add Configuration Issue
2021-09-14 ADDON-42117 If Inputs Page page size is more than 25, then the alignment of input details is not consistent
2021-08-31 ADDON-41472 Splunk_TA_aws Account creation fails for China region because cn sts domain not used

Third-party software attributions

Version 5.2.0 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Version 5.1.0

Version 5.1.0 of the Splunk Add-on for Amazon Web Services was released on July 2, 2021.

Compatibility

Version 5.1.0 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms:

Splunk platform versions 8.0 and later
CIM 4.18 and later
Supported OS for data collection Platform independent
Vendor products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, Inspector, Kinesis, S3, VPC Flow Logs, Billing services, Metadata, SQS, and SNS.

Versions 5.0.0 and above of the Splunk Add-on for AWS are Python 3 releases, and only compatible with Splunk platform versions 8.0.0 and later. To use version 5.0.0 or later of this add-on, upgrade your Splunk platform deployment to version 8.0.0 or later. For users of Splunk platforms 6.x.x and Splunk 7.x.x, the Splunk Add-on for Amazon Web Services version 4.6.1 is supported. Do not upgrade to Splunk Add-on for AWS 5.0.0 or above on these versions of the Splunk platform.

The field alias functionality is compatible with the current version of this add-on. The current version of this add-on does not support older field alias configurations.

For more information about the field alias configuration change, refer to the Splunk Enterprise Release Notes.


New features

Version 5.1.0 of the Splunk Add-on for AWS version contains the following new and changed features:

  • A new data input called Metadata. The Metadata input , which can be accessed in Splunk Web by clicking Create New Input > Description > Metadata, uses the boto3 package to collect Description data. See the Metadata input topic in this manual for more information.
  • Migrated the following data inputs from the boto2 package to the boto3 package:
    • Cloudtrail
    • Config
    • Cloudwatch logs.
    • Generic S3
  • Support for Regional endpoints for all data inputs. Each API call can be made to a region-specific endpoint, instead than a public endpoint.
  • Support for private endpoints for the following data inputs:
    • Billing Cost and Usage Reports (CUR)
    • Cloudtrail
    • Cloudwatch
    • Cloudwatch Logs
    • Generic S3
    • Incremental S3
    • Kinesis
    • SQS-based S3
    Private endpoints can perform account authentication and data collection for each supported input. For example, a Splunk instance within a Virtual Private Cloud (VPC) infrastructure.
  • Support for disabling the DLQ (Dead Letter Queue) check for SQS-based S3 Crowdstrike event inputs.

The Description input will be deprecated in a future release. The Metadata input has been added as a replacement. The best practice is to begin moving your workloads to the Metadata input.

Fixed issues

Version 5.1.0 of the Splunk Add-on for Amazon Web Services fixes the following, if any, issues.

Date resolved Issue number Description
2021-07-30 ADDON-38682 Generic S3 - AttributeError: 'S3KeyReader' object has no attribute 'seekable'
2021-07-05 ADDON-37996 AWS add-on | To confirm if Osaka region on AWS is supported by AWS add-on
2021-06-10 ADDON-37528 modular input does not skip over old "GLACIER" folders and keep trying
2021-05-04 ADDON-34844 AWS sns Alert fails to be sent, only during first occurrence, it works from second trigger onwards
2021-03-15 ADDON-32067 AWS 4.6.1 will not load input/config page
2021-03-08 ADDON-33998 Splunk Add-on for Amazon Web Services 5.0.3 - issues with non default management port
2021-02-11 ADDON-30834 AWS-TA Kinesis Stream Inputs time is wrong
2021-02-11 ADDON-33377 Description Mod input not appending results correctly
2021-02-07 ADDON-29812 AWS security-group-rule description is missing in AWS TA
2021-01-12 ADDON-29815 Wrong start time to S3 input is mistakenly accepted by TA-AWS
2020-12-29 ADDON-22096 AWS Add-on is reporting NULL for NACL data

Known issues

Version 5.1.0 of the Splunk Add-on for Amazon Web Services has the following, if any, known issues.


Date filed Issue number Description
2021-11-18 ADDON-44918 AWS TA SQS-based S3 inputs do not handle versioned buckets properly
2021-09-14 ADDON-42117 If Inputs Page page size is more than 25, then the alignment of input details is not consistent
2021-09-07 ADDON-41646 aws:metadata input is populating S3 buckets for AWS accounts where the bucket does not exist.
2021-08-31 ADDON-41472 Splunk_TA_aws Account creation fails for China region because cn sts domain not used
2021-08-24 ADDON-41009 cloudwatch input timeout issue
2021-07-01 ADDON-38997 custom sourcetype/props is not getting honored and causing the line breaking issue
2021-06-13 ADDON-38108 v5.0.3 - The provided token has expired
2021-06-09 ADDON-37958 The impact of the format change of unstractured field in data events
2021-06-09 ADDON-37970 inputs.conf config generate from code for cloudwatch is not grouped together
2021-05-20 ADDON-37297 Splunk Add-on for AWS fails with TypeError: cannot unpack non-iterable NoneType object
2021-05-19 ADDON-37230 Not ingesting logs on Cloudwatch using AWS add-on:5.0.3
2021-04-22 ADDON-36123 When a role is assumed and a user performs any activity, Splunk extracts the role name as the "username"

Workaround:
We can easily fix this by using a regex based extraction for userName and user - field=userIdentity.arn ".*\:(?<user_action_type>.*)\/(?<user_role>.*)\/(?<user>.*)"
2021-03-23 ADDON-35020 v5.0.3 fields not extracting correctly

Third-party software attributions

Version 5.1.0 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Version 5.0.4

Version 5.0.4 of the Splunk Add-on for Amazon Web Services was released on June 2, 2021.

Compatibility

Version 5.0.4 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms:

Splunk platform versions 8.0 and later
CIM 4.18 and later
Supported OS for data collection Platform independent
Vendor products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, Inspector, Kinesis, S3, VPC Flow Logs, Billing services, SQS, and SNS.

Versions 5.0.0 and above of the Splunk Add-on for AWS are Python 3 releases, and only compatible with Splunk platform versions 8.0.0 and later. To use version 5.0.0 or later of this add-on, upgrade your Splunk platform deployment to version 8.0.0 or later. For users of Splunk platforms 6.x.x and Splunk 7.x.x, the Splunk Add-on for Amazon Web Services version 4.6.1 is supported. Do not upgrade to Splunk Add-on for AWS 5.0.0 or above on these versions of the Splunk platform.

The field alias functionality is compatible with the current version of this add-on. The current version of this add-on does not support older field alias configurations.

For more information about the field alias configuration change, refer to the Splunk Enterprise Release Notes.


New features

Version 5.0.4 of the Splunk Add-on for AWS version contains the following new and changed features:

  • Simple Queue Service (SQS) modular input support for Crowdstrike Falcon Data Replicator (FDR)
  • Bug fixes.

Fixed issues

Version 5.0.4 of the Splunk Add-on for Amazon Web Services fixes the following issues.

Date resolved Issue number Description
2021-06-28 ADDON-36953 AWS TA is not loading kinesis data post upgrade from 4.5.0 to 5.0.3
2021-05-18 ADDON-36305 Getting error in splunkd.log when user tries to fresh install the addon and inputs page is not loading for the TA

Known issues

Version 5.0.4 of the Splunk Add-on for Amazon Web Services has the following, if any, known issues.

The Splunk Add-on for AWS version 5.x.x is incompatible with Splunk Enterprise versions 7.x.x and earlier.


Date filed Issue number Description
2021-09-14 ADDON-42117 If Inputs Page page size is more than 25, then the alignment of input details is not consistent
2021-06-25 ADDON-38682 Generic S3 - AttributeError: 'S3KeyReader' object has no attribute 'seekable'

Third-party software attributions

Version 5.0.4 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Version 5.0.3

Version 5.0.3 of the Splunk Add-on for Amazon Web Services was released on October 8, 2020.

Compatibility

Version 5.0.3 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms:

Splunk platform versions 8.0 and later
CIM 4.3 and later
Supported OS for data collection Platform independent
Vendor products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, Inspector, Kinesis, S3, VPC Flow Logs, Billing services, SQS, and SNS.

Versions 5.0.0 and above of the Splunk Add-on for AWS are Python 3 releases, and only compatible with Splunk platform versions 8.0.0 and later. To use version 5.0.0 or later of this add-on, upgrade your Splunk platform deployment to version 8.0.0 or later. For users of Splunk platforms 6.x.x and Splunk 7.x.x, the Splunk Add-on for Amazon Web Services version 4.6.1 is supported. Do not upgrade to Splunk Add-on for AWS 5.0.0 or above on these versions of the Splunk platform.

The field alias functionality is compatible with the current version of this add-on. The current version of this add-on does not support older field alias configurations.

For more information about the field alias configuration change, refer to the Splunk Enterprise Release Notes.


New features

Version 5.0.3 of the Splunk Add-on for AWS version contains the following new and changed features:


  • Bug fix with proxy behavior not working as expected.
  • Bug fix with no_proxy taking effect with https.
  • SQS modular input for proxy configuration code fix (Microsoft Windows only)

Fixed issues

Version 5.0.3 of the Splunk Add-on for Amazon Web Services fixes the following issues.

Known issues

Version 5.0.3 of the Splunk Add-on for Amazon Web Services has the following known issues.

The Splunk Add-on for AWS version 5.x.x is incompatible with Splunk Enterprise versions 7.x.x and earlier.


Date filed Issue number Description
2021-09-14 ADDON-42117 If Inputs Page page size is more than 25, then the alignment of input details is not consistent
2021-09-08 ADDON-41767 Add T3 burstable instances to the Metrics collection on addon

Workaround:
line 54 of Addon -> Bin -> splunk_ta_aws -> modinputs -> cloudwatch -> discovery -> ec2.py 

@classmethod def _create_metric_names(cls, *types):

   result = set()
   for typename in types:
       parts = [cls._METRIC_NAMES]
       if typename.startswith("t2"):
           parts.append(cls._T2_METRIC_NAMES)
       elif typename.startswith("c5") or typename.startswith("m5"):
           parts.append(cls._C5_M5_METRIC_NAMES)
       for name in itertools.chain(*parts):
           result.add(name)
   return result

By asking the customer to add additional OR statement to the code:

if typename.startswith("t2") or typename.startswith("t3"):

2021-08-24 ADDON-41009 cloudwatch input timeout issue
2021-07-01 ADDON-38997 custom sourcetype/props is not getting honored and causing the line breaking issue
2021-06-13 ADDON-38108 v5.0.3 - The provided token has expired
2021-06-11 ADDON-37996 AWS add-on | To confirm if Osaka region on AWS is supported by AWS add-on
2021-06-09 ADDON-37958 The impact of the format change of unstractured field in data events
2021-06-01 ADDON-37528 modular input does not skip over old "GLACIER" folders and keep trying
2021-05-20 ADDON-37297 Splunk Add-on for AWS fails with TypeError: cannot unpack non-iterable NoneType object
2021-05-12 ADDON-36953 AWS TA is not loading kinesis data post upgrade from 4.5.0 to 5.0.3
2021-04-29 ADDON-36305 Getting error in splunkd.log when user tries to fresh install the addon and inputs page is not loading for the TA
2021-04-22 ADDON-36123 When a role is assumed and a user performs any activity, Splunk extracts the role name as the "username"

Workaround:
We can easily fix this by using a regex based extraction for userName and user - field=userIdentity.arn ".*\:(?<user_action_type>.*)\/(?<user_role>.*)\/(?<user>.*)"
2021-03-26 ADDON-35220 In Splunk_TA_aws KeyError: 'LaunchConfigurationName' appearing when attempting to ingest cloudwatch data
2021-03-23 ADDON-35020 v5.0.3 fields not extracting correctly
2021-02-19 ADDON-33998 Splunk Add-on for Amazon Web Services 5.0.3 - issues with non default management port
2021-01-28 ADDON-33377 Description Mod input not appending results correctly
2020-12-22 ADDON-32067 AWS 4.6.1 will not load input/config page
2019-11-20 ADDON-24471 Billing input causes double-ingest of CUR billing files when splunk restarts during ingest

Workaround:
Each set of duplicate events for a given CUR assembly will have a unique txid (which is a timestamp) set by the Billing input.

Filter out events that don't have the largest value for txid in a given assembly.

Example:

| rex field=source "/(?<date_range>\d+-\d+)/(?<assemblyId>[^/]+)/" 
| eventstats max(txid) AS max_txid BY assemblyId
| where txid == max_txid

Third-party software attributions

Version 5.0.3 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.


Version 5.0.2

Version 5.0.2 of the Splunk Add-on for Amazon Web Services was released on August 22, 2020.

Compatibility

Version 5.0.2 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms:

Splunk platform versions 8.0 and later
CIM 4.3 and later
Supported OS for data collection Platform independent
Vendor products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, Inspector, Kinesis, S3, VPC Flow Logs, Billing services, SQS, and SNS.

Versions 5.0.0 and above of the Splunk Add-on for AWS are Python 3 releases, and only compatible with Splunk platform versions 8.0.0 and later. To use version 5.0.0 or later of this add-on, upgrade your Splunk platform deployment to version 8.0.0 or later. For users of Splunk platforms 6.x.x and Splunk 7.x.x, the Splunk Add-on for Amazon Web Services version 4.6.1 is supported. Do not upgrade to Splunk Add-on for AWS 5.0.0 or above on these versions of the Splunk platform.

New features

Version 5.0.2 of the Splunk Add-on for AWS version contains the following new and changed features:

  • Increased Network Traffic CIM data model compatibility.
  • Increased Change CIM data model compatibility.
  • Improved support for the Splunk Enterprise Security Assets and Identities Framework Interface

Fixed issues

Version 5.0.2 of the Splunk Add-on for Amazon Web Services fixes the following issues.


Date resolved Issue number Description
2020-08-24 ADDON-26632 Update cloudfront_web and cloudfront_rtmp regex to account for ipv6 addresses
2020-08-24 ADDON-26878 Installing AWS TA on Enterprise Security SH breaks Suppression Auditing: stanzas For aws:resthandler:log and aws:util:log are too generic
2020-07-13 ADDON-22785 AWS calls increase when using aws:description
2020-07-13 ADDON-26599 Support for newer formatted cloudwatch ELB metrics, exception handling for logs which don't have all log field populated

Known issues

Version 5.0.2 of the Splunk Add-on for Amazon Web Services has the following known issues.

The Splunk Add-on for AWS version 5.x.x is incompatible with Splunk Enterprise versions 7.x.x and earlier.


Date filed Issue number Description
2021-09-14 ADDON-42117 If Inputs Page page size is more than 25, then the alignment of input details is not consistent
2021-06-01 ADDON-37528 modular input does not skip over old "GLACIER" folders and keep trying
2020-10-03 ADDON-29815 Wrong start time to S3 input is mistakenly accepted by TA-AWS
2019-11-20 ADDON-24471 Billing input causes double-ingest of CUR billing files when splunk restarts during ingest

Workaround:
Each set of duplicate events for a given CUR assembly will have a unique txid (which is a timestamp) set by the Billing input.

Filter out events that don't have the largest value for txid in a given assembly.

Example:

| rex field=source "/(?<date_range>\d+-\d+)/(?<assemblyId>[^/]+)/" 
| eventstats max(txid) AS max_txid BY assemblyId
| where txid == max_txid

Third-party software attributions

Version 5.0.2 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Version 5.0.1

Version 5.0.1 of the Splunk Add-on for Amazon Web Services was released on May 13, 2020.

Compatibility

Version 5.0.1 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms:

Splunk platform versions 8.0 and later
CIM 4.3 and later
Supported OS for data collection Platform independent
Vendor products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, Inspector, Kinesis, S3, VPC Flow Logs, Billing services, SQS, and SNS.

Versions 5.0.0 and above of the Splunk Add-on for AWS are Python 3 releases, and only compatible with Splunk platform versions 8.0.0 and later. To use version 5.0.0 or later of this add-on, upgrade your Splunk platform deployment to version 8.0.0 or later. For users of Splunk platforms 6.x.x and Splunk 7.x.x, the Splunk Add-on for Amazon Web Services version 4.6.1 is supported. Do not upgrade to Splunk Add-on for AWS 5.0.0 or above on these versions of the Splunk platform.

New features

Version 5.0.1 of the Splunk Add-on for AWS version contains the following new and changed features:

  • FIPS compliance release for Python 3
  • Improved Support for the Authentication CIM Model.

Fixed issues

Version 5.0.1 of the Splunk Add-on for Amazon Web Services fixes the following issues.


Date resolved Issue number Description
2020-06-16 ADDON-25762 Generic AWS S3 inputs duplicating events after Splunk forwarder restart
2020-04-29 ADDON-24651 Improved ALB Access Logs parsing
2020-04-29 ADDON-21349, CMON-2382 Fix for S3 field extraction
2020-04-23 ADDON-21900 Input validation needed for AWS inputs to check for / (forward slash)
2020-04-23 ADDON-25454, ADDON-26096 Splunk Add-on for AWS repeatedly processing the same gzip file
2020-04-23 ADDON-25279 FIPS compliance release for Python 3
2020-04-23 ADDON-23358 Improvement to timestamp extraction for sourcetype aws:cloudwatchlogs:vpcflow
2020-04-23 ADDON-24325 AWS TA only ingesting up to 100 RDS instances.
2020-03-23 ADDON-13856, ADDON-13200 Add input name as part of Kinesis checkpoint file name
2020-03-11 ADDON-25546, ADDON-25289 Region support improved for AWS Description: adding ap-east-1, eu-north-1, eu-west-3 and me-south-1

Known issues

Version 5.0.1 of the Splunk Add-on for Amazon Web Services has the following known issues.

The Splunk Add-on for AWS version 5.x.x is incompatible with Splunk Enterprise versions 7.x.x and earlier.


Date filed Issue number Description
2020-05-26 ADDON-26878 Installing AWS TA on Enterprise Security SH breaks Suppression Auditing: stanzas For aws:resthandler:log and aws:util:log are too generic

Workaround:
Edit default/props.conf

and change the lines [source::...(/|\\)var(/|\\)log(/|\\)splunk(/|\\)*rest*.log*] [source::...(/|\\)var(/|\\)log(/|\\)splunk(/|\\)*util.log*]

to [source::...(/|\\)var(/|\\)log(/|\\)splunk(/|\\)*Splunk_TA_aws*rest*.log*] [source::...(/|\\)var(/|\\)log(/|\\)splunk(/|\\)*Splunk_TA_aws*util.log*]

2020-05-14 ADDON-26632 Update cloudfront_web and cloudfront_rtmp regex to account for ipv6 addresses

Workaround:
Update local/props.conf with the following changes

{code:java} [aws:cloudfront:accesslogs] EXTRACT-cloudfront_web = ^\s*(?P<date>[0-9-]+)\s+(?P

EXTRACT-cloudfront_rtmp = ^\s*(?P<date>[0-9-]+)\s+(?P


2020-05-13 ADDON-26599 Support for newer formatted cloudwatch ELB metrics, exception handling for logs which don't have all log field populated
2019-11-20 ADDON-24471 Billing input causes double-ingest of CUR billing files when splunk restarts during ingest

Workaround:
Each set of duplicate events for a given CUR assembly will have a unique txid (which is a timestamp) set by the Billing input.

Filter out events that don't have the largest value for txid in a given assembly.

Example:

| rex field=source "/(?<date_range>\d+-\d+)/(?<assemblyId>[^/]+)/" 
| eventstats max(txid) AS max_txid BY assemblyId
| where txid == max_txid
2019-08-02 ADDON-22785 AWS calls increase when using aws:description

Third-party software attributions

Version 5.0.1 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Version 5.0.0

Version 5.0.0 of the Splunk Add-on for Amazon Web Services was released on December 19, 2019.

Compatibility

Version 5.0.0 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms:

Splunk platform versions 8.0 and later
CIM 4.3 and later
Supported OS for data collection Platform independent
Vendor products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, Inspector, Kinesis, S3, VPC Flow Logs, Billing services, SQS, and SNS.

Version 5.0.0 of the Splunk Add-on for AWS is a Python 3 release and is only compatible with Splunk platform versions 8.0.0 and later. To use version 5.0.0 or later of this add-on, upgrade your Splunk platform deployment to version 8.0.0 or later. For users of Splunk platforms 6.x.x and Splunk 7.x.x, the Splunk Add-on for Amazon Web Services version 4.6.1 is supported. Do not upgrade to Splunk Add-on for AWS 5.0.0 on these versions of the Splunk platform.

New features

Version 5.0.0 of the Splunk Add-on for AWS version contains the following new and changed features:

  • Support for Python3
  • Python2 is no longer supported, starting in version 5.0.0 of the Splunk Add-on for AWS.

Fixed issues

Version 5.0.0 of the Splunk Add-on for Amazon Web Services fixes the following issues.


Date resolved Issue number Description
2020-09-02 ADDON-29101, ADDON-21459 Make the naming convention of CloudWatch metric events compatible with SAI

Known issues

Version 5.0.0 of the Splunk Add-on for Amazon Web Services has the following known issues.

  • The Splunk Add-on for AWS version 5.x.x is incompatible with Splunk Enterprise versions 7.x.x and earlier.


Date filed Issue number Description
2020-10-02 ADDON-29812 AWS security-group-rule description is missing in AWS TA
2020-05-14 ADDON-26632 Update cloudfront_web and cloudfront_rtmp regex to account for ipv6 addresses

Workaround:
Update local/props.conf with the following changes

{code:java} [aws:cloudfront:accesslogs] EXTRACT-cloudfront_web = ^\s*(?P<date>[0-9-]+)\s+(?P

EXTRACT-cloudfront_rtmp = ^\s*(?P<date>[0-9-]+)\s+(?P


2020-05-13 ADDON-26599 Support for newer formatted cloudwatch ELB metrics, exception handling for logs which don't have all log field populated
2020-03-23 ADDON-25762 Generic AWS S3 inputs duplicating events after Splunk forwarder restart

Workaround:
Lookup following code block in file bin/splunk_ta_aws/modinputs/generic_s3/s3_key_reader.py.

should be line 109 - 112

if size == 0:

   size = self.bufsize

data = self._config[asc.key_object].read(size) Insert two lines like this:


if size == 0:

   size = self.bufsize

if self._reached_eof:

   return b

data = self._config[asc.key_object].read(size)


2020-03-09 ADDON-25546, ADDON-25289 Region support improved for AWS Description: adding ap-east-1, eu-north-1, eu-west-3 and me-south-1
2020-03-04 ADDON-25454, ADDON-26096 Splunk Add-on for AWS repeatedly processing the same gzip file
2020-02-12 ADDON-25279 FIPS compliance release for Python 3
2019-12-12 ADDON-24651 Improved ALB Access Logs parsing
2019-11-20 ADDON-24471 Billing input causes double-ingest of CUR billing files when splunk restarts during ingest

Workaround:
Each set of duplicate events for a given CUR assembly will have a unique txid (which is a timestamp) set by the Billing input.

Filter out events that don't have the largest value for txid in a given assembly.

Example:

| rex field=source "/(?<date_range>\d+-\d+)/(?<assemblyId>[^/]+)/" 
| eventstats max(txid) AS max_txid BY assemblyId
| where txid == max_txid
2019-11-14 ADDON-24325 AWS TA only ingesting up to 100 RDS instances.
2019-09-22 ADDON-23358 Improvement to timestamp extraction for sourcetype aws:cloudwatchlogs:vpcflow

Workaround:
Manually update sourcetype aws:cloudwatchlogs:vpcflow with TIME_FORMAT and TIME_PREFIX settings.

For example:

TIME_FORMAT = %s
TIME_PREFIX = ^(?>\S+\s){10}
MAX_TIMESTAMP_LOOKAHEAD = 10
2019-08-02 ADDON-22785 AWS calls increase when using aws:description
2019-04-29 ADDON-21900 Input validation needed for AWS inputs to check for / (forward slash)
2019-02-15 ADDON-21349, CMON-2382 Fix for S3 field extraction
2017-02-24 ADDON-13856, ADDON-13200 Add input name as part of Kinesis checkpoint file name

Third-party software attributions

Version 5.0.0 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Version 4.6.1

Version 4.6.1 of the Splunk Add-on for Amazon Web Services was released on December 10, 2019.

Compatibility

Version 4.6.1 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms:

Splunk platform versions 6.5 and later
CIM 4.3 and later
Supported OS for data collection Platform independent
Vendor products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, Inspector, Kinesis, S3, VPC Flow Logs, Billing services, SQS, and SNS.

New features

Version 4.6.1 of the Splunk Add-on for AWS version contains the following new and changed features:

  • FIPS compliance
  • Updated third party components

Fixed issues

Version 4.6.1 of the Splunk Add-on for Amazon Web Services fixes the following issues. If no issues appear below, no issues have yet been fixed.


Known issues

Version 4.6.1 of the Splunk Add-on for Amazon Web Services has the following known issues. If no issues appear below, no issues have yet been reported.


Date filed Issue number Description
2021-09-08 ADDON-41767 Add T3 burstable instances to the Metrics collection on addon

Workaround:
line 54 of Addon -> Bin -> splunk_ta_aws -> modinputs -> cloudwatch -> discovery -> ec2.py 

@classmethod def _create_metric_names(cls, *types):

   result = set()
   for typename in types:
       parts = [cls._METRIC_NAMES]
       if typename.startswith("t2"):
           parts.append(cls._T2_METRIC_NAMES)
       elif typename.startswith("c5") or typename.startswith("m5"):
           parts.append(cls._C5_M5_METRIC_NAMES)
       for name in itertools.chain(*parts):
           result.add(name)
   return result

By asking the customer to add additional OR statement to the code:

if typename.startswith("t2") or typename.startswith("t3"):

2021-01-12 ADDON-32838 When using generic S3 to get S3 bucket, TA should start reading file from initial_scan_datetime
2020-12-22 ADDON-32067 AWS 4.6.1 will not load input/config page
2020-05-26 ADDON-26878 Installing AWS TA on Enterprise Security SH breaks Suppression Auditing: stanzas For aws:resthandler:log and aws:util:log are too generic

Workaround:
Edit default/props.conf

and change the lines [source::...(/|\\)var(/|\\)log(/|\\)splunk(/|\\)*rest*.log*] [source::...(/|\\)var(/|\\)log(/|\\)splunk(/|\\)*util.log*]

to [source::...(/|\\)var(/|\\)log(/|\\)splunk(/|\\)*Splunk_TA_aws*rest*.log*] [source::...(/|\\)var(/|\\)log(/|\\)splunk(/|\\)*Splunk_TA_aws*util.log*]

2020-03-09 ADDON-25546, ADDON-25289 Region support improved for AWS Description: adding ap-east-1, eu-north-1, eu-west-3 and me-south-1
2019-12-12 ADDON-24651 Improved ALB Access Logs parsing
2019-02-15 ADDON-21349, CMON-2382 Fix for S3 field extraction

Third-party software attributions

Version 4.6.1 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Version 4.6.0

Version 4.6.0 of the Splunk Add-on for Amazon Web Services was released on October 3, 2018.

Compatibility

Version 4.6.0 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms:

Splunk platform versions 6.5 and later
CIM 4.3 and later
Supported OS for data collection Platform independent
Vendor products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, Inspector, Kinesis, S3, VPC Flow Logs, Billing services, SQS, and SNS.

New features

Version 4.6.0 of the Splunk Add-on for AWS version contains the following new and changed features:

  • CloudWatch Metrics input to enable discovery of new entities without Splunk restart
  • Metrics store support (requires a Splunk forwarder version 7.2.0 or above.)
  • Ability to detect configuration of SSL on management port
  • Line/event breaking enforcement for ELB/S3 Access Logs
  • Support for Splunk Enterprise 7.2.0

Fixed issues

Version 4.6.0 of the Splunk Add-on for Amazon Web Services fixes the following issues.


Date resolved Issue number Description
2018-08-27 ADDON-18031 Small page size causing LimitExceededException error during Kinesis ListStreams operations
2018-07-17 ADDON-18087, SII-1746 Invalid AWS credentials can be added and interacted with as valid AWS credentials
2018-06-27 ADDON-17277 Line/event breaking enforcement for ELB/S3 Access Logs

Known issues

Version 4.6.0 of the Splunk Add-on for Amazon Web Services has the following known issues.


Date filed Issue number Description
2021-06-01 ADDON-37528 modular input does not skip over old "GLACIER" folders and keep trying
2020-11-08 ADDON-30834 AWS-TA Kinesis Stream Inputs time is wrong
2020-09-02 ADDON-29101, ADDON-21459 Make the naming convention of CloudWatch metric events compatible with SAI
2020-05-26 ADDON-26878 Installing AWS TA on Enterprise Security SH breaks Suppression Auditing: stanzas For aws:resthandler:log and aws:util:log are too generic

Workaround:
Edit default/props.conf

and change the lines [source::...(/|\\)var(/|\\)log(/|\\)splunk(/|\\)*rest*.log*] [source::...(/|\\)var(/|\\)log(/|\\)splunk(/|\\)*util.log*]

to [source::...(/|\\)var(/|\\)log(/|\\)splunk(/|\\)*Splunk_TA_aws*rest*.log*] [source::...(/|\\)var(/|\\)log(/|\\)splunk(/|\\)*Splunk_TA_aws*util.log*]

2019-11-20 ADDON-24471 Billing input causes double-ingest of CUR billing files when splunk restarts during ingest

Workaround:
Each set of duplicate events for a given CUR assembly will have a unique txid (which is a timestamp) set by the Billing input.

Filter out events that don't have the largest value for txid in a given assembly.

Example:

| rex field=source "/(?<date_range>\d+-\d+)/(?<assemblyId>[^/]+)/" 
| eventstats max(txid) AS max_txid BY assemblyId
| where txid == max_txid
2019-11-14 ADDON-24325 AWS TA only ingesting up to 100 RDS instances.
2019-09-22 ADDON-23358 Improvement to timestamp extraction for sourcetype aws:cloudwatchlogs:vpcflow

Workaround:
Manually update sourcetype aws:cloudwatchlogs:vpcflow with TIME_FORMAT and TIME_PREFIX settings.

For example:

TIME_FORMAT = %s
TIME_PREFIX = ^(?>\S+\s){10}
MAX_TIMESTAMP_LOOKAHEAD = 10
2019-08-02 ADDON-22785 AWS calls increase when using aws:description
2019-04-29 ADDON-21900 Input validation needed for AWS inputs to check for / (forward slash)
2019-02-15 ADDON-21349, CMON-2382 Fix for S3 field extraction
2018-08-16 ADDON-19138 Splunk 7.1 and below outputs 'Invalid key in stanza' warning on startup about INGEST_EVAL, METRIC-SCHEMA-MEASURES, and METRIC-SCHEMA-TRANSFORMS
2018-03-28 ADDON-17571 AWS TA and *nix TA lack spec files for eventgen.conf, which causes cluster bundle validation errors, and breaks Manage Indexes page in clustered Splunk Cloud

Workaround:
Splunk Cloud customers who cannot create indexes on their own due to this bug should file a support case when they need new indexes created.
2018-02-19 ADDON-17158 The style of multi-input text box is not correct
2018-02-19 ADDON-17157 The header view of customized page is inconsistent with the default NightLight style
2018-02-13 ADDON-17132 Create/edit input page layout is broken
2018-02-13 ADDON-17135 Placeholder tooltip is missing for dropdown
2018-01-05 ADDON-16518 When kinesis and cloudwatch inputs send large volumes of data over HEC, HEC can block the ingest pipeline, which breaks non-HEC inputs.

Workaround:
Set use_hec=false in [global_settings] stanza of aws_kinesis.conf and/or aws_cloudwatch.conf
2017-09-03 ADDON-15718 Duplicate cloudfront data in description when there are more than 1 regions
2017-08-22 ADDON-15603 Users can delete an account in use.
2017-03-29 ADDON-14287 After you replace an IAM role attached to an EC2 instance, the inputs that use the old IAM role stop collecting data.
2016-12-22 ADDON-12867, ADDON-11894 S3 input: large key numbers lead to excessively large checkpoint files

Workaround:
To migrate to SQS based S3 or Incremental S3. Large number of files always leads to large size of checkpoint by the nature of Generic S3.

This will improve the checkpoint file size, however, as long as the Jira is not fixed, the checkpoint file size might still be not as little as expected.

Third-party software attributions

Version 4.6.0 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.


Version 4.5.0

Version 4.5.0 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms.

Splunk platform versions 6.5 and later
CIM 4.3 and later
Supported OS for data collection Platform independent
Vendor products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, Inspector, Kinesis, S3, VPC Flow Logs, Billing services, SQS, and SNS.

New features

Version 4.5.0 of the Splunk Add-on for AWS version contains the following new and changed features:

  • Support for the configuration of billing inputs to collect Cost and Usage Report data (sourcetype: aws:billing:cur).

Fixed issues

Version 4.5.0 of the Splunk Add-on for Amazon Web Services fixes the following issues.


Date resolved Issue number Description
2018-01-22 ADDON-15918 AWS TA is unable to validate role ARNs with "/" in path
2018-01-22 ADDON-16435 AWS - Getting error trying to connect to CloudTrail using SQS Based S3 - EU-WEST-1

Known issues

Version 4.5.0 of the Splunk Add-on for Amazon Web Services has the following known issues.


Date filed Issue number Description
2020-05-26 ADDON-26878 Installing AWS TA on Enterprise Security SH breaks Suppression Auditing: stanzas For aws:resthandler:log and aws:util:log are too generic

Workaround:
Edit default/props.conf

and change the lines [source::...(/|\\)var(/|\\)log(/|\\)splunk(/|\\)*rest*.log*] [source::...(/|\\)var(/|\\)log(/|\\)splunk(/|\\)*util.log*]

to [source::...(/|\\)var(/|\\)log(/|\\)splunk(/|\\)*Splunk_TA_aws*rest*.log*] [source::...(/|\\)var(/|\\)log(/|\\)splunk(/|\\)*Splunk_TA_aws*util.log*]

2019-06-03 ADDON-22096 AWS Add-on is reporting NULL for NACL data
2019-02-15 ADDON-21349, CMON-2382 Fix for S3 field extraction
2018-08-22 ADDON-19171 Cannot add regions when configuring Inspector inputs for the TA for AWS
2018-05-17 ADDON-18087, SII-1746 Invalid AWS credentials can be added and interacted with as valid AWS credentials
2018-05-09 ADDON-18031 Small page size causing LimitExceededException error during Kinesis ListStreams operations
2018-05-02 ADDON-17910 Rest endpoint /splunk_ta_aws/settings/account should not be exposed to Splunk Web
2018-03-28 ADDON-17571 AWS TA and *nix TA lack spec files for eventgen.conf, which causes cluster bundle validation errors, and breaks Manage Indexes page in clustered Splunk Cloud

Workaround:
Splunk Cloud customers who cannot create indexes on their own due to this bug should file a support case when they need new indexes created.
2018-02-27 ADDON-17277 Line/event breaking enforcement for ELB/S3 Access Logs
2018-02-19 ADDON-17158 The style of multi-input text box is not correct
2018-02-19 ADDON-17157 The header view of customized page is inconsistent with the default NightLight style
2018-02-13 ADDON-17135 Placeholder tooltip is missing for dropdown
2018-02-13 ADDON-17132 Create/edit input page layout is broken
2018-01-05 ADDON-16518 When kinesis and cloudwatch inputs send large volumes of data over HEC, HEC can block the ingest pipeline, which breaks non-HEC inputs.

Workaround:
Set use_hec=false in [global_settings] stanza of aws_kinesis.conf and/or aws_cloudwatch.conf
2017-09-03 ADDON-15718 Duplicate cloudfront data in description when there are more than 1 regions
2017-09-01 ADDON-15712 It stops pulling Kinesis stream data when the Kinesis stream is resharded
2017-08-22 ADDON-15603 Users can delete an account in use.
2016-12-22 ADDON-12867, ADDON-11894 S3 input: large key numbers lead to excessively large checkpoint files

Workaround:
To migrate to SQS based S3 or Incremental S3. Large number of files always leads to large size of checkpoint by the nature of Generic S3.

This will improve the checkpoint file size, however, as long as the Jira is not fixed, the checkpoint file size might still be not as little as expected.

Third-party software attributions

Version 4.5.0 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Version 4.4.0

Version 4.4.0 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms.

Splunk platform versions 6.5 and later
CIM 4.3 and later
Platforms Platform independent
Vendor Products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, Inspector, Kinesis, S3, VPC Flow Logs, Billing services, SQS, and SNS.

New features

Version 4.4.0 of the Splunk Add-on for AWS version contains the following new and changed features:

  • Splunk Add-on for AWS 4.4.0 is only compatible with Splunk App for AWS 5.1.0. Previous versions of Splunk App for AWS are not supported.
  • Optimized Web UI for better usability and more streamlined configuration workflow
    • The Create New Input menu has been redesigned with all the menu options organized by the type of data to collect.
    • Two separate configuration pages are now available for Generic S3 and Incremental S3 input types respectively. Previously, the two different input types were configured in one configuration page.
    • Input configuration fields are now grouped into AWS Input Configuration, Splunk-related Configuration, and Advanced Settings sections on the Web UI.
    • Redesigned input configuration UIs for CloudWatch and Config input types let you create multiple inputs all at once.
  • Added a new Temp Folder setting to the Billing input type configuration, which lets you specify a non-default folder for temporarily storing downloaded detailed billing report .zip files when the system default temp folder does not provide sufficient space.
  • You can now configure SQS-based S3 inputs to index non-AWS custom logs in plain text in addition to its supported AWS log types.
  • SQS-based S3 input type now supports CloudTrail and Config SQS notifications.
  • Assume Role is now supported in SQS, Config Rule, and Inspector input types.
  • The Description input type now supports the iam_users service.

Upgrade

To upgrade from versions 4.3 and below, AWS users must be given permission to use the ec2:RunInstances API action, and depending on deployment, the following API actions:

API Action Description
ec2:DescribeImages Allows users to view and select an AMI.
ec2:DescribeVpcs Allows users to view the available EC2-Classic and virtual private clouds (VPCs) network options. This API action is required even if you are not launching into a VPC.
ec2:DescribeSubnets Allows users to view all available subnets for the chosen VPC, when launching into a VPC.
ec2:DescribeSecurityGroups Allows users to view the security groups page in the wizard. Users can select an existing security group.
ec2:DescribeKeyPairs or ec2:CreateKeyPair Allows users to select an existing key pair, or create a new key pair.

See the Configure Description permissions topic in this manual for more information on how to configure AWS permissions.

See the AWS documentation for more information on the DescribeImages function. https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeImages.html.

Fixed issues

Version 4.4.0 of the Splunk Add-on for Amazon Web Services fixes the following issues.


Date resolved Issue number Description
2017-08-03 ADDON-14890 Add-on truncates Kinesis stream dropdown to 20 items.
2017-07-27 ADDON-12700 Pagination issue in Account page.
2017-07-11 ADDON-11974 Cannot get CloudWatch data using some default configuration in Add-on
2017-05-25 ADDON-13282 Cannot change Description interval in UI more than once

Known issues

Version 4.4.0 of the Splunk Add-on for Amazon Web Services has the following known issues.


Date filed Issue number Description
2021-08-05 ADDON-40189 addon 2648 shows inputs page from app 3670 CISCO AMP for Endpoints Events Input
2021-08-05 ADDON-40188 addon 3185 shows inputs page from app 3670 CISCO AMP for Endpoints Events Input
2018-08-22 ADDON-19171 Cannot add regions when configuring Inspector inputs for the TA for AWS
2018-05-17 ADDON-18087, SII-1746 Invalid AWS credentials can be added and interacted with as valid AWS credentials
2018-05-02 ADDON-17910 Rest endpoint /splunk_ta_aws/settings/account should not be exposed to Splunk Web
2018-03-28 ADDON-17571 AWS TA and *nix TA lack spec files for eventgen.conf, which causes cluster bundle validation errors, and breaks Manage Indexes page in clustered Splunk Cloud

Workaround:
Splunk Cloud customers who cannot create indexes on their own due to this bug should file a support case when they need new indexes created.
2018-02-27 ADDON-17277 Line/event breaking enforcement for ELB/S3 Access Logs
2018-01-05 ADDON-16518 When kinesis and cloudwatch inputs send large volumes of data over HEC, HEC can block the ingest pipeline, which breaks non-HEC inputs.

Workaround:
Set use_hec=false in [global_settings] stanza of aws_kinesis.conf and/or aws_cloudwatch.conf
2017-12-20 ADDON-16435 AWS - Getting error trying to connect to CloudTrail using SQS Based S3 - EU-WEST-1
2017-09-21 ADDON-15918 AWS TA is unable to validate role ARNs with "/" in path
2017-09-03 ADDON-15718 Duplicate cloudfront data in description when there are more than 1 regions
2017-09-01 ADDON-15712 It stops pulling Kinesis stream data when the Kinesis stream is resharded
2017-08-22 ADDON-15603 Users can delete an account in use.
2017-08-19 ADDON-15578 On Windows, fails to rotate CloudWatch and Incremental S3 logs when indexing speed cannot catch up with data collection
2017-07-25 ADDON-15371 Add-on should support non-UTF fields in access logs.
2017-03-29 ADDON-14287 After you replace an IAM role attached to an EC2 instance, the inputs that use the old IAM role stop collecting data.
2017-02-24 ADDON-13856, ADDON-13200 Add input name as part of Kinesis checkpoint file name
2016-12-22 ADDON-12867, ADDON-11894 S3 input: large key numbers lead to excessively large checkpoint files

Workaround:
To migrate to SQS based S3 or Incremental S3. Large number of files always leads to large size of checkpoint by the nature of Generic S3.

This will improve the checkpoint file size, however, as long as the Jira is not fixed, the checkpoint file size might still be not as little as expected.

Third-party software attributions

Version 4.4.0 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Version 4.3.0

Version 4.3.0 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms.

Splunk platform versions 6.4 and later
CIM 4.3 and later
Platforms Platform independent
Vendor Products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, Inspector, Kinesis, S3, VPC Flow Logs, Billing services, SQS, and SNS.

New features

Version 4.3.0 of the Splunk Add-on for AWS contains the following new and changed features:

  • SQS-based S3 input type
    A multi-purpose input type that collects several types of logs in response to messages polled from SQS queues. A scalable and higher-performing alternative to the generic S3 and incremental S3 input types. See Multi-purpose input types.
  • Heath Check dashboards
    Health Overview and S3 Health dashboards to help you troubleshoot data collection errors and performance issues. See Health Check dashboards.
  • Optimized logging. See Internal logs.

Fixed issues

Version 4.3.0 of the Splunk Add-on for Amazon Web Services fixes the following issues.


Date resolved Issue number Description
2017-06-09 ADDON-13860 Configuring more AWS accounts increases CPU usage and lowers throughput performance due to increased API calls
2017-06-07 ADDON-13865 Cannot disable/enable inputs under sc_admin role in Splunk Cloud
2017-05-10 ADDON-14039 Incremental S3 input fails to decode non-utf8 encoded files
2017-05-10 ADDON-13651 Describe EC2 is blocked by API throttling of get EBS snapshot data
2017-03-23 ADDON-13492, ADDON-13015, ADDON-13855 Ingesting a continuous stream of large files (e.g., 20MB) from a single incremental S3 data input may cause out-of-memory error
2017-03-06 ADDON-11846, SPL-138046 Logging breaks on rotation when multiple inputs write to the same log. If > 6 inputs, some inputs cannot log
2017-02-28 ADDON-13867 Major performance issue for incremental S3 data inputs when ingesting large plain text files (max throughput only around 4MB/s for files of size 20MB)

Known issues

Version 4.3.0 of the Splunk Add-on for Amazon Web Services has the following known issues.


Date filed Issue number Description
2018-01-05 ADDON-16518 When kinesis and cloudwatch inputs send large volumes of data over HEC, HEC can block the ingest pipeline, which breaks non-HEC inputs.

Workaround:
Set use_hec=false in [global_settings] stanza of aws_kinesis.conf and/or aws_cloudwatch.conf
2017-09-21 ADDON-15918 AWS TA is unable to validate role ARNs with "/" in path
2017-09-03 ADDON-15718 Duplicate cloudfront data in description when there are more than 1 regions
2017-09-01 ADDON-15712 It stops pulling Kinesis stream data when the Kinesis stream is resharded
2017-07-25 ADDON-15371 Add-on should support non-UTF fields in access logs.
2017-06-29 ADDON-15188 Too long input name lead to modular input failure
2017-03-29 ADDON-14287 After you replace an IAM role attached to an EC2 instance, the inputs that use the old IAM role stop collecting data.
2017-02-24 ADDON-13856, ADDON-13200 Add input name as part of Kinesis checkpoint file name
2016-12-22 ADDON-12867, ADDON-11894 S3 input: large key numbers lead to excessively large checkpoint files

Workaround:
To migrate to SQS based S3 or Incremental S3. Large number of files always leads to large size of checkpoint by the nature of Generic S3.

This will improve the checkpoint file size, however, as long as the Jira is not fixed, the checkpoint file size might still be not as little as expected.

2016-12-14 ADDON-12700 Pagination issue in Account page.

Third-party software attributions

Version 4.3.0 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Version 4.2.3

Version 4.2.3 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms.

Splunk platform versions 6.4 and later
CIM 4.3 and later
Platforms Platform independent
Vendor Products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, Inspector, Kinesis, S3, VPC Flow Log, Billing services, SQS, and SNS.

New features

Version 4.2.3 of the Splunk Add-on for AWS does not contain any new features.

Fixed issues

Version 4.2.3 of the Splunk Add-on for Amazon Web Services fixes the following issues.


Date resolved Issue number Description
2017-04-26 ADDON-13891 The S3 incremental input fails to skip the Glacier storage type keys
2017-04-16 ADDON-11326 Unexpected timestamp format blocks data ingestion
2017-04-06 ADDON-13768 Upgrading the add-on causes the EC2 configuration in the Splunk App for AWS to fail with IAM Role

Known issues

Version 4.2.3 of the Splunk Add-on for Amazon Web Services has the following known issues.


Date filed Issue number Description
2017-09-03 ADDON-15718 Duplicate cloudfront data in description when there are more than 1 regions
2017-09-01 ADDON-15712 It stops pulling Kinesis stream data when the Kinesis stream is resharded
2017-06-22 ADDON-15124 Logging breaks on rotation for Billing & AWS Config when multiple inputs write to the same log.
2017-05-24 ADDON-14890 Add-on truncates Kinesis stream dropdown to 20 items.
2017-03-29 ADDON-14287 After you replace an IAM role attached to an EC2 instance, the inputs that use the old IAM role stop collecting data.
2017-03-09 ADDON-14038 Orphan process issue after master process been force killed
2017-03-09 ADDON-14039 Incremental S3 input fails to decode non-utf8 encoded files
2017-02-27 ADDON-13865 Cannot disable/enable inputs under sc_admin role in Splunk Cloud
2017-02-27 ADDON-13867 Major performance issue for incremental S3 data inputs when ingesting large plain text files (max throughput only around 4MB/s for files of size 20MB)
2017-02-27 ADDON-13879 Regional Reserve Instance is missing in description data
2017-02-26 ADDON-13860 Configuring more AWS accounts increases CPU usage and lowers throughput performance due to increased API calls

Workaround:
Consolidate AWS accounts when configuring the Splunk Add-on for AWS.
2017-02-24 ADDON-13856, ADDON-13200 Add input name as part of Kinesis checkpoint file name
2017-02-19 ADDON-13651 Describe EC2 is blocked by API throttling of get EBS snapshot data
2017-02-06 ADDON-13492, ADDON-13015, ADDON-13855 Ingesting a continuous stream of large files (e.g., 20MB) from a single incremental S3 data input may cause out-of-memory error
2017-01-13 ADDON-13282 Cannot change Description interval in UI more than once
2016-12-28 ADDON-12983 S3 dead loop when processing extremely large S3 files
2016-12-22 ADDON-12867, ADDON-11894 S3 input: large key numbers lead to excessively large checkpoint files

Workaround:
To migrate to SQS based S3 or Incremental S3. Large number of files always leads to large size of checkpoint by the nature of Generic S3.

This will improve the checkpoint file size, however, as long as the Jira is not fixed, the checkpoint file size might still be not as little as expected.

2016-12-14 ADDON-12700 Pagination issue in Account page.
2016-11-21 ADDON-12267 Disabling an active incremental s3 data input may cause duplicate data

Third-party software attributions

Version 4.2.3 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Version 4.2.2

Version 4.2.2 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms.

Splunk platform versions 6.3 and later
CIM 4.3 and later
Platforms Platform independent
Vendor Products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, Inspector, Kinesis, S3, VPC Flow Log, Billing services, SQS, and SNS.

New features

Version 4.2.2 of the Splunk Add-on for AWS does not contain any new features.

Fixed issues

Version 4.2.2 of the Splunk Add-on for Amazon Web Services fixes the following issues.


Date resolved Issue number Description
2017-01-21 ADDON-13369 Failed to list S3 buckets and Kinesis streams in GUI in proxy mode

Known issues

Version 4.2.2 of the Splunk Add-on for Amazon Web Services has the following known issues.


Date filed Issue number Description
2017-03-29 ADDON-14287 After you replace an IAM role attached to an EC2 instance, the inputs that use the old IAM role stop collecting data.
2017-03-09 ADDON-14038 Orphan process issue after master process been force killed
2017-03-09 ADDON-14039 Incremental S3 input fails to decode non-utf8 encoded files
2017-02-28 ADDON-13891 The S3 incremental input fails to skip the Glacier storage type keys
2017-02-27 ADDON-13865 Cannot disable/enable inputs under sc_admin role in Splunk Cloud
2017-02-27 ADDON-13867 Major performance issue for incremental S3 data inputs when ingesting large plain text files (max throughput only around 4MB/s for files of size 20MB)
2017-02-26 ADDON-13860 Configuring more AWS accounts increases CPU usage and lowers throughput performance due to increased API calls

Workaround:
Consolidate AWS accounts when configuring the Splunk Add-on for AWS.
2017-02-24 ADDON-13856, ADDON-13200 Add input name as part of Kinesis checkpoint file name
2017-02-19 ADDON-13651 Describe EC2 is blocked by API throttling of get EBS snapshot data
2017-02-06 ADDON-13492, ADDON-13015, ADDON-13855 Ingesting a continuous stream of large files (e.g., 20MB) from a single incremental S3 data input may cause out-of-memory error
2017-01-13 ADDON-13282 Cannot change Description interval in UI more than once
2016-12-28 ADDON-12983 S3 dead loop when processing extremely large S3 files
2016-12-22 ADDON-12867, ADDON-11894 S3 input: large key numbers lead to excessively large checkpoint files

Workaround:
To migrate to SQS based S3 or Incremental S3. Large number of files always leads to large size of checkpoint by the nature of Generic S3.

This will improve the checkpoint file size, however, as long as the Jira is not fixed, the checkpoint file size might still be not as little as expected.

2016-11-21 ADDON-12267 Disabling an active incremental s3 data input may cause duplicate data

Third-party software attributions

Version 4.2.2 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Version 4.2.1

Version 4.2.1 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms.

Splunk platform versions 6.3 and later
CIM 4.3 and later
Platforms Platform independent
Vendor Products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, Inspector, Kinesis, S3, VPC Flow Log, Billing services, SQS, and SNS.

New features

Added support for two new AWS regions: EU (London) and Canada (Central).

Fixed issues

Version 4.2.1 of the Splunk Add-on for Amazon Web Services fixes the following issues.


Date resolved Issue number Description
2017-01-12 ADDON-13260 Error message during restart Splunk on EC2 instance
2017-01-11 ADDON-13209 Unexpected SQS message increases the size of the checkpoint file in SQS-based CloudTrail input and causes performance drop
2017-01-09 ADDON-11838 Cloudtrail event username mismatch between AWS console and app
2017-01-06 ADDON-12874 ExpiredToken error when calling the ListObjects operation may terminate the process

Known issues

Version 4.2.1 of the Splunk Add-on for Amazon Web Services has the following known issues.


Date filed Issue number Description
2017-02-27 ADDON-13865 Cannot disable/enable inputs under sc_admin role in Splunk Cloud
2017-02-27 ADDON-13867 Major performance issue for incremental S3 data inputs when ingesting large plain text files (max throughput only around 4MB/s for files of size 20MB)
2017-02-26 ADDON-13860 Configuring more AWS accounts increases CPU usage and lowers throughput performance due to increased API calls

Workaround:
Consolidate AWS accounts when configuring the Splunk Add-on for AWS.
2017-02-24 ADDON-13856, ADDON-13200 Add input name as part of Kinesis checkpoint file name
2017-02-19 ADDON-13651 Describe EC2 is blocked by API throttling of get EBS snapshot data
2017-02-09 ADDON-13768 Upgrading the add-on causes the EC2 configuration in the Splunk App for AWS to fail with IAM Role
2017-02-06 ADDON-13492, ADDON-13015, ADDON-13855 Ingesting a continuous stream of large files (e.g., 20MB) from a single incremental S3 data input may cause out-of-memory error
2017-01-19 ADDON-13369 Failed to list S3 buckets and Kinesis streams in GUI in proxy mode
2017-01-13 ADDON-13282 Cannot change Description interval in UI more than once
2016-12-28 ADDON-12983 S3 dead loop when processing extremely large S3 files
2016-12-22 ADDON-12867, ADDON-11894 S3 input: large key numbers lead to excessively large checkpoint files

Workaround:
To migrate to SQS based S3 or Incremental S3. Large number of files always leads to large size of checkpoint by the nature of Generic S3.

This will improve the checkpoint file size, however, as long as the Jira is not fixed, the checkpoint file size might still be not as little as expected.

2016-11-21 ADDON-12267 Disabling an active incremental s3 data input may cause duplicate data

Third-party software attributions

Version 4.2.1 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Version 4.2.0

Version 4.2.0 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms.

Splunk platform versions 6.3 and later
CIM 4.3 and later
Platforms Platform independent
Vendor Products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, Inspector, Kinesis, S3, VPC Flow Log, Billing services, SQS, and SNS.

New features

Version 4.2.0 of the Splunk Add-on for Amazon Web Services supports the AWS Security Token Service (AWS STS) AssumeRole API action that lets you use IAM roles to delegate permissions to IAM users to access these AWS resources. You can configure accounts to use AssumeRole in these data inputs: S3 (general and incremental), Billing, Description, CloudWatch, Kinesis.

Fixed issues

Version 4.2.0 of the Splunk Add-on for Amazon Web Services fixes the following issues.


Date resolved Issue number Description
2016-12-27 ADDON-12918 API throttling error occurs during ingestion of ELB description input data and blocks ELB data collection
2016-12-12 ADDON-12600 Incorrect file name format blocks data ingestion
2016-12-12 ADDON-12660 Failed to retrieve cloudfront_distributions through proxy
2016-12-07 ADDON-12342 Poor list bucket performance in collecting S3 data
2016-12-07 ADDON-12344 Unwarranted config changed message in the S3 incremental input log
2016-12-06 ADDON-12236 Force killing splunkd leaves input orphan processes, which will be killed after splunkd restarts
2016-12-06 ADDON-12123, ADDON-12485 Race condition after checkpoint files are replaced
2016-12-06 ADDON-12397 One invalid Kinesis input blocks all other Kinesis inputs
2016-12-06 ADDON-12340 ReadTimeoutError - S3 data collection failed
2016-11-28 ADDON-11855, ADDON-11852 Performance degradation of AWS add-on modular input data collection in Splunk Platform 6.5.0
2016-11-27 ADDON-11894, ADDON-12867 S3-generic input ckpt file is too large

Known issues

Version 4.2.0 of the Splunk Add-on for Amazon Web Services has the following known issues.


Date filed Issue number Description
2018-01-05 ADDON-16518 When kinesis and cloudwatch inputs send large volumes of data over HEC, HEC can block the ingest pipeline, which breaks non-HEC inputs.

Workaround:
Set use_hec=false in [global_settings] stanza of aws_kinesis.conf and/or aws_cloudwatch.conf
2017-02-27 ADDON-13865 Cannot disable/enable inputs under sc_admin role in Splunk Cloud
2017-02-26 ADDON-13860 Configuring more AWS accounts increases CPU usage and lowers throughput performance due to increased API calls

Workaround:
Consolidate AWS accounts when configuring the Splunk Add-on for AWS.
2017-02-24 ADDON-13856, ADDON-13200 Add input name as part of Kinesis checkpoint file name
2017-02-19 ADDON-13651 Describe EC2 is blocked by API throttling of get EBS snapshot data
2017-02-06 ADDON-13492, ADDON-13015, ADDON-13855 Ingesting a continuous stream of large files (e.g., 20MB) from a single incremental S3 data input may cause out-of-memory error
2017-01-19 ADDON-13369 Failed to list S3 buckets and Kinesis streams in GUI in proxy mode
2017-01-13 ADDON-13282 Cannot change Description interval in UI more than once
2017-01-11 ADDON-13260 Error message during restart Splunk on EC2 instance
2017-01-05 ADDON-13209 Unexpected SQS message increases the size of the checkpoint file in SQS-based CloudTrail input and causes performance drop
2017-01-02 ADDON-13041 s3 indexing latency introduced by assumerole feature (even account do not have assumerole)
2016-12-28 ADDON-12983 S3 dead loop when processing extremely large S3 files
2016-12-27 ADDON-12931 Upgrading from version 4.0.0 to 4.2.0 causes the Start Date/Time field value to be displayed incorrectly on the UI
2016-12-24 ADDON-12874 ExpiredToken error when calling the ListObjects operation may terminate the process
2016-12-22 ADDON-12867, ADDON-11894 S3 input: large key numbers lead to excessively large checkpoint files

Workaround:
To migrate to SQS based S3 or Incremental S3. Large number of files always leads to large size of checkpoint by the nature of Generic S3.

This will improve the checkpoint file size, however, as long as the Jira is not fixed, the checkpoint file size might still be not as little as expected.

2016-12-14 ADDON-12700 Pagination issue in Account page.
2016-11-21 ADDON-12267 Disabling an active incremental s3 data input may cause duplicate data
2016-10-27 ADDON-11838 Cloudtrail event username mismatch between AWS console and app
2016-09-08 ADDON-11225 Fails to download Billing files due to "Operation timed out" error
2015-09-09 ADDON-12762 Selecting all regions and all services in CloudWatch input results in some invalid tasks.

Third-party software attributions

Version 4.2.0 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Version 4.1.2

Version 4.1.2 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms.

Splunk platform versions 6.3, 6.4, 6.5
CIM 4.3 or later
Platforms Platform independent
Vendor Products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, Inspector, Kinesis, S3, VPC Flow Log, Billing services, SQS, and SNS.

New features

Version 4.1.2 of the Splunk Add-on for Amazon Web Services contains no new features.

Fixed issues

Version 4.1.2 of the Splunk Add-on for Amazon Web Services fixes the following issues.


Date resolved Issue number Description
2016-11-16 ADDON-12078 S3 incremental orphan process issue
2016-11-08 ADDON-11960 App menu display issue in Splunk Light

Known issues

Version 4.1.2 of the Splunk Add-on for Amazon Web Services has the following known issues.


Date filed Issue number Description
2018-01-05 ADDON-16518 When kinesis and cloudwatch inputs send large volumes of data over HEC, HEC can block the ingest pipeline, which breaks non-HEC inputs.

Workaround:
Set use_hec=false in [global_settings] stanza of aws_kinesis.conf and/or aws_cloudwatch.conf
2017-02-27 ADDON-13865 Cannot disable/enable inputs under sc_admin role in Splunk Cloud
2017-02-24 ADDON-13856, ADDON-13200 Add input name as part of Kinesis checkpoint file name
2017-02-19 ADDON-13651 Describe EC2 is blocked by API throttling of get EBS snapshot data
2017-02-06 ADDON-13492, ADDON-13015, ADDON-13855 Ingesting a continuous stream of large files (e.g., 20MB) from a single incremental S3 data input may cause out-of-memory error
2017-01-13 ADDON-13282 Cannot change Description interval in UI more than once
2017-01-05 ADDON-13209 Unexpected SQS message increases the size of the checkpoint file in SQS-based CloudTrail input and causes performance drop
2016-12-26 ADDON-12918 API throttling error occurs during ingestion of ELB description input data and blocks ELB data collection
2016-12-12 ADDON-12660 Failed to retrieve cloudfront_distributions through proxy
2016-11-25 ADDON-12395 File descriptor leaking in generic S3 due to boto2 defects
2016-11-23 ADDON-12342 Poor list bucket performance in collecting S3 data
2016-11-23 ADDON-12340 ReadTimeoutError - S3 data collection failed
2016-11-21 ADDON-12267 Disabling an active incremental s3 data input may cause duplicate data
2016-11-18 ADDON-12236 Force killing splunkd leaves input orphan processes, which will be killed after splunkd restarts
2016-11-08 ADDON-11974 Cannot get CloudWatch data using some default configuration in Add-on
2016-10-28 ADDON-11846, SPL-138046 Logging breaks on rotation when multiple inputs write to the same log. If > 6 inputs, some inputs cannot log
2016-09-08 ADDON-11225 Fails to download Billing files due to "Operation timed out" error

Third-party software attributions

Version 4.1.2 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Version 4.1.1

Version 4.1.1 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms.

Splunk platform versions 6.3, 6.4 and 6.5
CIM 4.3 or later
Platforms Platform independent
Vendor Products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, Inspector, Kinesis, S3, VPC Flow Log, Billing services, SQS, and SNS.

New features

Version 4.1.1 of the Splunk Add-on for Amazon Web Services contains no new features.

Fixed issues

Version 4.1.1 of the Splunk Add-on for Amazon Web Services fixes the following issues.

Resolved date Issue number Description
2016-10-12 ADDON-11604 Incremental S3 fails to collect data using the IAM role.
2016-09-30 ADDON-11470 The inputs page cannot display more than 30 inputs (S3 as input).
2016-10-11 ADDON-11498, ADDON-11488 Ingesting data from aws:cloudwatchlogs results in invalid JSON format with extraneous trailing angle brackets.
2016-10-04 ADDON-11482 Cloudtrail/SQS fails to collect data using the IAM role.

Known issues

Version 4.1.1 of the Splunk Add-on for Amazon Web Services has the following known issues.


Date filed Issue number Description
2016-12-26 ADDON-12918 API throttling error occurs during ingestion of ELB description input data and blocks ELB data collection
2016-11-23 ADDON-12342 Poor list bucket performance in collecting S3 data
2016-11-17 ADDON-12232 Get S3 error during upgrade from 4.0.0
2016-11-14 ADDON-12123, ADDON-12485 Race condition after checkpoint files are replaced
2016-11-10 ADDON-12072 Do not support Splunk global proxy. Update it in add-on configuration if needed.
2016-11-10 ADDON-12078 S3 incremental orphan process issue
2016-11-08 ADDON-11974 Cannot get CloudWatch data using some default configuration in Add-on
2016-11-07 ADDON-11960 App menu display issue in Splunk Light
2016-11-02 ADDON-11893 IO exception in S3 input
2016-11-02 ADDON-11894, ADDON-12867 S3-generic input ckpt file is too large
2016-10-30 ADDON-11855, ADDON-11852 Performance degradation of AWS add-on modular input data collection in Splunk Platform 6.5.0
2016-10-28 ADDON-11847 s3 input zombie processes

Workaround:
Update the symbolic link so that /bin/sh targets /bin/bash.

$ debconf-set-selections <<< "dash dash/sh string false" $ dpkg-reconfigure -f noninteractive dash

2016-10-28 ADDON-11846, SPL-138046 Logging breaks on rotation when multiple inputs write to the same log. If > 6 inputs, some inputs cannot log
2016-09-22 ADDON-11415 The input name is case sensitive lead to failure on Windows platform
2016-09-13 ADDON-11295 Cloudtrail still delete SQS message even if failed to get S3 file
2016-09-08 ADDON-11225 Fails to download Billing files due to "Operation timed out" error
2016-08-18 ADDON-10957 Log level set to ERROR but still found INFO logs
2016-06-20 ADDON-10286 CloudWatch modular input generates duplicate events when the Splunk platform is restarted

Workaround:
dedup based on the _time field
2016-05-30 ADDON-9753 Proxy password does not support the special characters '|', ':' or '@'
2016-05-12 ADDON-9435 Wrong number of inputs listed on Account page.
2016-05-12 ADDON-9422 CloudWatch input can have data loss when empty results are returned twice in succession and then Splunk platform restarts before the input next collects data.
2016-05-11 ADDON-9408 Detailed Billing is not indexed using UTC timezone
2016-05-11 ADDON-9409 Checkpoints file will not be removed when deleting Config Rules
2016-05-07 ADDON-9332 fails to get latest cloudwatch data sometimes
2016-04-29 ADDON-9148 Updating directly from v2.0.0 to v4.0.0 makes existing accounts unavailable
2016-04-28 ADDON-9133 CloudWatch default configuration may not work in cases where there are millions of dimensions
2016-04-28 ADDON-9145 Error message shown on input creation screen has logic issues and is not as specific as we could be
2016-01-13 ADDON-7448 In the Description data input, the port range defaults to null in vpc_network_acls if no range is specified, which is confusing, because it actually has a range of "all".
2015-12-29 ADDON-7239 Using "/" in data input name causes exceptions. UI does not accept this character in the input names, but if you configure your input using conf files, you will find exceptions in logs.
2015-12-22 ADDON-7159 After removing all search peers, add-on still shows performance warnings.

Workaround:
Restart a Splunk platform instance after changing its role.
2015-12-16 ADDON-7035 Add-on ingests the header line of the CloudFront access log, but it should be skipped.
2015-11-26 ADDON-6701 EC2, RDS, ELB, and EC2 APIs do not consider pagination.
2015-10-14 ADDON-6056 S3 logging errors on Windows.
2015-10-13 ADDON-6043 SQS message mistakenly deleted when the add-on throws an error retrieving data from an S3 bucket.
2015-09-11 ADDON-5500 Preconfigured reports for billing data cannot handle reports that have a mix of different currencies. The report will use the first currency found and apply that to all costs.
2015-09-11 ADDON-5499 CloudWatch: Previous selected Metric namespace always exists in the list regardless of the region change
2015-09-10 ADDON-5471 Deleting a CloudWatch data input takes too long.
2015-09-10 ADDON-5481 The add-on configuration UI does not handle insufficient Splunk user permissions gracefully.
2015-09-07 ADDON-5355 Different error message for same error when creating duplicated data inputs.
2015-09-06 ADDON-5354 Using keyboard to delete selections from configuration dropdown multi-select field causes drop-down list to appear in corner of screen.
2015-09-01 ADDON-5309 UI default value is not read from default input config file
2015-09-01 ADDON-5295 Description inconsistent in the GUI for CloudTrail service and CloudTrail from S3 service blacklist behavior.
2015-04-02 ADDON-3578 S3: uppercase bucket names cause an error
2014-09-28 ADDON-2135 The list of regions shown in inputs configuration in Splunk Web shows all Amazon regions regardless of the permissions associated with the selected AWS account.
2014-09-26 ADDON-2118 Data inputs continue to work after user deletes the account used for that input.

Workaround:
Restart the Splunk platform after deleting or modifying an AWS account.
2014-09-25 ADDON-2113 The app.conf file includes a stanza for a proxy server configuration with a hashed password even if the user has not configured a proxy or password.

Workaround:
This behavior is expected because Splunk Enterprise automatically sets the proxy field to 0 and saves an encrypted entry in app.conf.
2014-09-16 ADDON-2029 In saved search "Monthly Cost till *" _time is displayed per day rather than per month.

Third-party software attributions

Version 4.1.1 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.


Version 4.1.0

Version 4.1.0 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms.

Splunk platform versions 6.3, 6.4
CIM 4.3 or later
Platforms Platform independent
Vendor Products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, Inspector, Kinesis, S3, VPC Flow Log, Billing services, SQS, and SNS.

New features

Version 4.1.0 of the Splunk Add-on for Amazon Web Services has the following new features.

Date Issue number Description
2016-09-22 ADDON-6145 Add AWS SQS modular input for Splunk add-on for AWS.
2016-09-22 ADDON-6146 Add custom alert to AWS SNS for Splunk add-on for AWS.
2016-09-22 ADDON-10952 Performance enhancement for AWS Cloudtrail modular input.
2016-09-22 ADDON-11149 Add Record Format field for AWS Kinesis modular input.
2016-09-22 ADDON-10917 Mapping to ITSI IaaS data module.
2016-09-22 ADDON-10941 Add new incremental data collection for S3 modular input.
2016-09-22 ADDON-10414 Checkpoint and performance enhancement for S3 modular input.
2016-09-22 ADDON-10906 Performance and API call enhancement for Cloudwatch modular input.

Fixed issues

Version 4.1.0 of the Splunk Add-on for Amazon Web Services fixes the following issues.

Resolved date Issue number Description
2016-09-20 ADDON-11251 There will be data loss of AWS S3 input if the network connection is bad.
2016-09-20 ADDON-11196 If there is a blank space at the beginning or the end of the input name (or both). The input name displays on the UI is not the consistent with the one saved in the configuration file.
2016-09-20 ADDON-11056 In the AWS Region list, it displays ap-northeast-2 instead of Seoul.
2016-09-20 ADDON-10980 Line breaker error for AWS S3 input.
2016-09-14 ADDON-10186 AWS Config fails to fetch S3 object in AWS GovCloud (US) region.
2016-09-09 ADDON-11009 Vanguard: Not getting data from 1 of 3 S3 inputs. This is considered critical for the customer as they have PS on site.
2016-08-18 ADDON-10137 If the number of the AWS input exceeds 30, some of the inputs cannot run successfully.
2016-09-14 ADDON-9778 There are some errors of AWS Kinesis modular input if the request from HEC exceeds its max limit.
2016-09-05 ADDON-9732 Failed to get proxy credentials when password includes # character.
2016-08-28 ADDON-9533 The default Dimension Name is empty square brackets for Autoscaling and EBS namespaces.
2016-08-08 ADDON-9328 CloudWatch data input encounters API rate limit for large metrics.
2016-09-09 ADDON-8758 Mixing log types or gzip with plain text in the same stream causes knowledge extraction to fail for CloudWatch Logs data collected through Kinesis

Known issues

Version 4.1.0 of the Splunk Add-on for Amazon Web Services has the following known issues.


Date filed Issue number Description
2016-10-30 ADDON-11855, ADDON-11852 Performance degradation of AWS add-on modular input data collection in Splunk Platform 6.5.0
2016-10-28 ADDON-11846, SPL-138046 Logging breaks on rotation when multiple inputs write to the same log. If > 6 inputs, some inputs cannot log
2016-10-12 ADDON-11611 Fails to publish search result to SNS using the IAM role.
2016-10-11 ADDON-11604 S3 incremental failed to fetch data using IAM role
2016-10-05 ADDON-11498 Trailing angle bracket and invalid JSON in aws:cloudwatchlogs
2016-10-04 ADDON-11488 aws_cloudwatch_logs_data_loader.py#L88
2016-10-03 ADDON-11482 Cloudtrail input error after upgrade to 4.1
2016-09-28 ADDON-11470 Inputs page doesn't show more than 30 inputs (S3 as input)
2016-09-22 ADDON-11415 The input name is case sensitive lead to failure on Windows platform
2016-09-19 ADDON-11326 Unexpected timestamp format blocks data ingestion
2016-09-13 ADDON-11295 Cloudtrail still delete SQS message even if failed to get S3 file
2016-09-08 ADDON-11225 Fails to download Billing files due to "Operation timed out" error
2016-08-18 ADDON-10957 Log level set to ERROR but still found INFO logs
2016-06-20 ADDON-10286 CloudWatch modular input generates duplicate events when the Splunk platform is restarted

Workaround:
dedup based on the _time field
2016-05-30 ADDON-9753 Proxy password does not support the special characters '|', ':' or '@'
2016-05-30 ADDON-9745 Add-on does not support proxy accounts that do not require passwords
2016-05-12 ADDON-9435 Wrong number of inputs listed on Account page.
2016-05-12 ADDON-9422 CloudWatch input can have data loss when empty results are returned twice in succession and then Splunk platform restarts before the input next collects data.
2016-05-11 ADDON-9408 Detailed Billing is not indexed using UTC timezone
2016-05-11 ADDON-9409 Checkpoints file will not be removed when deleting Config Rules
2016-05-07 ADDON-9332 fails to get latest cloudwatch data sometimes
2016-04-29 ADDON-9148 Updating directly from v2.0.0 to v4.0.0 makes existing accounts unavailable
2016-04-28 ADDON-9133 CloudWatch default configuration may not work in cases where there are millions of dimensions
2016-04-28 ADDON-9145 Error message shown on input creation screen has logic issues and is not as specific as we could be
2016-01-13 ADDON-7448 In the Description data input, the port range defaults to null in vpc_network_acls if no range is specified, which is confusing, because it actually has a range of "all".
2015-12-29 ADDON-7239 Using "/" in data input name causes exceptions. UI does not accept this character in the input names, but if you configure your input using conf files, you will find exceptions in logs.
2015-12-22 ADDON-7159 After removing all search peers, add-on still shows performance warnings.

Workaround:
Restart a Splunk platform instance after changing its role.
2015-12-16 ADDON-7035 Add-on ingests the header line of the CloudFront access log, but it should be skipped.
2015-11-26 ADDON-6701 EC2, RDS, ELB, and EC2 APIs do not consider pagination.
2015-10-14 ADDON-6056 S3 logging errors on Windows.
2015-10-13 ADDON-6043 SQS message mistakenly deleted when the add-on throws an error retrieving data from an S3 bucket.
2015-09-11 ADDON-5500 Preconfigured reports for billing data cannot handle reports that have a mix of different currencies. The report will use the first currency found and apply that to all costs.
2015-09-11 ADDON-5499 CloudWatch: Previous selected Metric namespace always exists in the list regardless of the region change
2015-09-10 ADDON-5471 Deleting a CloudWatch data input takes too long.
2015-09-10 ADDON-5481 The add-on configuration UI does not handle insufficient Splunk user permissions gracefully.
2015-09-07 ADDON-5355 Different error message for same error when creating duplicated data inputs.
2015-09-06 ADDON-5354 Using keyboard to delete selections from configuration dropdown multi-select field causes drop-down list to appear in corner of screen.
2015-09-01 ADDON-5309 UI default value is not read from default input config file
2015-09-01 ADDON-5295 Description inconsistent in the GUI for CloudTrail service and CloudTrail from S3 service blacklist behavior.
2015-04-02 ADDON-3578 S3: uppercase bucket names cause an error
2014-09-28 ADDON-2135 The list of regions shown in inputs configuration in Splunk Web shows all Amazon regions regardless of the permissions associated with the selected AWS account.
2014-09-26 ADDON-2118 Data inputs continue to work after user deletes the account used for that input.

Workaround:
Restart the Splunk platform after deleting or modifying an AWS account.
2014-09-25 ADDON-2113 The app.conf file includes a stanza for a proxy server configuration with a hashed password even if the user has not configured a proxy or password.

Workaround:
This behavior is expected because Splunk Enterprise automatically sets the proxy field to 0 and saves an encrypted entry in app.conf.
2014-09-16 ADDON-2029 In saved search "Monthly Cost till *" _time is displayed per day rather than per month.

Third-party software attributions

Version 4.1.0 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Version 4.0.0

Version 4.0.0 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms.

Splunk platform versions 6.2.X and later
CIM 4.0 and later
Platforms Platform independent
Vendor Products Amazon Web Services CloudTrail, CloudWatch, CloudWatch Logs, Config, Config Rules, Inspector, Kinesis, S3, VPC Flow Log, and Billing services

Upgrade

If you are upgrading from a previous version of the Splunk Add-on for AWS, be aware of the following changes which may require some actions to preserve the functionality of your existing accounts and inputs:

  • This release includes three new inputs that each require new IAM permissions. Be sure to adjust the IAM permissions of your existing accounts if you want to use them to collect these new data sources. See Configure AWS permissions for the Splunk Add-on for AWS for details.
  • If you are upgrading directly from version 2.0.0 or earlier of this add-on to the 4.0.0 version, you need to open and resave the AWS accounts using the Splunk Add-on for AWS account UI.
  • In this version, the CloudWatch input is rearchitected for better performance and improved stability. One result of this new architecture is that the input has a built in four minute delay after a polling period has ended for any given metric before the actual data collection occurs. This change ensures that there is no data loss due to latency on the AWS side.
  • This version requires a single selection for the Region Category for each AWS account. If you added accounts before region category selection was required, or if you added accounts and selected more than one region category for a single account, the upgrade to version 4.0.0 will put these accounts into an error state until you edit them to select a single region category. On your data collection node, open the add-on and check your Configuration tab to see if any of your existing accounts are missing a region category. If they are, edit the account to add the region category. Any inputs using accounts that were determined to be in error stop collecting data until the account has a region category assigned. Once the account error is resolved, the affected inputs start collecting data again automatically starting from the point when data collection stopped.

New Features

Version 4.0.0 of the Splunk Add-on for Amazon Web Services has the following new features.

Resolved date Issue number Description
2016-04-29 ADDON-7042 CloudWatch input configuration UI now provides auto-filled correct default JSON for metrics and dimensions in each namespace.
2016-04-08 ADDON-7587 Support for AWS Signature V.4 managed keys for S3 related data collection.
2016-04-05 ADDON-7818 New input and CIM mapping for Amazon Inspector data.
2016-04-05 ADDON-7817 New input and CIM mapping for AWS Config rules data.
2016-04-05 ADDON-5391 New input for data from Kinesis streams, including high volume VPC flow log data.
2016-03-31 ADDON-6811 Support for using an EC2 IAM role instead of an AWS account when the add-on's collection node is on your own managed AWS instance.
2016-03-23 ADDON-7872 Support for the Seoul region.
2016-01-08 ADDON-7311 Support for setting an initial scan time in the Billing input if configuring using the conf files.

Fixed issues

Version 4.0.0 of the Splunk Add-on for Amazon Web Services fixes the following issues.

Resolved date Defect number Description
2016-05-04 ADDON-9169
Monthly Billing is not indexed using the UTC timezone
2016-04-19 ADDON-8801
Billing initial scan time should not use last modified time of S3 key
2016-04-15 ADDON-8721
Sourcetype="aws:cloudwatchlogs:vpcflow" handles src and dest incorrectly
2016-04-11 ADDON-8686
S3 input UI cannot display custom source types when user edits the input.
2016-04-03 ADDON-8547
S3 modular input loses data if new keys are generated during the key listing process
2016-04-02 ADDON-8546
S3 logging is unclear, should include indication of which input stanza is involved.
2016-03-31 ADDON-8548
CloudWatch collection failing with "Failed to get proxy information Empty"
2016-03-15 ADDON-8299
S3 input cannot progress if keys are deleted during the data collection.
2016-02-29 ADDON-8705
Add-on throws "is not JSON serializable" error when calling AWS API for ELB information
2016-02-25 ADDON-7969
CloudWatch has performance problems in large AWS accounts.
2016-02-24 ADDON-7957
Unnecessary tag expansion slows performance.
2016-02-24 ADDON-7926
Default value of max_file_size_csv_zip_in_bytes is too small to handle large detailed billing reports
2016-02-22 ADDON-7897
s3util.py list_cloudwatch_namespaces has performance issue
2016-02-19 ADDON-7877
Upon upgrade from version 2.0.X, S3 inputs experience two problems.
Workaround: 1. Inputs with a S3 key prefix specified stop collecting data from AWS.

Workaround: Stop splunkd and go to $SPLUNK_HOME/var/lib/modinputs/aws_s3/, find the checkpoint file for that data input (ls -lh to list and find the large files), open the file, and note the last_modified_time in the file. Remove the checkpoint file and update the data input in inputs.conf using the last_modified_time value that you observed in the checkpoint file for the initial_scan_time in the new input. Reboot splunkd. 2. The polling_interval does not persist automatically. Workaround: In Splunk Web, open the input configuration, go to Settings, set an interval value, then click Update. Or, in local/inputs.conf, set the polling_interval to a value that matches your needs, then save the file.

2016-02-14 ADDON-7777
Not all fields are parsed for CloudFront
2016-02-13 ADDON-7778
Cannot create new input when Splunk does not have a user named "admin"
2016-02-13 ADDON-7776
CloudFront logs should be urldecoded
2016-01-25 ADDON-7573
CloudWatch input requests too many data points in long time windows.
2016-01-18 ADDON-7701
CloudWatch fails to gather data when no metrics appear in a namespace for more than 12 hours.
2015-09-11 ADDON-5498
Unclear error: Unexpected error "<class 'socket.error'>" from python handler: " Connection refused" when user specifies all regions in CloudWatch for one namespace, saves the configuration, and reloads it.
2015-09-10 ADDON-5469
Missing or improper default value for un-required fields.

Known issues

Version 4.0.0 of the Splunk Add-on for Amazon Web Services has the following known issues.


Date filed Issue number Description
2016-10-28 ADDON-11847 s3 input zombie processes

Workaround:
Update the symbolic link so that /bin/sh targets /bin/bash.

$ debconf-set-selections <<< "dash dash/sh string false" $ dpkg-reconfigure -f noninteractive dash

2016-10-28 ADDON-11846, SPL-138046 Logging breaks on rotation when multiple inputs write to the same log. If > 6 inputs, some inputs cannot log
2016-10-05 ADDON-11498 Trailing angle bracket and invalid JSON in aws:cloudwatchlogs
2016-09-22 ADDON-11415 The input name is case sensitive lead to failure on Windows platform
2016-09-19 ADDON-11326 Unexpected timestamp format blocks data ingestion
2016-09-12 ADDON-11266 Chrome failed to create account
2016-09-10 ADDON-11251 Data loss when creating multi inputs to ingesting data
2016-09-08 ADDON-11225 Fails to download Billing files due to "Operation timed out" error
2016-09-06 ADDON-11196 Strip blank space in input name
2016-08-29 ADDON-11056 Region shows "ap-northeast-2" but not Seoul
2016-08-24 ADDON-11009 Vanguard: Not getting data from 1 of 3 S3 inputs. This is considered critical for the customer as they have PS on site
2016-08-22 ADDON-10978 S3 data loss after disable/enable
2016-08-22 ADDON-10980 S3 line breaker error
2016-08-18 ADDON-10957 Log level set to ERROR but still found INFO logs
2016-07-20 ADDON-10643 Rest handler splunk_ta_aws_settings_account_region is missing
2016-07-17 ADDON-10574 Log level for can't find checkpoint should not be ERROR
2016-07-06 ADDON-10450 REST handler s3buckets still returns status 200 while connection failed
2016-06-20 ADDON-10286 CloudWatch modular input generates duplicate events when the Splunk platform is restarted

Workaround:
dedup based on the _time field
2016-06-13 ADDON-10186 AWS Config fails to fetch S3 object in AWS GovCloud (US) region
2016-05-31 ADDON-9778 HEC max limit needs to take padding into account to avoid 413 "Content-Length of <value> too large" errors
2016-05-30 ADDON-9745 Add-on does not support proxy accounts that do not require passwords
2016-05-30 ADDON-9753 Proxy password does not support the special characters '|', ':' or '@'
2016-05-27 ADDON-9732 failed to get proxy credentials when password includes # sign
2016-05-18 ADDON-9533 Dimensions default to empty square brackets for Autoscaling and EBS namespaces
2016-05-16 ADDON-9451 Monthly billing date is displayed as next month for some timezones
2016-05-12 ADDON-9435 Wrong number of inputs listed on Account page.
2016-05-12 ADDON-9434, ADDON-10137 Rest Handler Of List Data Inputs Truncates Result.

Workaround:
1) Navigate to /opt/splunk/etc/apps/Splunk_TA_aws/bin/splunktalib/rest.py

2) Change line 44 of this script from: resp, content = http.request(splunkd_uri, method=method, to resp, content = http.request(splunkd_uri + "?count=-1", method=method, 3) Save and exit

2016-05-12 ADDON-9422 CloudWatch input can have data loss when empty results are returned twice in succession and then Splunk platform restarts before the input next collects data.
2016-05-12 ADDON-9431 further save the cost with more efficient API call
2016-05-11 ADDON-9408 Detailed Billing is not indexed using UTC timezone
2016-05-11 ADDON-9409 Checkpoints file will not be removed when deleting Config Rules
2016-05-07 ADDON-9332 fails to get latest cloudwatch data sometimes
2016-05-06 ADDON-9328 CloudWatch data input encounters API rate limit for large metrics

Workaround:
Increase your granularity and polling interval in order to make fewer API calls, or contact AWS to increase your allowed number of API calls per month.
2016-04-29 ADDON-9148 Updating directly from v2.0.0 to v4.0.0 makes existing accounts unavailable
2016-04-28 ADDON-9145 Error message shown on input creation screen has logic issues and is not as specific as we could be
2016-04-28 ADDON-9133 CloudWatch default configuration may not work in cases where there are millions of dimensions
2016-04-27 ADDON-9117 Using EC2 IAM role for data collection does not work in China or GovCloud regions.
2016-04-20 ADDON-8905 Add-on throws "connection refused" error when Splunk platform restarts
2016-04-19 ADDON-8758 Mixing log types or gzip with plain text in the same stream causes knowledge extraction to fail for CloudWatch Logs data collected through Kinesis
2016-03-01 ADDON-8113 Excessive S3 API calls
2016-01-13 ADDON-7448 In the Description data input, the port range defaults to null in vpc_network_acls if no range is specified, which is confusing, because it actually has a range of "all".
2015-12-29 ADDON-7239 Using "/" in data input name causes exceptions. UI does not accept this character in the input names, but if you configure your input using conf files, you will find exceptions in logs.
2015-12-22 ADDON-7159 After removing all search peers, add-on still shows performance warnings.

Workaround:
Restart a Splunk platform instance after changing its role.
2015-12-16 ADDON-7035 Add-on ingests the header line of the CloudFront access log, but it should be skipped.
2015-11-26 ADDON-6701 EC2, RDS, ELB, and EC2 APIs do not consider pagination.
2015-10-14 ADDON-6056 S3 logging errors on Windows.
2015-10-13 ADDON-6043 SQS message mistakenly deleted when the add-on throws an error retrieving data from an S3 bucket.
2015-09-11 ADDON-5500 Preconfigured reports for billing data cannot handle reports that have a mix of different currencies. The report will use the first currency found and apply that to all costs.
2015-09-11 ADDON-5499 CloudWatch: Previous selected Metric namespace always exists in the list regardless of the region change
2015-09-10 ADDON-5471 Deleting a CloudWatch data input takes too long.
2015-09-10 ADDON-5481 The add-on configuration UI does not handle insufficient Splunk user permissions gracefully.
2015-09-07 ADDON-5355 Different error message for same error when creating duplicated data inputs.
2015-09-06 ADDON-5354 Using keyboard to delete selections from configuration dropdown multi-select field causes drop-down list to appear in corner of screen.
2015-09-01 ADDON-5309 UI default value is not read from default input config file
2015-09-01 ADDON-5295 Description inconsistent in the GUI for CloudTrail service and CloudTrail from S3 service blacklist behavior.
2015-07-06 ADDON-6177 When tmp file system runs out of space, aws_billing.py fails with IOError: No space left on device.
2015-04-02 ADDON-3578 S3: uppercase bucket names cause an error
2015-03-25 ADDON-3460 On OSs (like Debian and Ubuntu) that use dash for shell scripts, aws_cloudwatch.py spawns zombie processes.

Workaround:
Kill the processes and restart. Use bash to prevent re-occurrence.
2014-09-28 ADDON-2135 The list of regions shown in inputs configuration in Splunk Web shows all Amazon regions regardless of the permissions associated with the selected AWS account.
2014-09-26 ADDON-2118 Data inputs continue to work after user deletes the account used for that input.

Workaround:
Restart the Splunk platform after deleting or modifying an AWS account.
2014-09-25 ADDON-2113 The app.conf file includes a stanza for a proxy server configuration with a hashed password even if the user has not configured a proxy or password.

Workaround:
This behavior is expected because Splunk Enterprise automatically sets the proxy field to 0 and saves an encrypted entry in app.conf.
2014-09-16 ADDON-2029 In saved search "Monthly Cost till *" _time is displayed per day rather than per month.

Third-party software attributions

Version 4.0.0 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Version 3.0.0

Version 3.0.0 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms.

Splunk platform versions 6.2.X and later
CIM 4.0 and later
Platforms Platform independent
Vendor Products AWS CloudTrail, CloudWatch, CloudWatch Logs, Config, Billing, S3

Upgrade guide

This release includes some changes to the S3 input configuration that break backwards compatibility. If you are upgrading from a previous version and had previously used any of the following parameters, review the new behavior noted here and make any necessary changes in your existing S3 inputs:

  • interval now refers to how long splunkd should wait before checking the health of the modular input and restarting it if it has crashed. The new argument polling_interval, still shown as Interval in the UI, handles the data collection interval. If you had a custom value configured, the 3.0.0 version of the add-on copies your custom setting to the polling_interval value so that your data collection behavior does not change. However, you may wish to tune the interval value to enable splunkd to check for input crashes more frequently.
  • is_secure is deprecated and removed, but the parameter is retained in default/inputs.conf to avoid spec file violations. All traffic is over https. If you have this parameter in your local/inputs.conf, it will have no effect.
  • max_items is deprecated and removed, but the parameter is retained in default/inputs.conf to avoid spec file violations. It is set to 100000 items. If you have this parameter in your local/inputs.conf, it will have no effect.
  • queueSize is deprecated and removed. If you have this parameter in your local/inputs.conf, remove it to avoid potential data loss.
  • persistentQueueSize is deprecated and removed. If you have this parameter in your local/inputs.conf, remove it to avoid potential data loss.
  • recursion_depth is deprecated and removed, but the parameter is retained in default/inputs.conf to avoid spec file violations. The input recursively scans all subdirectories. If you have this parameter in your local/inputs.conf, it will have no effect.
  • ct_excluded_events_index is deprecated and removed, but the parameter is retained in default/inputs.conf to avoid spec file violations. Excluded events will be discarded. If you have this parameter in your local/inputs.conf, it will have no effect.

New features

Version 3.0.0 of the Splunk Add-on for Amazon Web Services has the following new features.

Resolved date Issue number Description
2015-11-16 ADDON-6690 Add-on configuration screen serves a warning message when you access it on a Splunk search head to remind you to configure it on heavy forwarders as a best practice.
2015-12-23 ADDON-6870 Support for GovCloud and China regions in the configuration UI.
2015-12-22 ADDON-6862 Support in the configuration UI and backend for new source types: aws:s3:accesslogs,

aws:cloudfront:accesslogs, aws:elb:accesslogs

2015-12-17 ADDON-6190 CloudWatch input refreshes the resource ID list every few hours so as to include additional resources to a wildcarded statement.
2015-12-17 ADDON-6187 CloudWatch collects S3 key count and total size of all keys in buckets.
2015-12-15 ADDON-6864 S3 modular input backend automatically detects the region, thus supporting bucket names with dots in them without user's needing to specify a region-specific endpoint.
2015-12-15 ADDON-6854 Deprecation of character_set parameter for S3 input. Input supports auto-detection among UTF-8 with/without BOM, UTF-16LE/BE with BOM, UTF-32BE/LE with BOM. Other character sets are not supported.
2015-12-15 ADDON-6189 Support for collecting ELB access logs using the aws:elb:accesslogs.
2015-12-14 ADDON-6869 Support for S3 buckets in the Frankfurt region with V4 signature only.
2016-12-14 ADDON-6866 Improved auditing information for log enrichment.
2015-12-14 ADDON-6859 S3 input blacklist has improved performance.
2015-12-14 ADDON-6857 S3 input whitelist has improved performance.
2015-12-14 ADDON-6860 Improved handling of process failures without duplication or loss of data.
2015-12-14 ADDON-6861 Support for checkpoint deletion behavior for the S3 input to avoid running into collection limits.
2015-12-14 ADDON-6865 Support for initial scan time in the S3 input, as well as in the new aws:s3:accesslogs, aws:cloudfront:accesslogs, and aws:elb:accesslogs source types.
2015-12-14 ADDON-6863 Improved collection behavior in the S3 input: if the key is updated without content changes, the add-on indexes the key again. If the key is changed during data collection, the add-on starts over with the data collection.
2015-12-14 ADDON-6868 The S3 input supports standard server-side KMS encrypted objects.
2015-12-14 ADDON-6855 The S3 input supports bin files.
2015-12-14 ADDON-6852 Improved performance for S3 input. Approximately 300% performance enhancement against 2.0.1 release. Over 8000% performance improvement for small files. See Performance reference for the S3 input in the Splunk Add-on for AWS for details.
2015-12-14 ADDON-6434 UI support for configuring alternate source types within the S3 input.
2015-12-14 ADDON-6196 Support for collecting CloudFront access logs with the aws:cloudfront:accesslogs source type.
2015-12-14 ADDON-6526 S3 input recognizes and skips S3 buckets with contents that have been moved to Glacier.
2015-12-14 ADDON-6188 New source type for S3 access logs: aws:s3:accesslogs.
2015-12-03 ADDON-6433 Improvements to the Description input's API and interval configuration UI.
2015-12-01 ADDON-6519 Improved timeout behavior in the configuration UI.
2015-11-26 ADDON-6194 Improvements to field aliasing for AWS regions.
2015-11-26 ADDON-6207 Gather metadata through the Description input for EBS, VPC, Security Group, Subnet, Network ACL, Key Pairs, ELB, CloudFront, RDS.

Fixed issues

Version 3.0.0 of the Splunk Add-on for Amazon Web Services fixes the following issues.

Resolved date Defect number Description
2016-01-14 ADDON-7291 S3 data input only shows 30 entries at maximum.
2016-01-03 ADDON-7258 Configuration screen needs to show better error message when user may be trying to use an invalid AWS account.
2015-12-31 ADDON-7253 Default initial_scan_datetime should be ISO8601 instead of the current default of current time minus 7 days.
2015-12-16 ADDON-7031 UI errors when using the base URL via reverse proxy.
2015-12-15 ADDON-6754 Typo in aws_cloudtrail.py script throws critical error in aws_cloudtrail.log with "NameError: global name 'taaw' is not defined".
2015-12-15 ADDON-7008 Add-on is not indexing ELB data through Description input.
2015-12-14 ADDON-6308 S3 input should validate key name does not include invalid characters such as leading or trailing whitespace.
2015-11-26 ADDON-6698 AWS Billing account ID should be payer's account ID instead of linked account ID.
2015-12-22 ADDON-5491 The add-on configuration UI displays all regions instead of those within the selected account's permission scope.
2015-12-20 ADDON-6958 / ADDON-5474 No detailed error shown while getting S3 buckets via REST endpoint with wrong proxy or account settings.
2015-01-22 ADDON-3050/
SPL-96729/
SPL-64904
S3 input is breaking lines incorrectly and inconsistently indexing only partial events due to use of persistentQueueSize.
2014-08-14 ADDON-1827 Checkpoints are not cleared after data inputs are removed or the add-on is uninstalled, thus if you create a new input with the same name as the deleted one, the add-on uses the checkpoint from the old input.

Known issues

Version 3.0.0 of the Splunk Add-on for Amazon Web Services has the following known issues.

Date filed Defect number Description
2016-05-04 ADDON-9169
Monthly Billing is not indexed by using UTC timezone
2016-04-28 ADDON-9145
Error message shown on input creation screen has logic issues and is not as specific as we could be
2016-04-19 ADDON-8801
Billing initial scan time should not use last modified time of S3 key
2016-04-15 ADDON-8721
Sourcetype="aws:cloudwatchlogs:vpcflow" handles src and dest incorrectly
2016-04-11 ADDON-8686
S3 input UI cannot display custom source types when user edits the input.
2016-04-03 ADDON-8547
S3 modular input loses data if new keys are generated during the key listing process
2016-04-02 ADDON-8546
S3 logging is unclear, should include indication of which input stanza is involved.
2016-03-31 ADDON-8548
Cloudwatch Collection failing with Failed to get proxy information Empty
2016-03-15 ADDON-8299
S3 input cannot progress if keys are deleted during the data collection.
2016-02-29 ADDON-8705
Add-on throws "is not JSON serializable" error when calling AWS API for ELB information
2016-02-25 ADDON-7969
CloudWatch has performance problems in large AWS accounts.
2016-02-24 ADDON-7957
Unnecessary tag expansion slows performance.
2016-02-24 ADDON-7926
Default value of max_file_size_csv_zip_in_bytes is too small to handle large detailed billing reports
2016-02-22 ADDON-7897
s3util.py list_cloudwatch_namespaces has performance issue
2016-02-19 ADDON-7877
Upon upgrade from version 2.0.X, S3 inputs experience two problems.
Workaround: 1. Inputs with a S3 key prefix specified stop collecting data from AWS.

Workaround: Stop splunkd and go to $SPLUNK_HOME/var/lib/modinputs/aws_s3/, find the checkpoint file for that data input (ls -lh to list and find the large files), open the file, and note the last_modified_time in the file. Remove the checkpoint file and update the data input in inputs.conf using the last_modified_time value that you observed in the checkpoint file for the initial_scan_time in the new input. Reboot splunkd. 2. The polling_interval does not persist automatically. Workaround: In Splunk Web, open the input configuration, go to Settings, set an interval value, then click Update. Or, in local/inputs.conf, set the polling_interval to a value that matches your needs, then save the file.

2016-02-14 ADDON-7777
Not all fields are parsed for CloudFront
2016-02-13 ADDON-7778
Cannot create new input when Splunk does not have a user named "admin"
2016-02-13 ADDON-7776
CloudFront logs should be urldecoded
2016-02-11 ADDON-7764
FIPS mode is not supported by this add-on.
2016-01-25 ADDON-7573
CloudWatch input requests too many data points in long time windows.
2016-01-18 ADDON-7701
CloudWatch fails to gather data when no metrics appear in a namespace for more than 12 hours.
2016-01-13 ADDON-7448
In the Description data input, the port range defaults to null in vpc_network_acls if no range is specified, which is confusing, because it actually has a range of "all".
2015-12-29 ADDON-7239
Using "/" in data input name causes exceptions. UI does not accept this character in the input names, but if you configure your input using conf files, you will find exceptions in logs.
2015-12-22 ADDON-7160
Add-on throws a timeout error in the UI when user attempts to create a new S3 input, but successfully creates the input in the backend, causing errors if the user tries to create the same input again.
2015-12-22 ADDON-7159
After removing all search peers, add-on still shows performance warnings.
Workaround: Restart a Splunk platform instance after changing its role.
2015-12-21 ADDON-7077
Infrequent Access storage type not supported
2015-12-16 ADDON-7035
Add-on ingests the header line of the CloudFront access log, but it should be skipped.
2015-11-26 ADDON-6701
EC2, RDS, ELB, and EC2 APIs do not consider pagination.
2015-10-14 ADDON-6056
S3 logging errors on Windows.
2015-10-13 ADDON-6043
SQS message mistakenly deleted when the add-on throws an error retrieving data from an S3 bucket.
2015-09-11 ADDON-5500
Preconfigured reports for billing data cannot handle reports that have a mix of different currencies. The report will use the first currency found and apply that to all costs.
2015-09-11 ADDON-5499
CloudWatch: Previous selected Metric namespace always exists in the list regardless of the region change
2015-09-11 ADDON-5498
Unclear error: Unexpected error "<class 'socket.error'>" from python handler: " Connection refused" when user specifies all regions in CloudWatch for one namespace, saves the configuration, and reloads it.
2015-09-10 ADDON-5481
The add-on configuration UI does not handle insufficient Splunk user permissions gracefully.
2015-09-10 ADDON-5471
Deleting a CloudWatch data input takes too long.
2015-09-10 ADDON-5469
Missing or improper default value for un-required fields.
2015-09-07 ADDON-5355
Different error message for same error when creating duplicated data inputs.
2015-09-06 ADDON-5354
Using keyboard to delete selections from configuration dropdown multi-select field causes drop-down list to appear in corner of screen.
2015-09-01 ADDON-5309
UI default value is not read from default input config file
2015-09-01 ADDON-5295
Description inconsistent in the GUI for CloudTrail service and CloudTrail from S3 service blacklist behavior.
2015-08-31 ADDON-5212
Chrome highlights "misspelling" of configuration text in the GUI.
2015-07-06 ADDON-6177
When tmp file system runs out of space, aws_billing.py fails with IOError: No space left on device.
2015-04-02 ADDON-3578
S3: uppercase bucket names cause an error
2015-03-25 ADDON-3460
On OSs (like Debian and Ubuntu) that use dash for shell scripts, aws_cloudwatch.py spawns zombie processes.
Workaround: Kill the processes and restart. Use bash to prevent re-occurrence.
2014-09-28 ADDON-2135
The list of regions shown in inputs configuration in Splunk Web shows all Amazon regions regardless of the permissions associated with the selected AWS account.
2014-09-26 ADDON-2118
Data inputs continue to work after user deletes the account used for that input.
Workaround: Restart the Splunk platform after deleting or modifying an AWS account.
2014-09-25 ADDON-2113
The app.conf file includes a stanza for a proxy server configuration with a hashed password even if the user has not configured a proxy or password.
Workaround: This behavior is expected because Splunk Enterprise automatically sets the proxy field to 0 and saves an encrypted entry in app.conf.
2014-09-16 ADDON-2029
In saved search "Monthly Cost till *" _time is displayed per day rather than per month.

Third-party software attributions

Version 3.0.0 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Version 2.0.1

Version 2.0.1 of the Splunk Add-on for Amazon Web Services has the same compatibility specifications as version 3.0.0.

Fixed issues

Version 2.0.1 of the Splunk Add-on for Amazon Web Services fixes the following issues.

Resolved date Defect number Description
2015-11-04 ADDON-5813 S3 input cannot handle bucket names with "." in them. See "Add an S3 input for the Splunk Add-on for AWS" for details of the solution.
2015-10-28 ADDON-6125 Add-on makes too many unnecessary get_log_event API calls, causing inefficiencies in environments with many spot instances.
2015-10-26 ADDON-5785 Corrupt VPC Flow checkpointer file in race condition.
2015-10-20 ADDON-5612 When CloudTrail userName is null, add-on coalesces the userName to "root" instead of "unknown".
2015-10-15 ADDON-6004 Add-on GUI does not allow user to select an index that is only defined on the indexers.
2015-10-11 ADDON-6003 Incorrect regions shown in region drop-down list.
2015-10-11 ADDON-6001 Config fails to fetch events from an S3 bucket in a different region.
2015-10-09 ADDON-5833 AWS CloudWatch log formatting exception.
2015-10-09 ADDON-4505 Cloudwatchlog deadlocks due to throttling exceptions when an input task includes a large number of log groups.
2015-10-09 ADDON-5782 A corrupted checkpointer file for VPC Flow blocks other logstreams.

Known issues

Version 2.0.1 of the Splunk Add-on for Amazon Web Services has the following known issues.

Date filed Defect number Description
2015-12-15 ADDON-7930 Data collection for Cloudwatch S3 metrics does not support wildcard in BucketName or array length > 1.
2015-11-09 ADDON-6371 In some cases, Splunk Cloud does not save the AWS account credentials after they are correctly entered. Workaround: File a support request to redeploy the add-on and restart the instance.
2015-10-14 ADDON-6056 S3 logging errors on Windows.
2015-10-13 ADDON-6043 SQS message mistakenly deleted when the add-on throws an error retrieving data from an S3 bucket.
2015-09-11 ADDON-5500 Preconfigured reports for billing data cannot handle reports that have a mix of different currencies. The report will use the first currency found and apply that to all costs.
2015-09-11 ADDON-5499 CloudWatch: Previous selected Metric namespace always exists in the list regardless of the region change.
2015-09-11 ADDON-5498 Unclear error message: Failed to load options for Metric namespace. Detailed Error: Unexpected error "<class 'socket.error'>" from python handler: "[Errno 111] Connection refused" when user specifies all regions in CloudWatch for one namespace, saves the configuration, and reloads it.
2015-09-10 ADDON-5481 The add-on configuration UI does not handle insufficient Splunk user permissions gracefully.
2015-09-10 ADDON-5474 No detailed error shown while getting S3 buckets via REST endpoint with wrong proxy or account settings.
2015-09-10 ADDON-5471 Deleting a CloudWatch data input takes too long.
2015-09-10 ADDON-5469 Missing or improper default value for un-required fields.
2015-09-10 ADDON-5491 The add-on configuration UI displays all regions instead of those within the selected account's permission scope.
2015-09-07 ADDON-5355 Different error message for same error when creating duplicated data inputs.
2015-09-06 ADDON-5354 Using keyboard to delete selections from configuration dropdown multi-select field causes drop-down list to appear in corner of screen.
2015-09-01 ADDON-5309 UI default value is not read from default input config file.
2015-09-01 ADDON-5295 Description inconsistent in the GUI for CloudTrail service and CloudTrail from S3 service blacklist behavior.
2015-08-31 ADDON-5212 Chrome highlights "misspelling" of configuration text in the GUI.
2015-07-09 ADDON-3460 /
CO-4749 /
SPL-55904
On OSs (like Debian and Ubuntu) that use dash for shell scripts, aws_cloudwatch.py spawns zombie processes. Workaround: Kill the processes and restart. Use bash to prevent re-occurrence.
2015-07-06 ADDON-6177 aws_billing.py fails with IOError: [Errno 28] No space left on device.
2015-04-03 ADDON-3578 Uppercase bucket name causes errors.
2015-01-22 ADDON-3050/
SPL-96729/
SPL-64904
S3 input is breaking lines incorrectly and inconsistently indexing only partial events.
Workaround: Disable the persistent queue for the S3 input by changing persistentQueueSize = 24MB to persistentQueueSize = 0 in local/inputs.conf.
2015-01-25 ADDON-3070 The add-on does not index the Configuration.State.Code change from SQS that is reported to users on the AWS Config UI. Splunk Enterprise only indexes configuration snapshots from S3 as new events, and only after a "ConfigurationHistoryDeliveryCompleted" notification is recieved by SQS.
2014-09-26 ADDON-2118 Data inputs continue to work after user deletes the account used for that input. Workaround: Restart Splunk Enterprise after deleting or modifying an AWS account.
2014-09-28 ADDON-2135 The list of regions shown in inputs configuration in Splunk Web shows all Amazon regions regardless of the permissions associated with the selected AWS account.
2014-09-26 ADDON-2116/
SPL-91709
On Windows 2012, Splunk Web shows a timeout error when a user attempts to add or delete an AWS account on the setup page. Workaround: Refresh the page.
2014-09-25 ADDON-2113 The app.conf file includes a stanza for a proxy server configuration with a hashed password even if the user has not configured a proxy or password. This behavior is expected because Splunk Enterprise automatically sets the proxy field to 0 and saves an encrypted entry in app.conf.
2014-09-16 ADDON-2029 In saved search "Monthly Cost till *" _time is displayed per day rather than per month.
2014-09-09 ADDON-1983 /
ADDON-1938 /
SPL-81771
Errors can occur in checkpointing if modular input stdout is prematurely closed during termination. Checkpoint and retry time do not log correctly when Splunkd stops.
2014-08-26 ADDON-1919 If a user changes the configuration to use a different AWS account, Splunk Web continues to list buckets for the previously configured account.
2014-08/17 ADDON-1854 After initial configuration, adjusting Max trackable items might cause data loss.
2014-08-14 ADDON-1827 Checkpoints are not cleared after data inputs are removed or the add-on is uninstalled, thus if you create a new input with the same name as the deleted one, the add-on uses the checkpoint from the old input. Workaround: create unique input names to avoid picking up old checkpoint files.

Third-party software attributions

Version 2.0.1 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Version 2.0.0

Version 2.0.0 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms.

Splunk platform versions 6.3, 6.2
CIM 4.0 and above
Platforms Platform independent
Vendor Products AWS CloudTrail, CloudWatch, CloudWatch Logs, Config, Billing, S3

New features

Version 2.0.0 of the Splunk Add-on for Amazon Web Services has the following new features.

Resolved date Defect number Description
2015-09-08 ADDON-1671 New configuration UI.
2015-09-08 ADDON-2126 / ADDON-5466 Ability to manually enter S3 bucket names, SQS queue names, and metric namespaces in Splunk Web fields, in case connection to AWS is poor or user account lacks permissions to list buckets.
2015-07-14 ADDON-4543 Added unified field for AWS account ID across all data inputs: aws_account_id.
2015-07-06 ADDON-3189 Currency field added to AWS billing report data, allowing users to more accurately judge financial impact.
2015-07-03 ADDON-4260 / ADDON-1665 Support for data ingestion from AWS CloudWatch Logs service, including VPC Flow Logs.
2015-07-03 ADDON-4259 CIM mapping for VPC Flow Logs data.
2015-06-30 ADDON-4158 Support for Config snapshot collection.
2015-06-29 ADDON-2364 Support for collecting archives of CloudTrail data via S3 buckets by configuring the sourcetype aws:cloudtrail in an S3 input.
2015-06-29 ADDON-4413 Support for multiple regions in a single CloudWatch input.
2015-06-29 ADDON-3235 Support for disabling SSL proxies using the is_secure parameter in local/aws_global_settings.conf to alter the behavior of connections to AWS.
2015-06-29 ADDON-4180 Support for inventory metadata collection from AWS.

Fixed issues

Version 2.0.0 of the Splunk Add-on for Amazon Web Services fixes the following issues.

Resolved date Defect number Description
2015-09-14 ADDON-5158 CloudTrail data missing some CIM tagging.
2015-08-31 ADDON-5200 CloudWatch input calls AWS API inefficiently, using separate API call for each instance-metric combination.
2015-08-31 ADDON-2006 Unfriendly error message when user specifies invalid account.
2015-08-31 ADDON-1932 Unfriendly error message when configuring proxy incorrectly.
2015-08-31 ADDON-1926 Splunk Web allows you to update and delete an AWS account for the add-on simultaneously.
2015-09-09 ADDON-4822 / CO-4912 Some instances of Splunk Cloud show blank screens for all data input pages. Workaround: Set up a heavy forwarder on-prem to handle data inputs and forward the data to Splunk Cloud.

Known issues

Version 2.0.0 of the Splunk Add-on for Amazon Web Services has the following known issues.

Date filed Defect number Description
2015-10-14 ADDON-6056 S3 logging errors on Windows.
2015-10-13 ADDON-6043 SQS message mistakenly deleted when the add-on throws an error retrieving data from an S3 bucket.
2015-10-09 ADDON-6004 Add-on GUI does not allow user to select an index that is only defined on the indexers.
2015-10-09 ADDON-6003 Incorrect regions shown in region drop-down list.
2015-10-09 ADDON-6001 Confg fails to fetch events from an S3 bucket in a different region.
2015-10-03 ADDON-5833 AWS CloudWatch log formatting exception.
2015-09-28 ADDON-5813 S3 input cannot handle bucket names with "." in them.
2015-09-24 ADDON-5785 Corrupt VPC Flow checkpointer file in race condition.
2015-09-24 ADDON-5782 A corrupted checkpointer file for VPC Flow blocks other logstreams.
2015-09-17 ADDON-5612 When CloudTrail userName is null, add-on coalesces the userName to "root" instead of "unknown".
2015-09-11 ADDON-5500 Preconfigured reports for billing data cannot handle reports that have a mix of different currencies. The report will use the first currency found and apply that to all costs.
2015-09-11 ADDON-5499 CloudWatch: Previous selected Metric namespace always exists in the list regardless of the region change.
2015-09-11 ADDON-5498 Unclear error message: Failed to load options for Metric namespace. Detailed Error: Unexpected error "<class 'socket.error'>" from python handler: "[Errno 111] Connection refused" when user specifies all regions in CloudWatch for one namespace, saves the configuration, and reloads it.
2015-09-10 ADDON-5481 The add-on configuration UI does not handle insufficient Splunk user permissions gracefully.
2015-09-10 ADDON-5491 The add-on configuration UI displays all regions instead of those within the selected account's permission scope.
2015-09-10 ADDON-5474 No detailed error shown while getting S3 buckets via REST endpoint with wrong proxy or account settings.
2015-09-10 ADDON-5471 Deleting a CloudWatch data input takes too long.
2015-09-10 ADDON-5469 Missing or improper default value for un-required fields.
2015-09-07 ADDON-5355 Different error message for same error when creating duplicated data inputs.
2015-09-06 ADDON-5354 Using keyboard to delete selections from configuration dropdown multi-select field causes drop-down list to appear in corner of screen.
2015-09-01 ADDON-5309 UI default value is not read from default input config file.
2015-09-01 ADDON-5295 Description inconsistent in the GUI for CloudTrail service and CloudTrail from S3 service blacklist behavior.
2015-08-31 ADDON-5212 Chrome highlights "misspelling" of configuration text in the GUI.
2015-07-10 ADDON-4505 Cloudwatchlog deadlocks due to throttling exceptions when an input task includes a large number of log groups.
2015-07-09 ADDON-3460 / CO-4749 / SPL-55904 On OSs (like Debian and Ubuntu) that use dash for shell scripts, aws_cloudwatch.py spawns zombie processes. Workaround: Kill the processes and restart. Use bash to prevent re-occurrence.
2015-07-06 ADDON-6177 aws_billing.py fails with IOError: [Errno 28] No space left on device.
2015-04-03 ADDON-3578 Uppercase bucket name causes errors.
2015-01-22 ADDON-3050/
SPL-96729/
SPL-64904
S3 input is breaking lines incorrectly and inconsistently indexing only partial events.
Workaround: Disable the persistent queue for the S3 input by changing persistentQueueSize = 24MB to persistentQueueSize = 0 in local/inputs.conf.
2015-01-25 ADDON-3070 The add-on does not index the Configuration.State.Code change from SQS that is reported to users on the AWS Config UI. Splunk Enterprise only indexes configuration snapshots from S3 as new events, and only after a "ConfigurationHistoryDeliveryCompleted" notification is recieved by SQS.
2014-09-26 ADDON-2118 Data inputs continue to work after user deletes the account used for that input. Workaround: Restart Splunk Enterprise after deleting or modifying an AWS account.
2014-09-28 ADDON-2135 The list of regions shown in inputs configuration in Splunk Web shows all Amazon regions regardless of the permissions associated with the selected AWS account.
2014-09-26 ADDON-2116/
SPL-91709
On Windows 2012, Splunk Web shows a timeout error when a user attempts to add or delete an AWS account on the setup page. Workaround: Refresh the page.
2014-09-25 ADDON-2113 The app.conf file includes a stanza for a proxy server configuration with a hashed password even if the user has not configured a proxy or password. This behavior is expected because Splunk Enterprise automatically sets the proxy field to 0 and saves an encrypted entry in app.conf.
2014-09-16 ADDON-2029 In saved search "Monthly Cost till *" _time is displayed per day rather than per month.
2014-09-09 ADDON-1983 / ADDON-1938 / SPL-81771 Errors can occur in checkpointing if modular input stdout is prematurely closed during termination. Checkpoint and retry time do not log correctly when Splunkd stops.
2014-08-26 ADDON-1919 If a user changes the configuration to use a different AWS account, Splunk Web continues to list buckets for the previously configured account.
2014-08/17 ADDON-1854 After initial configuration, adjusting Max trackable items might cause data loss.
2014-08-14 ADDON-1827 Checkpoints are not cleared after data inputs are removed or the add-on is uninstalled, thus if you create a new input with the same name as the deleted one, the add-on uses the checkpoint from the old input. Workaround: create unique input names to avoid picking up old checkpoint files.

Third-party software attributions

Version 2.0.0 of the Splunk Add-on for Amazon Web Services incorporates the following third-party libraries.

Version 1.1.1

Version 1.1.1 of the Splunk Add-on for Amazon Web Services is compatible with the following software, CIM versions, and platforms.

Splunk Enterprise versions 6.2, 6.1
CIM 4.2, 4.1, 4.0
Platforms Platform independent
Vendor Products AWS Billing, CloudTrail, CloudWatch, Config, S3

Fixed issues

Version 1.1.1 of the Splunk Add-on for Amazon Web Services fixes the following issues.

Resolved date Defect number Description
04/24/15 ADDON-3512 Timeout error on new account definition. Users can now set splunkdConnectionTimeout = 3000 in $SPLUNK_HOME/etc/system/local/web.conf to avoid setup timeout problems.
04/21/15 ADDON-3612 Add-on cannot parse multi-account message format from SQS and CloudTrail.
04/21/15 ADDON-3577 Input configuration timeout on retrieving bucket/key list from S3.
03/01/15 ADDON-3119 Add-on fails to collect payloads from GovCloud region.

Known issues

Version 1.1.1 of the Splunk Add-on for Amazon Web Services has the following known issues.

Date Defect number Description
08/27/15 ADDON-5158 CloudTrail data missing some CIM tagging.
08/06/15 ADDON-4822 / CO-4581 Some instances of Splunk Cloud show blank screens for all data input pages. Workaround: Set up a heavy forwarder on-prem to handle data inputs and forward the data to Splunk Cloud.
04/10/15 ADDON-3652 Billing reports are not performant.
04/03/15 ADDON-3578 Uppercase bucket name causes errors.
01/22/15 ADDON-3050/
SPL-96729/
SPL-64904
S3 input is breaking lines incorrectly and inconsistently indexing only partial events.
Workaround: Disable the persistent queue for the S3 input by changing persistentQueueSize = 24MB to persistentQueueSize = 0 in local/inputs.conf.
01/25/15 ADDON-3070 The add-on does not index the Configuration.State.Code change from SQS that is reported to users on the AWS Config UI. Splunk Enterprise only indexes configuration snapshots from S3 as new events, and only after a "ConfigurationHistoryDeliveryCompleted" notification is recieved by SQS.
01/06/15 ADDON-2910 Splunk Cloud customers cannot access props.conf to configure line breaking on S3 events.
10/10/14 ADDON-2154 Billing input data has a non-ISO-8601 timestamp appended to the source field of each event. Workaround: Add a new field named "source2" in the suggested format:

..... | rex field=source "(?<source_file>s3://[^:])" | rex field=source "(?<source_date>(csv|zip):.$)" | eval source2=strftime(strptime(substr(source_date, 9),"%d %b %Y %H:%M:%S"),"%Y-%m-%dT%H:%M:%S%z") | eval source2=source_file+":"+source2.

09/26/14 ADDON-2118 Data inputs continue to work after user deletes the account used for that input. Workaround: Restart Splunk Enterprise after deleting or modifying an AWS account.
09/28/14 ADDON-2135 The list of regions shown in inputs configuration in Splunk Web shows all Amazon regions regardless of the permissions associated with the selected AWS account.
09/26/14 ADDON-2116/
SPL-86716
On Windows 2012, Splunk Web shows a timeout error when a user attempts to add or delete an AWS account on the setup page. Workaround: Refresh the page.
09/26/14 ADDON-2115 If user does not provide a friendly name when configuring an AWS account in the setup screen, account is not configured but no error message appears
09/25/14 ADDON-2113 The app.conf file includes a stanza for a proxy server configuration with a hashed password even if the user has not configured a proxy or password. This behavior is expected because Splunk Enterprise automatically sets the proxy field to 0 and saves an encrypted entry in app.conf.
09/25/14 ADDON-2110 In Splunk 6.2, when network is unstable, some input configuration fields fail to display in Splunk Web and no error message is shown.
09/16/14 ADDON-2029 In saved search "Monthly Cost till *" _time is displayed per day rather than per month.
09/11/14 ADDON-2006 Unfriendly error message when user specifies invalid account.
09/09/14 ADDON-1983 If Splunk Enterprise restarts while indexing S3 data, data duplication might occur. Workaround: Use AWS command line tools.
08/28/14 ADDON-1938 Checkpoint and retry time do not log correctly when Splunkd stops.
08/28/14 ADDON-1932 Unfriendly error message when configuring proxy incorrectly.
08/26/14 ADDON-1926 Splunk Web allows you to update and delete an AWS account for the add-on simultaneously.
08/26/14 ADDON-1919 If a user changes the configuration to use a different AWS account, Splunk Web continues to list buckets for the previously configured account.
08/24/14 ADDON-1895 If user tries to update a billing report manually using Microsoft Excel, the add-on cannot process the modified file and throws "failed to parse key" error.
08/21/14 ADDON-1885 Splunk Enterprise does not validate Amazon Web Services credentials during add-on setup.
08/17/14 ADDON-1854 After initial configuration, adjusting Max trackable items might cause data loss.
08/14/14 ADDON-1827 Checkpoints are not cleared after data inputs are removed or the add-on is uninstalled, thus if you create a new input with the same name as the deleted one, the add-on uses the checkpoint from the old input. Workaround: create unique input names to avoid picking up old checkpoint files.
03/12/14 SPL-81771 Errors can occur in checkpointing if modular input stdout is prematurely closed during termination.

Third-party software attributions

Version 1.1.1 of the Splunk Add-on for Amazon Web Services incorporates boto - AWS for Python.


Version 1.1.0

Version 1.1.0 had the same compatibility specifications as Version 1.1.1.

New features

Version 1.1.0 of the Splunk Add-on for Amazon Web Services has the following new features.

Date Issue number Description
02/12/15 ADDON-3148 Support for the SNS Subscription attributes for Raw Message Delivery for AWS Config and CloudTrail.
02/09/15 ADDON-1644 Pre-built panels for CloudWatch, CloudTrail, and Billing data.
12/18/14 ADDON-2678 Allow users to configure the log level.
11/12/14 ADDON-2202 New modular input for AWS Config data.

Fixed issues

Version 1.1.0 of the Splunk Add-on for Amazon Web Services fixes the following issues.

Resolved date Defect number Description
02/11/15 ADDON-2533 Internal logs are source typed as "this-too-small".
02/10/15 ADDON-2679 Process for fetching logs runs in a loop.
02/09/15 ADDON-3154 Support AssumedRole user name for CloudTrail.

Known issues

Version 1.1.0 of the Splunk Add-on for Amazon Web Services has the following known issues.

Date Defect number Description
01/22/15 ADDON-3050 S3 input is breaking lines incorrectly.
01/25/15 ADDON-3070 The add-on does not index the Configuration.State.Code change from SQS that is reported to users on the AWS Config UI. Splunk Enterprise only indexes configuration snapshots from S3 as new events, and only after a "ConfigurationHistoryDeliveryCompleted" notification is recieved by SQS.
01/06/15 ADDON-2910 Splunk Cloud customers cannot access props.conf to configure line breaking on S3 events.
09/28/14 ADDON-2135 The list of regions shown in inputs configuration in Splunk Web shows all Amazon regions regardless of the permissions associated with the selected AWS account.
09/26/14 ADDON-2116 On Windows 2012, Splunk Web shows a timeout error when a user attempts to add or delete an AWS account on the setup page. Workaround: Refresh the page.
09/26/14 ADDON-2115 If user does not provide a friendly name when configuring an AWS account in the setup screen, account is not configured but no error message appears
09/25/14 ADDON-2113 The app.conf file includes a stanza for a proxy server configuration with a hashed password even if the user has not configured a proxy or password. This behavior is expected because Splunk Enterprise automatically sets the proxy field to 0 and saves an encrypted entry in app.conf.
09/25/14 ADDON-2110 In Splunk 6.2, when network is unstable, some input configuration fields fail to display in Splunk Web and no error message is shown.
09/16/14 ADDON-2029 In saved search "Monthly Cost till *" _time is displayed per day rather than per month.
09/11/14 ADDON-2006 Unfriendly error message when user specifies invalid account.
09/09/14 ADDON-1983 If Splunk Enterprise restarts while indexing S3 data, data duplication might occur. Workaround: Use AWS command line tools.
08/28/14 ADDON-1938 Checkpoint and retry time do not log correctly when Splunkd stops.
08/28/14 ADDON-1932 Unfriendly error message when configuring proxy incorrectly.
08/26/14 ADDON-1926 Splunk Web allows you to update and delete an AWS account for the add-on simultaneously.
08/26/14 ADDON-1919 If a user changes the configuration to use a different AWS account, Splunk Web continues to list buckets for the previously configured account.
08/24/14 ADDON-1895 If user tries to update a billing report manually using Microsoft Excel, the add-on cannot process the modified file and throws "failed to parse key" error.
08/21/14 ADDON-1885 Splunk Enterprise does not validate Amazon Web Services credentials during add-on setup.
08/17/14 ADDON-1854 After initial configuration, adjusting Max trackable items might cause data loss.
08/14/14 ADDON-1827 Checkpoints are not cleared after data inputs are removed or the add-on is uninstalled, thus if you create a new input with the same name as the deleted one, the add-on uses the checkpoint from the old input. Workaround: create unique input names to avoid picking up old checkpoint files.
03/12/14 SPL-81771 Errors can occur in checkpointing if modular input stdout is prematurely closed during termination.

Third-party software attributions

Version 1.1.0 of the Splunk Add-on for Amazon Web Services incorporates boto - AWS for Python.


Version 1.0.1

Version 1.0.1 of the Splunk Add-on for Amazon Web Services was compatible with the following software, CIM versions, and platforms.

Splunk Enterprise versions 6.2, 6.1
CIM 4.1, 4.0, 3.0
Platforms Platform independent
Vendor Products AWS Billing, CloudTrail, CloudWatch, S3

Fixed issues

Version 1.0.1 of the Splunk Add-on for Amazon Web Services fixed the following issues.

Resolved date Defect number Description
12/16/14 ADDON-2530 New version of boto library required to support eu-central-1 region.
12/11/14 ADDON-2359 Unexpected SQS messages can block inputs.

Known issues

Version 1.0.1 of the Splunk Add-on for Amazon Web Services has the following known issues.

  • Internal log files are incorrectly sourcetyped as N-too-small. (ADDON-2533)
  • Errors can occur in checkpointing if modular input stdout is prematurely closed during termination. (SPL-81771)
  • After initial configuration, adjusting Max trackable items might cause data loss. (ADDON-1854)
  • Splunk Enterprise does not validate Amazon Web Services credentials during add-on setup. (ADDON-1885)
  • If user tries to update a billing report manually using Microsoft Excel, the add-on cannot process the modified file and throws "failed to parse key" error. (ADDON-1895)
  • If a user changes the configuration to use a different AWS account, Splunk Web continues to list buckets for the previously configured account. (ADDON-1919)
  • Splunk Web allows you to update and delete an AWS account for the add-on simultaneously. (ADDON-1926)
  • Setup and configuration pages in Splunk Web give unfriendly error messages when given invalid inputs (ADDON-1932, ADDON-2006)
  • If Splunk Enterprise restarts while indexing S3 data, data duplication might occur. Workaround: Use AWS command line tools. (ADDON-1983 and ADDON-1938)
  • In saved search "Monthly Cost till *" _time is displayed per day rather than per month. (ADDON-2029)
  • The app.conf file includes a stanza for a proxy server configuration with a hashed password even if the user has not configured a proxy or password. This behavior is expected because Splunk Enterprise automatically sets the proxy field to 0 and saves an encrypted entry in app.conf. (ADDON-2113)
  • If user does not provide a friendly name when configuring an AWS account in the setup screen, account is not configured but no error message appears (ADDON-2115)
  • On Windows 2012, Splunk Web shows a timeout error when a user attempts to add or delete an AWS account on the setup page. Workaround: Refresh the page. (ADDON-2116)
  • The list of regions shown in inputs configuration in Splunk Web shows all Amazon regions regardless of the permissions associated with the selected AWS account. (ADDON-2135)
  • In Splunk 6.2, when network is unstable, some input configuration fields fail to display in Splunk Web and no error message is shown. (ADDON-2110)

Third-party software attributions

Version 1.0.1 of the Splunk Add-on for Amazon Web Services incorporated boto - AWS for Python.

Version 1.0.0

Version 1.0.0 of the Splunk Add-on for Amazon Web Services had the same compatibility specifications as version 1.0.1.

Known issues

Version 1.0.0 of the Splunk Add-on for Amazon Web Services had the following known issues:

  • Errors can occur in checkpointing if modular input stdout is prematurely closed during termination. (SPL-81771)
  • After initial configuration, adjusting Max trackable items might cause data loss. (ADDON-1854)
  • Splunk Enterprise does not validate Amazon Web Services credentials during add-on setup. (ADDON-1885)
  • If user tries to update a billing report manually using Microsoft Excel, the add-on cannot process the modified file and throws "failed to parse key" error. (ADDON-1895)
  • If a user changes the configuration to use a different AWS account, Splunk Web continues to list buckets for the previously configured account. (ADDON-1919)
  • Splunk Web allows you to update and delete an AWS account for the add-on simultaneously. (ADDON-1926)
  • Setup and configuration pages in Splunk Web give unfriendly error messages when given invalid inputs (ADDON-1932, ADDON-2006)
  • If Splunk Enterprise restarts while indexing S3 data, data duplication might occur. Workaround: Use AWS command line tools. (ADDON-1983 and ADDON-1938)
  • In saved search "Monthly Cost till *" _time is displayed per day rather than per month. (ADDON-2029)
  • The app.conf file includes a stanza for a proxy server configuration with a hashed password even if the user has not configured a proxy or password. This behavior is expected because Splunk Enterprise automatically sets the proxy field to 0 and saves an encrypted entry in app.conf. (ADDON-2113)
  • If user does not provide a friendly name when configuring an AWS account in the setup screen, account is not configured but no error message appears (ADDON-2115)
  • On Windows 2012, Splunk Web shows a timeout error when a user attempts to add or delete an AWS account on the setup page. Workaround: Refresh the page. (ADDON-2116)
  • The list of regions shown in inputs configuration in Splunk Web shows all Amazon regions regardless of the permissions associated with the selected AWS account. (ADDON-2135)
  • In Splunk 6.2, when network is unstable, some input configuration fields fail to display in Splunk Web and no error message is shown. (ADDON-2110)

Third-party software attributions

Version 1.0.0 of the Splunk Add-on for Amazon Web Services incorporated boto - AWS for Python.

Last modified on 12 March, 2024
PREVIOUS
Release notes for the Splunk Add-on for AWS
 

This documentation applies to the following versions of Splunk® Supported Add-ons: released


Was this documentation topic helpful?


You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters