Create the Amazon Security Lake subscriber for data ingestion

This topic covers the first part of the Create subscribers step of the workflow for creating an Amazon Security Lake federated provider. Before you attempt this step, you must first complete the Provider basics step. See Begin defining an Amazon Security Lake federated provider.

To give Federated Analytics the capacity to ingest data from your Amazon Security Lake account, you must define an Amazon Security Lake subscriber that provides data ingestion access.

The Amazon Security Lake subscriber for data ingestion allows Federated Analytics to directly access the S3 objects in your Amazon Security Lake account and ingest them into data lake indexes on your Splunk Cloud Platform deployment. In addition, the Amazon Security Lake subscriber for data ingestion notifies Federated Analytics when new S3 objects are available.

Setting up an Amazon Security Lake subscriber to grant data ingestion access to your Federated Analytics federated provider requires that you do the following things:

• First, create an Amazon Security Lake subscriber for data ingestion. As you do so, link your Federated Analytics federated provider to that Amazon Security Lake subscriber.
  • In Amazon Security Lake, open a form for a new subscriber for S3 data ingestion.
  • In Federated Analytics, copy the values of the Splunk's AWS account and External ID fields from the Create an Amazon Security Lake Subscriber for data ingest section of the Create subscribers page into corresponding fields on the Amazon Security Lake subscriber form.
  • Create the Amazon Security Lake subscriber for S3 data ingestion. Give the subscriber an S3 notification type of SQS queue.
• Second, link the Amazon Security Lake subscriber for data ingestion to your federated provider.
  • In Amazon Security Lake, copy the values of the AWS Role ARN and the Subscription endpoint fields in the detail page of the subscriber that you just created.
  • In Federated Analytics, paste those values into the corresponding AWS Role ARN and Subscription endpoint fields on the Create subscribers page of your federated provider definition.

Prerequisites

• You must have an AWS account with Amazon Security Lake activated. Your Amazon Security Lake must be in the same AWS Region as your Splunk Cloud Platform deployment.

To give an Amazon Security Lake subscriber for data ingest access to source data from multiple regions, you can specify the Region where you create the subscriber as a rollup Region and have other AWS Regions contribute data to it. Your Splunk Cloud Platform deployment must belong to the rollup Region.

• Verify that your AWS role has the necessary IAM policies and permissions for subscriber creation. Contact your AWS administrator if you need assistance.

For more information about rollup Regions, see Managing Regions in the Amazon Security Lake User Guide.

For more information about managing AWS roles and permissions, contact your AWS administrator, or see Managing data access for Security Lake subscribers in the Amazon Security Lake User Guide.

Steps

1. On your Splunk Cloud Platform deployment, in Splunk Web, at the Create subscribers page of the Add a new federated provider workflow, note the value of the AWS region field. This is the AWS Region of your Splunk Cloud Platform deployment.
2. In the Amazon Security Lake console, use the AWS Region drop-down in the upper-right corner of the page to select the AWS Region to which your Splunk Cloud Platform deployment belongs.

   If your Splunk Cloud Platform deployment's AWS Region contributes to a rollup Region, and you want to be able to ingest data from the regions represented by the rollup region, select that rollup Region.

3. Follow the instructions at Creating a subscriber with data access in Security Lake in the Amazon Security Lake User Guide to create an Amazon Security Lake subscriber for data ingest.
   
   In the Amazon Security Lake console, in the Subscriber credentials section of the Create subscriber form, copy and paste in two values from the Create an Amazon Security Lake subscriber for data ingest section of the Create subscribers page in the Add a new federated provider workflow in Splunk Web. Select the copy icon () for each field to ensure an accurate copy and paste operation.
   • Copy the value for the federated provider's Splunk's AWS account field. Paste that value into the Account ID field for the Amazon Security Lake subscriber for data ingest.
   • Copy the value for the federated provider's External ID field. Paste that value into the External ID field for the Amazon Security Lake subscriber for data ingest.
   
   

   The External ID for the Amazon Security Lake subscriber for data ingest is different from the External ID for the Amazon Security Lake Subscriber for for federated search access. Do not try to use the federated search subscriber External ID for the data ingest subscriber.

   In the Security Lake console, when you define the Amazon Security Lake subscriber for data ingest, make sure you give the subscriber an S3 notification type of SQS queue. Contrary to the Amazon Security Lake documentation, this selection is not optional when you are creating this kind of subscriber for an Amazon Security Lake federated provider.

   When you create an Amazon Security Lake subscriber for data ingest in the Amazon Security Lake console, you are restricted to 10 data sources. If you select Specific log and event sources and identify more than 10 sources, you will receive an error message when you try to create the subscriber.
4. After you create the Amazon Security Lake subscriber for data ingest, open its detail page by selecting its name in the My subscribers list.
5. In the Create subscribers page of the Add a new federated provider workflow in Splunk Web, copy and paste in two values from the detail page for the Amazon Security Lake subscriber for data ingest. Select the copy icon () where it is available to ensure an accurate copy and paste operation.
   • Copy the value of the AWS Role ARN field for the Amazon Security Lake subscriber for data ingest. Paste that value into the federated provider's AWS Role ARN field.
   • Copy the value for the Subscription endpoint field for the Amazon Security Lake subscriber for data ingest. Paste that value into the federated provider's Subscription endpoint field.
6. Go on to create the Amazon Security Lake subscriber for federated search access to complete the second part of the Create subscribers step.