Splunk® App for Data Science and Deep Learning

Use the Splunk App for Data Science and Deep Learning

Set up the Splunk App for Data Science and Deep Learning using AWS and EKS

Amazon Web Services (AWS) Elastic Kubernetes Service (EKS) is a frequently used managed service for running container workloads in the Cloud. Splunk App for Data Science and Deep Learning (DSDL) customers can use this managed service on AWS, connect DSDL to EKS, and run workloads there.

Prerequisites

The following is required for a successful set up:

  • An AWS account with sufficient administrator permissions
  • DSDL app installed and configured on your Splunk platform instance

Set up steps

Complete these steps to use DSDL with AWS:

Create a new IAM user and role

AWS Identity and Access Management (IAM) is an Amazon web service that helps you securely control access to any AWS resources. IAM enables shared access to your AWS account and resources while protecting your AWS account root-user information.

  1. In your AWS account, create a new user. Create a user name that identifies it as DSDL related, for example, dltk-admin.
  2. For the AWS Credential type, select Access key - programmatic access.
  3. For permissions, choose Add user to group and Create group. For the permission boundary, choose "Create user without a permission boundary".
  4. Create a name for the new group that identifies it as DSDL related, for example, dltk-admin-group. Select "AdministratorAccess" from the policy menu.
  5. Click Create group.
  6. Add the user you created in step 1 to the group you created in step 3. When complete, keep the confirmation page open in order to use the Access Key ID and Secret Access Key in the next section.

Install and configure the AWS CLI

Download and configure the AWS command line interface using the Access Key ID and Secret Access Key login credentials from the previous step. For more information, see https://aws.amazon.com/cli/

Install the eksctl command line utility

You can use Amazon Elastic Kubernetes Service (Amazon EKS) to run Kubernetes on AWS without installing or maintaining your own Kubernetes control plane or nodes. The eksctl command line tool is used for creating and managing Kubernetes clusters on Amazon EKS.

For more information,, see https://docs.aws.amazon.com/eks/latest/userguide/eksctl.html and https://eksctl.io/introduction/#installation

Create an Amazon EKS cluster

In Amazon EKS, create a cluster. You can refer to the following example for guidance:

eksctl create cluster --name dltk-cluster --version 1.21 --region eu-central-1 --nodegroup-name dltk-nodes --node-type t2.xlarge --nodes 2

This image shows an example cluster in Amazon EKS.

Add the cluster to Amazon EKS and verify the cluster is up and running.

This image shows an example cluster in Amazon EKS.

Set up an EFS CSI driver in EKS

Verify you can use the Storage Class of efs-sc and set up and configure the Amazon Elastic File System (EFS) Container Storage Interface (CSI) driver.

This image shows an example cluster in Amazon EKS.

For more information, see https://docs.aws.amazon.com/eks/latest/userguide/efs-csi.html

Connect DSDL to the AWS EKS cluster

Complete the following steps:

  1. In DSDL, go to Configuration > Setup and scroll down to the input panel for Kubernetes. Fill in the fields to connect DSDL to the AWS EKS cluster, using the cluster name created in a previous step.
    This image shows the Setup page of the DSDL app. Specifically, the fields to set up Kubernetes. Field names include EKS Cluster Name and AWS Access Key ID.

    On Splunk Cloud the inbound and outbound traffic to reach the cluster and its endpoints must be enabled. For example, using ACS. For more information, see Splunk Cloud Self-Service: Announcing The New Admin Config Service API.

  2. From Configuration > Containers , launch a development (DEV) container and verify the Kubernetes container is running.
    The image shows the Containers tab of the DSDL app.

  3. In AWS EKS, verify that the DEV deployment exists.
    This image shows a view within AWS EKS. A series of folders shows under Clusters.

  4. If you are setting up DSDL for the first time, verify that one of the Examples is running without any errors. Running an example after the container environment setup is a simple way to confirm that an algorithm can run.
    This image shows the Examples tab of the DSDL app.

(Optional) Configure Observability

If you want to use Splunk Observability Cloud (O11y) to monitor the Kubernetes that DSDL uses take steps to configure observability. To learn more about this product offering, see Splunk Observability.

Complete the following steps:

  1. In Splunk Observability Cloud configure a new integration as shown in the following image. This image shows the Configure Integration page in the Splunk Observability product.
    Complete the Configure Integration fields as follows:
    Field name Value
    Splunk Access Token Default
    Cluster Name Enter the name of the cluster you created in an earlier step
    Provider Amazon Web Services
    Distribution Amazon EKS
    Add Gateway No
    Log Collection True
  2. Next, follow the Install Integration steps as shown in the following image. This image shows the Install Inttegration page in the Splunk Observability product.
  3. Lastly, look over the Review Inventory page for the cluster added as shown in the following image. Pay attention to CPU utilization on the cluster when running a DSDL container workload.
    This image shows the Review Inventory page in the Splunk Observability product.
Last modified on 11 July, 2024
Splunk App for Data Science and Deep Learning certificate settings and JupyterLab password   Leverage provided examples of the Splunk App for Data Science and Deep Learning

This documentation applies to the following versions of Splunk® App for Data Science and Deep Learning: 5.1.1, 5.1.2


Was this topic useful?







You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters