Docs » Instrument serverless functions for Splunk Observability Cloud » Instrument AWS Lambda functions for Splunk Observability Cloud » Instrument your AWS Lambda function for Splunk Observability Cloud » Instrument your AWS Lambda function using a Collector gateway in EC2

Instrument your AWS Lambda function using a Collector gateway in EC2 🔗

You can automatically instrument your AWS Lambda function for Splunk Observability Cloud using a layer for your runtime and a Collector gateway in EC2. Using a Collector gateway helps aggregate data and reduce load.

Follow these instructions to install the Splunk OpenTelemetry Lambda layer for Java, Python, or Node.js functions. See Instrument your AWS Lambda function for Splunk Observability Cloud for the all-in-one default deployment instructions.

Check compatibility and requirements 🔗

The Splunk OpenTelemetry Lambda layer supports the following runtimes in AWS Lambda:

  • Java 8.al2, 11, 17, and 21

  • Python 3.8 and 3.9

  • Node.js 14 and higher

  • Go 1.20

The AWS Lambda layer requires 90 MB on-disk in standard x86_64 systems.

For more information, search for Lambda runtimes and other system requirements on the AWS documentation website.

Install the AWS Lambda layer for your language 🔗

Follow these steps to instrument your function using the Splunk OpenTelemetry Lambda layer:

  1. In the AWS Lambda console, select the function that you want to instrument.

  2. In the Layers section, select Add a layer, then select Specify an ARN.

  3. Copy the Amazon Resource Name (ARN) for the region of your AWS Lambda function from the list that matches your architecture:

    <h3>Splunk OpenTelemetry Lambda Java Layer</h3>

<pre>
arn:aws:lambda:us-east-2:254067382080:layer:splunk-apm-java:16
arn:aws:lambda:us-east-1:254067382080:layer:splunk-apm-java:3
arn:aws:lambda:eu-central-1:254067382080:layer:splunk-apm-java:3
arn:aws:lambda:us-west-1:254067382080:layer:splunk-apm-java:3
arn:aws:lambda:us-west-2:254067382080:layer:splunk-apm-java:3
arn:aws:lambda:ap-south-1:254067382080:layer:splunk-apm-java:3
arn:aws:lambda:ap-northeast-1:254067382080:layer:splunk-apm-java:3
arn:aws:lambda:ap-northeast-2:254067382080:layer:splunk-apm-java:3
arn:aws:lambda:ap-southeast-1:254067382080:layer:splunk-apm-java:3
arn:aws:lambda:ap-southeast-2:254067382080:layer:splunk-apm-java:3
arn:aws:lambda:ca-central-1:254067382080:layer:splunk-apm-java:3
arn:aws:lambda:eu-west-1:254067382080:layer:splunk-apm-java:3
arn:aws:lambda:eu-west-2:254067382080:layer:splunk-apm-java:3
arn:aws:lambda:eu-west-3:254067382080:layer:splunk-apm-java:3
arn:aws:lambda:eu-north-1:254067382080:layer:splunk-apm-java:3
arn:aws:lambda:sa-east-1:254067382080:layer:splunk-apm-java:3
arn:aws:lambda:eu-south-1:254067382080:layer:splunk-apm-java:3
arn:aws:lambda:ap-northeast-3:254067382080:layer:splunk-apm-java:3
arn:aws:lambda:ap-east-1:254067382080:layer:splunk-apm-java:3
arn:aws:lambda:af-south-1:254067382080:layer:splunk-apm-java:3
arn:aws:lambda:me-south-1:254067382080:layer:splunk-apm-java:3
</pre>

  4. Paste the selected ARN in the Specify an ARN field and select Add.

  5. Check that the Splunk layer appears in the Layers table.

  6. (Optional) Repeat the previous steps to install AWS Lambda metrics extension layer for Splunk Infrastructure Monitoring.

    <h3>Splunk Metrics Extension Layer</h3>

<pre>
arn:aws:lambda:us-east-2:254067382080:layer:splunk-lambda-metrics:16
arn:aws:lambda:us-east-1:254067382080:layer:splunk-lambda-metrics:3
arn:aws:lambda:eu-central-1:254067382080:layer:splunk-lambda-metrics:3
arn:aws:lambda:us-west-1:254067382080:layer:splunk-lambda-metrics:3
arn:aws:lambda:us-west-2:254067382080:layer:splunk-lambda-metrics:3
arn:aws:lambda:ap-south-1:254067382080:layer:splunk-lambda-metrics:3
arn:aws:lambda:ap-northeast-1:254067382080:layer:splunk-lambda-metrics:3
arn:aws:lambda:ap-northeast-2:254067382080:layer:splunk-lambda-metrics:3
arn:aws:lambda:ap-southeast-1:254067382080:layer:splunk-lambda-metrics:3
arn:aws:lambda:ap-southeast-2:254067382080:layer:splunk-lambda-metrics:3
arn:aws:lambda:ca-central-1:254067382080:layer:splunk-lambda-metrics:3
arn:aws:lambda:eu-west-1:254067382080:layer:splunk-lambda-metrics:3
arn:aws:lambda:eu-west-2:254067382080:layer:splunk-lambda-metrics:3
arn:aws:lambda:eu-west-3:254067382080:layer:splunk-lambda-metrics:3
arn:aws:lambda:eu-north-1:254067382080:layer:splunk-lambda-metrics:3
arn:aws:lambda:sa-east-1:254067382080:layer:splunk-lambda-metrics:3
arn:aws:lambda:eu-south-1:254067382080:layer:splunk-lambda-metrics:3
arn:aws:lambda:ap-northeast-3:254067382080:layer:splunk-lambda-metrics:3
arn:aws:lambda:ap-east-1:254067382080:layer:splunk-lambda-metrics:3
arn:aws:lambda:af-south-1:254067382080:layer:splunk-lambda-metrics:3
arn:aws:lambda:me-south-1:254067382080:layer:splunk-lambda-metrics:3
</pre>

Note

You can automate the update of the Lambda layers using the AWS CLI or other automation tools.

Configure the Splunk OpenTelemetry Lambda layer 🔗

Follow these steps to add the required configuration for the Splunk OpenTelemetry Lambda layer:

  1. In the AWS Lambda console, open the function that you are instrumenting.

  2. Navigate to Configuration, then Environment variables.

  3. Select Edit.

  4. Add each of the following environment variables by selecting Add environment variable:

    Environment variable

    Description

    SPLUNK_REALM

    To find your Splunk realm, see Note about realms.

    SPLUNK_ACCESS_TOKEN

    A Splunk authentication token that lets exporters send data directly to Splunk Observability Cloud. See Authentication token. To send data to a Splunk OTel Collector in EC2, see Set up a Collector gateway in EC2.

    AWS_LAMBDA_EXEC_WRAPPER

    Set the value for the environment variable given your programming language:

    # Select the most appropriate value

# Wraps regular handlers that implement RequestHandler
/opt/otel-handler

# Same as otel-handler, but proxied through API Gateway,
# with HTTP context propagation activated
/opt/otel-proxy-handler

# Wraps streaming handlers that implement RequestStreamHandler
/opt/otel-stream-handler

    Note

    Only AWS SDK v2 instrumentation is activated by default. To instrument other libraries, modify your code to include the corresponding library instrumentation from the OpenTelemetry Java SDK.

    OTEL_SERVICE_NAME

    The name of your service.

    (Optional) OTEL_RESOURCE_ATTRIBUTES

    Define the name of the deployment environment of your function by setting this environment variable to deployment.environment=<name-of-your-environment>.

  5. Select Save and check that the environment variables appear in the table.

To configure the mode of metric ingest, see Metrics configuration.

Note

By default, the layer sends telemetry to a Collector instance on localhost using the Collector layer. If you don’t want to use a local Collector, you must specify the address of a Collector in data forwarding mode. See Set up a Collector gateway in EC2.

Set up a Collector gateway in EC2 🔗

To send spans from the instrumented AWS Lambda function to the Collector gateway running in EC2, do the following:

  1. Deploy the Collector in gateway mode in a service your function can reach, for example EC2. See Data forwarding (gateway) mode.

  2. Make sure that the AWS Lambda functions you want to instrument can reach the Collector gateway in EC2 and are in the same VPC.

  3. Navigate to Configuration, then Environment variables, then select Edit.

  4. Add the following environment variables to your instrumented functions:

    • OTEL_EXPORTER_OTLP_TRACES_ENDPOINT with the address of the EC2 instance that runs the gateway, for example 10.0.0.123:4317

    • OTEL_TRACES_EXPORTER with the value otlp

    • OTLP_EXPORTER_OTLP_TRACES_PROTOCOL with the value http/protobuf

    • SPLUNK_LAMBDA_LOCAL_COLLECTOR_ENABLED with the value false

  5. If you’ve already set the access token and realm in the Collector configuration, delete the SPLUNK_ACCESS_TOKEN and SPLUNK_REALM environment variables.

Check that data appears in Splunk Observability Cloud 🔗

Each time the AWS Lambda function runs, trace and metric data appears in Splunk Observability Cloud. If no data appears, see Troubleshoot the Splunk OpenTelemetry Lambda layer.