Splunk® Data Stream Processor

Use the Data Stream Processor

On April 3, 2023, Splunk Data Stream Processor reached its end of sale, and will reach its end of life on February 28, 2025. If you are an existing DSP customer, please reach out to your account team for more information.

All DSP releases prior to DSP 1.4.0 use Gravity, a Kubernetes orchestrator, which has been announced end-of-life. We have replaced Gravity with an alternative component in DSP 1.4.0. Therefore, we will no longer provide support for versions of DSP prior to DSP 1.4.0 after July 1, 2023. We advise all of our customers to upgrade to DSP 1.4.0 in order to continue to receive full product support from Splunk.

Masking sensitive data in the

The allows you to mask or obfuscate sensitive data before sending your data to a destination. This means that you can make sure that private data stays private, and you can mask sensitive data according to your compliance standards and needs.

To mask data, use the replace scalar function. In this example, we'll use replace with the eval function.

Masking sensitive data with replace

The exact steps for masking your sensitive data vary depending on the specific data that you are working with. The replace function takes string as input, so if you want to mask data from a field that is not a string, you must cast that field to be a string first. This is demonstrated in the example. You can confirm the data types of your fields by selecting a function in your pipeline, then selecting the View Configurations tab, and then checking the list of fields under Input Fields.

Example: masking phone numbers with the replace function

In this example, we'll use the replace function to mask personal phone numbers in the incoming records. The replace scalar function allows you to mask sensitive data by using regular expression patterns.

Assume that your incoming records contain the following in the body.

18C4DF96F5A69E35952134948DB94424,98B4686144A13EE8378510888F22D782,Game Card,12.5,2018-01-13 09:15:00,2018-01-13 09:29:00,-73.986061,40.727932, 800-111-1111

Prerequisites

These steps assume that you've already configured SCloud. See Get started with SCloud.

Steps

Use the eval and replace function to mask sensitive data.

  1. From the homepage, click Pipeline and select Splunk DSP Firehose as your data source.
  2. From the Canvas View, click the + icon and add the Eval function to your pipeline.
  3. In the Eval function, cast body to be a string. Then, enter a regular expression pattern to detect phone numbers in body. The following regular expression pattern replaces the phone number in the body field with <redacted>.
    body=replace(cast(body, "string"), /[0-9]{3}[-.][0-9]{3}[-.][0-9]{4}/, "<redacted>")
  4. Click the Start Preview Start Preview button button and select the Eval function.
  5. Log in to SCloud.
    ./scloud login

    SCloud doesn't return your login metadata or access token. If you want to see your access token you must log in to SCloud using the verbose flag: ./scloud login --verbose.

  6. Send a sample record to your pipeline to verify that your data is being masked.
    ./scloud ingest post-events <<< "3CACE6A20EB462544D4F0F3DA1303EDC,D9D06E5E87D7BF8D0CC768BDECE121BD,Game Card,8.5,2018-01-13 09:22:00,2018-01-13 09:30:00,-73.98584,40.731003, 123-123-1234"

See also

Functions
Eval
Replace
Casting
Related topics
About regular expressions
Last modified on 21 March, 2022
Test your pipeline configuration   Remove unwanted fields from your data

This documentation applies to the following versions of Splunk® Data Stream Processor: 1.3.0, 1.3.1, 1.4.0, 1.4.1, 1.4.2, 1.4.3, 1.4.4, 1.4.5


Was this topic useful?







You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters