Splunk® Data Stream Processor

Use the Data Stream Processor

Acrobat logo Download manual as PDF

Acrobat logo Download topic as PDF

Masking sensitive data in the

The allows you to mask or obfuscate sensitive data before sending your data to a destination. This means that you can make sure that private data stays private, and you can mask sensitive data according to your compliance standards and needs.

To mask data, use the replace scalar function. In this example, we'll use replace with the eval function.

Masking sensitive data with replace

The exact steps for masking your sensitive data vary depending on the specific data that you are working with. The replace function takes string as input, so if you want to mask data from a field that is not a string, you must cast that field to be a string first. This is demonstrated in the example. You can confirm the data types of your fields by selecting a function in your pipeline, then selecting the View Configurations tab, and then checking the list of fields under Input Fields.

Example: masking phone numbers with the replace function

In this example, we'll use the replace function to mask personal phone numbers in the incoming records. The replace scalar function allows you to mask sensitive data by using regular expression patterns.

Assume that your incoming records contain the following in the body.

18C4DF96F5A69E35952134948DB94424,98B4686144A13EE8378510888F22D782,Game Card,12.5,2018-01-13 09:15:00,2018-01-13 09:29:00,-73.986061,40.727932, 800-111-1111


These steps assume that you've already configured SCloud. See Get started with SCloud.


Use the eval and replace function to mask sensitive data.

  1. From the homepage, click Build Pipeline and select Splunk DSP Firehose as your data source.
  2. From the Data Pipelines Canvas view, click the + icon and add the Eval function to your pipeline.
  3. In the Eval function, cast body to be a string. Then, enter a regular expression pattern to detect phone numbers in body. The following regular expression pattern replaces the phone number in the body field with <redacted>.
    body=replace(cast(body, "string"), /[0-9]{3}[-.][0-9]{3}[-.][0-9]{4}/, "<redacted>")
  4. Click Start Preview and select the Eval function.
  5. Log in to SCloud.
    ./scloud login

    SCloud doesn't return your login metadata or access token. If you want to see your access token you must log in to SCloud using the verbose flag: ./scloud login --verbose.

  6. Send a sample record to your pipeline to verify that your data is being masked.
    ./scloud ingest post-events <<< "3CACE6A20EB462544D4F0F3DA1303EDC,D9D06E5E87D7BF8D0CC768BDECE121BD,Game Card,8.5,2018-01-13 09:22:00,2018-01-13 09:30:00,-73.98584,40.731003, 123-123-1234"

See also

Related topics
About regular expressions
Last modified on 17 December, 2020
Test your pipeline configuration with preview sessions
Remove unwanted fields from your data

This documentation applies to the following versions of Splunk® Data Stream Processor: 1.2.0, 1.2.1

Was this documentation topic helpful?

You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters