Masking sensitive data in the

The allows you to mask or obfuscate sensitive data before sending your data to a destination. This means that you can make sure that private data stays private, and you can mask sensitive data according to your compliance standards and needs.

To mask data, use the replace scalar function. In this example, we'll use replace with the eval function.

Masking sensitive data with replace

The exact steps for masking your sensitive data vary depending on the specific data that you are working with. The replace function takes string as input, so if you want to mask data from a field that is not a string, you must cast that field to be a string first. This is demonstrated in the example. You can confirm the data types of your fields by selecting a function in your pipeline, then selecting the View Configurations tab, and then checking the list of fields under Input Fields.

Example: masking phone numbers with the replace function

In this example, we'll use the replace function to mask personal phone numbers in the incoming records. The replace scalar function allows you to mask sensitive data by using regular expression patterns.

Assume that your incoming records contain the following in the body.

18C4DF96F5A69E35952134948DB94424,98B4686144A13EE8378510888F22D782,Game Card,12.5,2018-01-13 09:15:00,2018-01-13 09:29:00,-73.986061,40.727932, 800-111-1111

Prerequisites

These steps assume that you've already configured SCloud. See Get started with SCloud.

Steps

Use the eval and replace function to mask sensitive data.

1. From the homepage, click Pipeline and select Splunk DSP Firehose as your data source.
2. From the Canvas view, click the + icon and add the Eval function to your pipeline.
3. In the Eval function, cast body to be a string. Then, enter a regular expression pattern to detect phone numbers in body. The following regular expression pattern replaces the phone number in the body field with <redacted>.

   body=replace(cast(body, "string"), /[0-9]{3}[-.][0-9]{3}[-.][0-9]{4}/, "<redacted>")

4. Click Start Preview and select the Eval function.
5. Log in to SCloud.

   ./scloud login

   SCloud doesn't return your login metadata or access token. If you want to see your access token you must log in to SCloud using the verbose flag: ./scloud login --verbose.

6. Send a sample record to your pipeline to verify that your data is being masked.

   ./scloud ingest post-events <<< "3CACE6A20EB462544D4F0F3DA1303EDC,D9D06E5E87D7BF8D0CC768BDECE121BD,Game Card,8.5,2018-01-13 09:22:00,2018-01-13 09:30:00,-73.98584,40.731003, 123-123-1234"

See also

Functions

Eval

Replace

Casting

Related topics

About regular expressions