On October 30, 2022, all 1.2.x versions of the Splunk Data Stream Processor will reach its end of support date. See the Splunk Software Support Policy for details.
Masking sensitive data in the
The allows you to mask or obfuscate sensitive data before sending your data to a destination. This means that you can make sure that private data stays private, and you can mask sensitive data according to your compliance standards and needs.
To mask data, use the replace scalar function. In this example, we'll use replace with the eval function.
Masking sensitive data with replace
The exact steps for masking your sensitive data vary depending on the specific data that you are working with. The replace function takes string as input, so if you want to mask data from a field that is not a string, you must cast that field to be a string first. This is demonstrated in the example. You can confirm the data types of your fields by selecting a function in your pipeline, then selecting the View Configurations tab, and then checking the list of fields under Input Fields.
Example: masking phone numbers with the replace function
In this example, we'll use the replace function to mask personal phone numbers in the incoming records. The replace scalar function allows you to mask sensitive data by using regular expression patterns.
Assume that your incoming records contain the following in the body
.
18C4DF96F5A69E35952134948DB94424,98B4686144A13EE8378510888F22D782,Game Card,12.5,2018-01-13 09:15:00,2018-01-13 09:29:00,-73.986061,40.727932, 800-111-1111
Prerequisites
These steps assume that you've already configured SCloud. See Get started with SCloud.
Steps
Use the eval and replace function to mask sensitive data.
- From the homepage, click Pipeline and select Splunk DSP Firehose as your data source.
- From the Canvas view, click the + icon and add the Eval function to your pipeline.
- In the Eval function, cast
body
to be a string. Then, enter a regular expression pattern to detect phone numbers inbody
. The following regular expression pattern replaces the phone number in thebody
field with <redacted>.body=replace(cast(body, "string"), /[0-9]{3}[-.][0-9]{3}[-.][0-9]{4}/, "<redacted>")
- Click Start Preview and select the Eval function.
- Log in to SCloud.
./scloud login
SCloud doesn't return your login metadata or access token. If you want to see your access token you must log in to SCloud using the verbose flag:
./scloud login --verbose
. - Send a sample record to your pipeline to verify that your data is being masked.
./scloud ingest post-events <<< "3CACE6A20EB462544D4F0F3DA1303EDC,D9D06E5E87D7BF8D0CC768BDECE121BD,Game Card,8.5,2018-01-13 09:22:00,2018-01-13 09:30:00,-73.98584,40.731003, 123-123-1234"
See also
- Functions
- Eval
- Replace
- Casting
- Related topics
- About regular expressions
Test your pipeline configuration with preview sessions | Remove unwanted fields from your data |
This documentation applies to the following versions of Splunk® Data Stream Processor: 1.2.0, 1.2.1-patch02, 1.2.1, 1.2.2-patch02, 1.2.4, 1.2.5
Feedback submitted, thanks!