Scenario: Find data exfiltration using Splunk Enterprise security

Enterprise Security provides statistics and interesting events on security domain specific dashboards. Using the dashboards together, you can build a workflow for investigating threats by reviewing the results, isolating the events that require attention, and using the contextual information provided to drill down into the issue.

This scenario provides an example of detecting potential data exfiltration involving the domain dataker.ch. Use this scenario as an example of how to perform a similar investigation in your own environment.

Prerequisites

• Verify that a Splunk platform instance with Splunk Enterprise Security is installed and configured.
• Verify that these CIM data models contain data: Network Traffic, Network Resolution, Email, and Web. Data sources include web proxy or next-gen firewall (NGFW) logs, Splunk Stream, Bro, Exchange, Sendmail, and DNS logs.
• Verify that the Splunk App for Stream is installed and the Splunk Stream add-on is configured.

Start with Incident Review

Enterprise Security includes correlation searches that report on suspicious activity across security domains. Some common paths for data exfiltration are tracked by the correlation searches.

Correlation search	Description	Data Models : Sources
Prohibited Port Activity Detected	Detects the use of ports that are not approved. Unapproved port detection is useful for detecting the installation of new software (potentially unapproved) or a successful compromise of a host (such as the presence of a backdoor or a system communicating with a botnet).	Sources that populate the Network Traffic Data Model: Splunk Stream, firewall traffic, Bro, etc.
High Volume Email Activity to Non-corporate Domains by User	Alerts on high volume email activity by a user to non-corporate domains.	Sources that populate the Email data model: Splunk Stream, Bro, Exchange, Sendmail, etc.
Host Sending Excessive Email	Alerts when a host not designated as an e-mail server sends excessive e-mail to one or more target hosts.	Sources that populate the Email data model: Splunk Stream, Bro, Exchange, Sendmail, etc.
Excessive DNS Queries	Alerts when a host starts sending excessive DNS queries	Sources that populate the Network resolution data model: Splunk Stream, Bro, Microsoft DNS, bind, Infoblox, etc.
Substantial Increase In Port Activity	Alerts when a statistically significant increase in events on a given port is observed.	Sources that populate the Network Traffic Data Model: Splunk Stream, firewall traffic, Bro, etc.
Web Uploads to Non-corporate Sites by Users	Alerts on high volume web uploads by a user to non-corporate domains.	Sources that populate the Web data model: web proxy, next-gen firewall (NGFW) logs, etc.
Personally Identifiable Information Detected	Detects personally identifiable information (PII) in the form of payment card data in machine-generated data. Some systems or applications inadvertently include sensitive information in logs thus exposing it in unexpected ways.	No specific data model: system log files, application log files, network traffic payloads, etc.

Assign notable events for investigation

1. From the Enterprise Security menu bar select Incident Review.
2. Use the Search option on the Incident Review dashboard to look for a specific notable event.
3. (Optional) Reprioritize the notable event by changing the Urgency before assigning it.
4. Assign a notable event to an analyst for review and investigation.

While analysts review any notable events representing possible data exfiltration attempts, you can investigate other dashboard panels for signs of anomalous behavior.

Review the User Activity dashboard

The User Activity dashboard displays panels representing common risk-generating user activities.

1. On the Enterprise Security menu bar, select Security Intelligence > User Intelligence > User Activity.
2. View the key indicators NonCorp Web Volume and NonCorp Email Volume for evidence of suspicious changes over the last 24 hours.

Non-corporate Web Uploads

Examine the Non-corporate Web Uploads panel to identify suspicious activity involving data being uploaded to external locations. Also look for unknown users or credentials, Watchlisted identities, and large data transfers indicated in the size column.

Non-corporate Email Activity

Review the Non-corporate Email Activity panel to look for suspiciously large email messages to addresses outside the organization. Also look for uncommon user names, Watchlisted identities, and large messages or a large number of smaller messages.

If suspicious activity is found, create a notable event and assign it to an analyst for investigation. Continue to look at other dashboards for indications of compromise.

Review the Email Activity dashboard

The Email Activity dashboard displays metrics relevant to the email infrastructure.

1. On the Enterprise Security menu bar, select Security Intelligence > Protocol Intelligence > Email Activity.

Top Email Sources

Examine the Top Email Sources panel to find surges in email counts by IP address. Look for unfamiliar addresses sending a large numbers of messages. Use the sparklines to identify consistent spikes of activity from a host, as it can be an indicator of automated or scripted activity.

On a panel with dense search results and many fields, use the Open in Search icon in the lower right corner to open the results in a separate search view.

Large Emails

Review the Large Emails panel and look for emails larger than 2MB that were sent to internal or external addresses.

Selecting a record on either panel will drill down into the Email Search dashboard, where you can continue to investigate the email traffic. If suspicious activity is found, create a notable event and assign it to an analyst for investigation.

Review the DNS Activity dashboard

The DNS Activity dashboard displays metrics relevant to the DNS infrastructure.

1. On the Enterprise Security menu bar, select Security Intelligence > Protocol Intelligence > DNS Activity.

Queries Per Domain

Examine the Queries Per Domain panel to find unfamiliar domains receiving a large number of queries from internal hosts. You see there are a large number of DNS queries for subdomains of "dataker.ch", and choose to examine the DNS traffic as a first step. Selecting a record on the Queries Per Domain panel will drill down to the DNS Search page.

Follow the drilldown to the DNS Search dashboard

A new browser window opens to the DNS Search dashboard and begins to search on the selected domain over the time range. You determine that the src_ip of all of the queries is in the corporate desktop range. Use the Source' filter to restrict the search to one subnet. Looking at the events, you see a large amount of traffic that includes base64 encoded subdomains.

Utilizing DNS queries with encoded information is a known method to exfiltrate data. But you do not know if the work is being initiated by malware on the asset of an innocent user, or as an insider threat. Reviewing the asset might provide some clarity.

1. Select a raw event in the DNS Search dashboard,
2. Use the arrow on the left to expand the field results.
3. Find the src field and open the Actions menu.
4. Select Asset Center.

Examine the asset in Asset Center

The Asset Center dashboard reports on known values for a specified asset. The asset responsible for sending the encoded DNS queries is reported as a standard user desktop. As the asset details did not provide any additional clues, you choose to continue the investigation as an insider threat. You expect to find malware running on the asset as the tool used to exfiltrate data, but tracking the user's activity is an appropriate preemptive step. Let's create a new notable event to track our investigation.

Create a new notable event

On the Enterprise Security menu bar, open Configure > Incident Management and select New Notable Event.

Field	Details
Title	Possible data exfiltration: <Asset>, <User>, <Date>
Security Domain	Threat
Urgency	Critical. This investigation is a top priority.
Owner	<ANALYST>
Status	In Progress
Description	There might be data exfiltration via DNS. Begin enhanced monitoring of <User>, their access controls, and the <Asset>. Notify the SecOps Manager and HR regarding possible insider threat.

After updating and assigning the notable event, monitor the network for encoded DNS data.

Use Splunk Stream to capture DNS

Monitoring the network traffic to determine if DNS queries include encoded data requires a tool to monitor and sort the data before feeding the results into Enterprise Security.

Splunk offers About Splunk Stream as the preferred method of capturing encoded DNS traffic on the network.

Build a search that utilizes Stream results. Begin by using the Deployment Server to install the Stream Add-on onto the asset. The add-on monitors the network traffic and sends the DNS data to Enterprise Security for evaluation.

DNS search for encoded data

On the Enterprise Security menu bar, open Search and select Search.

Now that the Stream add-on is capturing the DNS data, we need a search to find Base64 encoded content in DNS queries. The goal is to examine the DNS query field of the data stream to find subdomain streams that contain only Base64 valid characters.

sourcetype="stream:dns" (message_type=RESPONSE OR message_type=TXT) | rex field=query "(?<subdomain>.*?)\..*" | regex subdomain="^(([A-Za-z0-9+/]{4})+)$|^([A-Za-z0-9+/]{4})+(([A-Za-z0-9+/]{2}[AEIMQUYcgkosw048]=)|([A-Za-z0-9+/][AQgw]==)).*$" | stats count by subdomain

The query can result in false positive matches if the subdomain contains a number of characters divisible by 4, and contains only alphanumeric characters. A visual inspection of the search results by an analyst will be required.

Data exfiltration summary

The notable events generated by Splunk Enterprise Security provided a starting point for the investigation by detecting common sources of suspicious behavior. The User and Email Activity dashboards expose recurring or large data transfers to known and unknown domains. The Stream add-on allows the capture and filtering of network data from internal hosts. By using the tools and searches provided with ES, an investigator can check common data exfiltration paths and establish active monitoring of compromised machines.