Web Intelligence dashboards

Use the Web Intelligence dashboards to identify potential and persistent threats in your environment.

HTTP Category Analysis dashboard

The HTTP Category Analysis dashboard looks at categories of traffic data. Any traffic data, such as firewall, router, switch, or network flows, can be summarized and viewed in this dashboard.

• Compare statistical data to identify traffic outliers, or traffic different from what is typically found in your environment.
• Look for category counts that fall outside of the norm (small or large) that may indicate a possible threat.
• Find low volume traffic activity and drill down from the summarized data to investigate events.
• Use sparklines to identify suspicious patterns of activity by category.

Unknown traffic categories

Use the "Show only unknown categories" filter on the HTTP Category Analysis dashboard to filter and view unknown categories of web traffic.

Before you can filter unknown traffic, define which categories are unknown.

1. Select Settings > Tags.
2. Click List by tag name.
3. Select an App context of DA-ESS-NetworkProtection or a related network add-on, such as TA-websense.
4. Click New.
5. Type a Tag name of unknown.
6. Type a Field-value pair to define as unknown traffic.
   For example, category=undetected.
7. Click Save.

Dashboard filters

Filters can help refine the HTTP category list.

Filter by	Description
Time Range	Select the time range to represent.
Advanced Filter	Click to see the list of category events that can be filtered for this dashboard. See Configure per-panel filtering in Splunk Enterprise Security in Administer Splunk Enterprise Security for information.

Dashboard panels

Click chart elements or table rows to display raw events. See Drill down to raw events for more information on this feature. The following table describes the panels for this dashboard.

Panel	Description
Key Indicators	Displays the metrics relevant to the dashboard sources over the past 48 hours. Key indicators represent summary information and appear at the top of the dashboard. See Key indicators in Splunk Enterprise Security.
Category Distribution	Displays category counts as a scatter plot, with count as the x-axis and src_count as the y-axis. The chart updates when you change filters or the time range. Hover over an item to see details.
Category Details	Displays details of the HTTP categories, including a sparkline that represents the activity for that HTTP category over the last 24 hours.

HTTP User Agent Analysis dashboard

Use the HTTP User Agent Analysis dashboard to investigate user agent strings in your proxy data and determine if there is a possible threat to your environment.

• A bad user agent string, where the browser name is misspelled (like Mozzila) or the version number is completely wrong (v666), can indicate an attacker or threat.
• Long user agent strings are often an indicator of malicious access.
• User agent strings that fall outside of the normal size (small or large) may indicate a possible threat that should be looked at and evaluated.

The Advanced Filter can be used to include or exclude specific user agents. Use the statistical information to visually identify outliers. In the summarized data, you can evaluate user agents for command and control (C&C) activity, and find unexpected HTTP communication activity.

Dashboard filters

The dashboard includes a number of filters that can help refine the user agent list.

Filter by	Description
Standard Deviation Index	The percentage (%) shows the amount of data that will be filtered out if that number of standard deviations is selected. Choose a higher number of deviations to see fewer user agent strings, or choose a lower number of deviations to see a greater number of user agent strings.
Time Range	Select the time range to represent.
Advanced Filter	Click to see the list of category events that can be filtered for this dashboard. See Configure per-panel filtering in Splunk Enterprise Security in Administer Splunk Enterprise Security for information.

Dashboard panels

Click chart elements or table rows to display raw events. See Drill down to raw events for more information on this feature. The following table describes the panels for this dashboard.

Panel	Description
Key Indicators	Displays the metrics relevant to the dashboard sources over the past 48 hours. Key indicators represent summary information and appear at the top of the dashboard. See Key indicators in Splunk Enterprise Security.
User Agent Distribution	Displays user agent strings as a scatter plot, with length as the x-axis and count as the y-axis. The chart updates when you change the filters or the time range. Hover over an item to see details about the raw data.
User Agent Details	Displays details of the user agents in your environment, including the string value of the user agent and a sparkline that represents the activity for that user agent string over the last 24 hours.

New Domain Analysis dashboard

The New Domain Analysis dashboard shows any new domains that appear in your environment. These domains can be newly registered, or simply newly seen by ES. Panels display New Domain Activity events, New Domain Activity by Age, New Domain Activity by Top Level Domain (TLD), and Registration Details for these domains.

• View hosts talking to recently registered domains.
• Discover outlier activity directed to newly registered domains in the New Domain Activity by Age panel.
• Identify unexpected top level domain activity in the New Domain Activity by TLD panel.
• Investigate high counts of new domains to find out if your network has an active Trojan, botnet, or other malicious entity.

Dashboard filters

The dashboard includes a number of filters to refine the list of domains displayed.

Filter by	Description
Domain	Enter the domain (Access, Endpoint, Network).
New Domain Type	Select Newly Registered or Newly Seen to filter the types of domains to be viewed.
Maximum Age (days)	The time range for the newly seen or newly registered domains. The default is 30 days.
Time Range	Select the time range to represent.
Advanced Filter	Click to see the list of category events that can be filtered for this dashboard. See Configure per-panel filtering in Splunk Enterprise Security in Administer Splunk Enterprise Security for information.

Dashboard panels

Click chart elements or table rows to display raw events. See Drill down to raw events for more information on this feature. The following table describes the panels for this dashboard.

Panel	Description
New Domain Activity	Table view of information about new domain activity
New Domain Activity by Age	Scatter plot that displays Age as the x-axis and Count as the y-axis. Hover over a square for the exact age and number of new domains.
New Domain Activity by TLD (Top Level Domain)	A bar chart with Count as the x-axis and TLD as the y-axis. Hover over a bar for the current number of events for a top level domain.
Registration Details	A table view of information about new domain registrations. Click a domain in the table to open a search on that domain and view the raw events.

Configure the external API for WHOIS data

To see data in the New Domain Analysis dashboard, you must configure a connection to an external domain lookup data source. You can use the example domain lookup data source provided in ES or you can use one of your choice. The dashboard will only report whether or not a domain is newly seen until this modular input is configured and enabled.

The example uses the external domain source domaintools.com, which provides a paid API for WHOIS data.

1. Sign up for a domaintools.com account.
2. Collect the API host name and your API access credentials from the site. Note that the API access credentials are different from your account email address.

Use the API information to set up a modular input in Splunk Enterprise Security.

1. From the ES menu bar, Select Configure > Data Enrichment > WHOIS Management.
2. Click Enable next to whois_domaintools.
3. Click the name of the modular input to add the API hostname and username used to access the domaintools API.
4. Save the API credentials on the Credential Management view. See Manage input credentials in Splunk Enterprise Security.

If you choose to use a different domain source, complete the following steps.

1. From the ES menu bar, Select Configure > Data Enrichment > WHOIS Management.
2. Click New.
3. Enter the name of the modular input to add the API hostname and username used to access the API.
4. Save the API credentials on the Credential Management view. See Manage input credentials in Splunk Enterprise Security.
5. Click Enable next to the name of the modular input you just created.

Until you enable the modular input, domains processed by the input will not be queued. This prevents the checkpoint directory from filling up with files.

After enabling the modular input, enable the outputcheckpoint_whois macro to create checkpoint data.

1. Select Configure > General > General Settings.
2. Select Enable for the Domain Analysis setting to enable WHOIS tracking.

The modular input stores information in the whois_tracker.csv lookup file. After a file exists in the $SPLUNK_HOME/var/lib/splunk/modinputs/whois directory, the whois index will begin to populate with data. After they are processed, checkpoint files will be deleted.

Errors versus normal behavior

• If you see 404 errors in the logs, this is normal behavior when querying domains that don't exist.
• If you see 400 errors in the logs returned from the domaintools API, this is normal behavior when querying domains with invalid top level domains.
• If you don't see new events in the whois index, this might be normal behavior if using HTTP:// the api_url when it should be HTTPS://. You can use either HTTP:// or HTTPS:// in the url. However, if you don't pick HTTP:// or HTTPS://, then HTTP:// is prepended to the api_url by default .

URL Length Analysis dashboard

The URL Length Analysis dashboard looks at any proxy or HTTP data that includes URL string information. Any traffic data containing URL string or path information, such as firewall, router, switch, or network flows, can be summarized and viewed in this dashboard.

• Compare each URL statistically to identify outliers.
• Investigate long URLs that have no referrer.
• Look for abnormal length URLs that contain embedded SQL commands for SQL injections, cross-site scripting (XSS), embedded command and control (C&C) instructions, or other malicious content.
• Use the details table to see how many assets are communicating with the URL.

Use the key indicators to compare each new URL and to identify outlier URL strings, ones that are different from what is typically found in your environment. URLs that fall outside of the normal size (small or large) may indicate a possible threat. Unusually long URL paths from unfamiliar sources and/or to unfamiliar destinations are often indicators of malicious access and should be examined.

Dashboard filters

Use the filters to refine the URL length events represented on the dashboard.

Filter by	Description
Standard Deviation Index	The percentage (%) shows the amount of data that will be filtered out if that number of standard deviations is selected. Choose a higher number of deviations to see fewer user agent strings, or choose a lower number of deviations to see a greater number of user agent strings.
Time Range	Select the time range to represent.
Advanced Filter	Click to see the list of category events that can be filtered for this dashboard. See Configure per-panel filtering in Splunk Enterprise Security in Administer Splunk Enterprise Security for information.

Dashboard panels

Click chart elements or table rows to display raw events. See Drill down to raw events for more information on this feature. The following table describes the panels for this dashboard.

Panel	Description
Key Indicators	Displays the metrics relevant to the dashboard sources over the past 48 hours. Key indicators represent summary information and appear at the top of the dashboard. See Key indicators in Splunk Enterprise Security in this manual.
URL Length Anomalies Over Time	The chart displays a count of URL length anomalies across time. It displays URL lengths greater than the number of standard deviations selected in the filter (2 by default) displayed in a line graph with time as the x-axis and count as the y-axis.
URL Length Details	Table that displays the URL strings and details such as the full URI string. If there is more that one event from a source IP address, the count column shows how many events are seen. Z indicates the standard deviations for the URL length.