Asset and Identity management

To effectively detect security intrusions, an organization must be able to correlate events in log data with specific assets and identities that may be responsible for, or affected by the intrusion. Splunk Enterprise Security uses an asset and identity system to correlate asset and identity information with events to enrich and provide context to your data. This system takes information from external data sources to populate lookups, which are then correlated with events at search time.

The Identity Management dashboard

The asset and identity lookups are managed through the Identity Management configuration dashboard. Browse to Configure > Data Enrichment and select Identity Management to review a list of configured assets and identities.

Field	Description
Name	A short descriptive name for the list. Selecting the link will open the Identity Manager Settings page and display the configuration of the chosen list.
Category	A descriptive category name for the list.
Description	A description of the contents.
Type	Defines the lookup as an asset or an identity list.
Source	The lookup definition name for the list. Selecting the link will open the Edit Lookup page with the contents of the chosen list.
Status	Enabled or Disabled. Changing the status will initiate a merge at the next scheduled interval.
Actions	Selecting Clone opens the Identity Manger Settings page with duplicated information from the selected list.

Edit an existing asset or identity list

Edit an asset or identity list from Identity Management:

1. In Enterprise Security, go to Configure > Data Enrichment and select Identity Management. A list of asset and identity files for Enterprise Security are displayed.
2. Find the name of the asset or identity list you want to edit, and select Source. The list will open in an interactive editor.
3. Use the scroll bars to view the columns and rows in the table. Double click a cell to add, change, or remove content.
4. Click Save when you are finished.

You can also use the lookup editor in Lists and Lookups to edit an existing list.

1. In Enterprise Security, go to Configure > Data Enrichment and click Lists and Lookups.
2. Select the name of the list you want to edit. The list will open in an interactive editor.
3. Use the scroll bars to view the columns and rows in the table. Double click a cell to add, change, or remove content.
4. Click Save when you are finished.

Changes made to an asset or identity list will be reflected in search results after the next scheduled merge. For more information on the merging of asset and identity lists, see "Merging the asset and identity lists" in this topic.

Adding asset data

An asset represents any device or system in the environment that generates data. Asset correlation allows indexed events to be matched against a defined list of assets. When a match occurs, the original indexed event gains new fields through association with the asset, enriching the event with information on the asset's priority, location, or other details.

An asset list provides external information about the devices on your system, such as the asset priority, owner, and business unit; the geographic location of the asset; and the asset's DNS and Windows machine name. Some of these fields, such as latitude, longitude, and priority are used on dashboard charts. Other fields, such as business unit and category, are used by the filters at the top of the various domain dashboards. For an overview of asset fields with examples, see "Asset correlation" in this manual.

To add an asset source to Splunk Enterprise Security, several steps are required.

1. Extract the asset data from a source
2. Format the data as an asset lookup
3. Configure an input for the asset list
4. Merge the asset lists

Extract the asset data from a source

The preferred method of adding asset information into ES is through automated capture from an existing asset database. For a list of potential asset sources and collection methods, see "Collection methods for assets and identities" in this topic.

For an example of extracting asset data from events indexed in the Splunk platform, see "Add asset information from indexed events" in this topic.

To populate an asset list manually, see "Static asset and identity information" in this topic.

Format the data as an asset lookup

For a list of the fields and values in an asset list, see "Asset lookup fields" in this manual. The resulting file must be a plain text, csv formatted file with Unix line endings, and must include a .csv filename extension.

For an example asset list, review the demo_assets.csv.default file in SA-IdentityManagement/package/lookups.

Defining multihomed hosts

When adding multihomed hosts or devices to an asset list, define each IP address as a unique record with an identical DNS name. The merging process does not support defining a multi-homed host as one record in an asset list.

Configure an input for the asset list

For instructions, see "Configuring a new asset or identity list" in this topic.

Merge the asset lists

For details, see "Merging the asset and identity lists" in this topic.

Adding identity data

An identity represents a user, credential, or a role used to grant access to a device or system. Identity correlation allows indexed events to be matched against a defined list of users or system accounts. When a match occurs, the original indexed event gains new fields through association with an identity, enriching the event with information on the identity's priority, role, or the functional area to which it belongs.

To add an identity source for use in Enterprise Security:

1. Extract the identity data from a source
2. Format the data as an identity lookup
3. Configure an input for the identity list
4. Merge the identity lists

The identity lists provides information about the users on your system, such as the screen or login name, first and last name, and email address. Some of these fields, such as priority, watchlist, and endDate are used for dashboard charts and to calculate the urgency of notable events associated with identities. Other fields, such as business unit and category, are used by the filters at the top of the domain dashboards.

Extract the identity data from a source

The preferred method of adding identity information into ES is through automated capture from an existing identity database. For a list of potential sources and collection methods, see "Collection methods for assets and identities" in this topic.

To populate an identity list manually, see "Static asset and identity information" in this topic.

Format the data as an identity lookup

For a list of the fields and values in an identity list, see "Identity lookup fields" in this manual. The resulting file must be a plain text, csv formatted file with Unix line endings, and must include a .csv filename extension.

For an example identity list, review the demo_identities.csv.default file in SA-IdentityManagement/package/lookups.

Configure an input for the identity list

For instructions, see "Configuring a new asset or identity list" in this topic.

Merge the identity lists

For details, see "Merging the asset and identity lists" in this topic.

Configuring a new asset or identity list

1. Configure and upload the new lookup table file.
   1. Browse to Settings > Lookups > Lookup table files.
   2. Choose Add New.
   3. Select a Destination App of SA-IdentityManagement.
   4. Select the lookup file to upload. The file must be a plain text csv format file with Unix line endings and include a .csv filename extension. Example: network_assets_from_CMDB.csv
   5. Provide the destination file name. Enter the name this lookup table file will have on the Splunk server. The name should include a .csv filename extension. For example, network_assets_from_CMDB.csv
      
   6. Save.
2. Set the permissions on the lookup table file.
   1. In Lookup Table Files find the new lookup and select Permissions.
   2. Set Object should appear in to All apps.
   3. Set Read access for Everyone.
   4. Set Write access for admin or other roles. See "Adding capabilities to a role" in the Installation and Upgrade Manual.
   5. Save.
3. Add a new lookup definition.
   1. Browse to Settings > Lookups > Lookup definitions.
   2. Choose Add New.
   3. Select a Destination App of SA-IdentityManagement.
   4. Provide a name for the lookup source. The name defined here must be the name used in the Identity Management input stanza definition. Example: network_assets_from_CMDB
   5. Select a Type of File based.
   6. Select the lookup table file created. Example: network_assets_from_CMDB.csv
   7. Save.
4. Set the permissions on the lookup definition.
   1. In Lookup definitions, find the new definition by its name and select Permissions.
   2. Set Object should appear in to All apps.
   3. Set Read access for Everyone.
   4. Set Write access for admin or other roles. See "Adding capabilities to a role" in the Installation and Upgrade Manual.
   5. Save.
5. Add a new source input stanza.
   1. Browse to Configure > Identity Management > Identity Manager.
   2. Select New.
   3. Add the information about the list to the fields in Identity Manager Settings. Fill out the required fields in the new source input.
   4. Define a Category, the new asset or identity list's short descriptive name. For example, CMDB_network_assets.
   5. Add a description of the contents.
   6. Set a Type of "asset" or "identity". For example, asset.
   7. Set Source to refer to the lookup definition name. For example, lookup://network_assets_from_CMDB.
   8. Save.
6. Verify the new lookup based source was imported. See "Validate assets and identities are working" in this topic.

Merging the asset and identity lists

The contents of all configured and enabled asset and idenitity lists in Identity Management are merged by a modular input scheduled to run every 5 minutes.

For asset correlation, the files are merged and then expanded, cross-referenced lookup files are created. When an event contains the fields: src, dest, host, orig_host, dvc two comparisons are performed: one to check if the field value corresponds to a value in the asset table using a string match, and the other to check if the field value corresponds to a value in the asset table using a CIDR subnet match.

For identity correlation, the lookup files are merged and two expanded lookup files are created. When an event contains the fields: user and src_user a comparison is performed to check if the field value corresponds to a value in the identities table.

Function	Table Name	Lookup name
String-based asset correlation	assets_by_str.csv	LOOKUP-zu_asset_lookup_host_as_str_only LOOKUP-zu_asset_lookup_orig_host_as_str_only LOOKUP-zu_asset_lookup_src_as_str_only LOOKUP-zu_asset_lookup_dest_as_str_only LOOKUP-zu_asset_lookup_dvc_as_str_only
CIDR subnet-based asset correlation	assets_by_cidr.csv	LOOKUP-zv_asset_lookup_host_as_cidr_only LOOKUP-zv_asset_lookup_orig_host_as_cidr_only LOOKUP-zv_asset_lookup_src_as_cidr_only LOOKUP-zv_asset_lookup_dest_as_cidr_only LOOKUP-zv_asset_lookup_dvc_as_cidr_only
String-based identity correlation	identities_expanded.csv	LOOKUP-zy_identity_lookup_src_user_only LOOKUP-zy_identity_lookup_user_only
Default field correlation	asset_identity_lookup_default_fields .csv	LOOKUP-zz-asset_identity_lookup_default_fields

The automatic lookups that drive ES asset and identity correlation reside in the SA-IdentityManagement app, and are defined in the SA-IdentityManagement/default/props.conf file. All asset and identities lookup files are stored in the path: $SPLUNK_HOME/etc/apps/SA-IdentityManagement/lookups/

The asset and identities lookups are applied to every search on the system, and not scoped to a source or sourcetype.

Force a merge

To perform an immediate check and merge of updates to the assets and identities lists, the modular input can be run from the CLI. Calling an input script from the CLI requires the $SPLUNK_HOME environment variable to be set; in order to do this, run the following:

On *nix: source /opt/splunk/bin/setSplunkEnv

On Windows: splunk.exe envvars > setSplunkEnv.bat & setSplunkEnv.bat

Run merge:

$SPLUNK_HOME/bin/splunk cmd splunkd print-modinput-config identity_manager | $SPLUNK_HOME/bin/python $SPLUNK_HOME/etc/apps/SA-IdentityManagement/bin/identity_manager.py --username=admin

Credentials are required. The user will be prompted to provide a password for the --username defined.

When the identity manager input is triggered, it will evaluate all enabled lookups and check for changes. If no changes have been made to any lookups since the last run, the identity manager input will not regenerate the merged lookup files.

To force a merge of the assets or identities lists at the next interval, a file creation in the modinputs folder can be run from the CLI.

touch $SPLUNK_HOME/var/lib/splunk/modinputs/identity_manager/force_asset

touch $SPLUNK_HOME/var/lib/splunk/modinputs/identity_manager/force_identity

Enabling, disabling, or changing the configuration of an asset or identity list in Identity Management will begin a merge at the next scheduled interval.

Verify the merging process

To verify that the expansion process has completed, search the _internal index.

To display the last time the merge occurred: index=_internal source=*python_modular_input.log "Updated: target lookup table"

To display successive runs adding the events with no merging required: index=_internal source=*python_modular_input.log "Updated: target lookup table" OR "No merging required"

The most common reason for failure is incorrect formatting or invalid data in the asset or identities lookup files.

Note: The merge process checks for new input every 5 minutes, but does not perform any work unless an asset or identity table has been modified. To force a merge of the contents in the merged asset and identity lists, update a lookup table with new information, or disable and enable an list in Identity Management.

Validate assets and identities are working

To test an asset lookup using a search, choose a record with data the ip, mac, nt_host, or dns fields from an asset list and search for it:
| stats count | eval src="1.2.3.4" | `get_asset(src)`

To view the available assets using a dashboard, browse to Security Domains > Identity > Asset Center. For more information, see "Asset Center dashboard" in this manual.

To test an identity lookup using a search, choose any record's identity field from an identities list and search for it:
| stats count | eval user="VanHelsing" | `get_identity4events(user)`

To view the available identities using a dashboard, browse to Security Domains > Identity > Identity Center. For more information, see "Identity Center dashboard" in this manual.

To view all available assets using the search command. | `assets`

To view all available assets using the data model. |`datamodel("Identity_Management", "All_Assets")` |`drop_dm_object_name("All_Assets")`

Updating assets and identities

As an organization's asset and identities information changes frequently, it is best to update these lists automatically. This reduces the overhead and maintenance that manual updating requires, and improves data integrity. There are several ways to do this.

• Use DBConnect or another Splunk platform add-on to connect to an external database or repository.
• Use scripted inputs to import and format the lists.
• Use events indexed in the Splunk platform with a search to collect, sort, and export the data to a list.

Static asset and identity information

Edit the "static_assets" and "static_identities" lists to manually include new asset or identity information.

1. In Enterprise Security, go to Configure > Data Enrichment and click Lists and Lookups.
2. Select the "static_assets" or "static_identities" list. The list will open in an interactive editor.
3. Use the scroll bars to view the columns and rows in the table. Double click in a cell to add, change, or remove content.
4. Click Save when you are finished.

Collection methods for assets and identities

The preferred collection method for asset or identity information is through a Splunk platform add-on. Many add-ons can be used to automate connections to external systems for data collection. Use an add-on to connect, collect, and return data to Enterprise Security.

Suggested collection methods for assets and identiites.

Technology	Assets or Identities	Collection methods
Active Directory	Both	SA-ldapsearch and a custom search. For an example, see "Add identity information from Active Directory" in this topic.
LDAP	Both	SA-ldapsearch and a custom search.
CMDB	Assets	DB Connect and a custom search.
ServiceNow	Both	Splunk Add-on for ServiceNow
Asset Discovery	Assets	Asset Discovery App
Bit9	Assets	Splunk Add-on for Bit9 and a custom search.
Cisco ISE	Both	Splunk Add-on for Cisco ISE and a custom search.
Microsoft SCOM	Assets	Splunk Add-on for Microsoft SCOM and a custom search.
Okta	Identities	Splunk Add-on for Okta and a custom search.
Sophos	Assets	Splunk Add-on for Sophos and a custom search.
Symantec Endpoint Protection	Assets	Splunk Add-on for Symantec Endpoint Protection and a custom search.

Adding information from Active Directory

1. Install and configure the "Splunk Support for Active Directory" app.
2. Create and add a lookup file as a source of asset or identity information. See "Loading a new asset or identities list" for details. This lookup file configuration will become the target for a saved search to populate the lookup table file with information from Active Directory. When testing the AD integration, consider disabling the new lookup file configuration to prevent unnecessary merges by the Identity Manager modular input.
3. Using the "ldapsearch" command provided with SA-ldapsearch, construct a search that polls Active Directory (AD) and places the results into a file. The exact syntax for this search will vary depending on the AD configuration.

Identity collection example:

|ldapsearch domain=<domain_name> search="(&(objectclass=user)(!(objectClass=computer)))"
|makemv userAccountControl
|search userAccountControl="NORMAL_ACCOUNT"	
|eval suffix=""
|eval priority="medium"
|eval category="normal"
|eval watchlist="false"
|eval endDate="" 
|table sAMAccountName,personalTitle,displayName,givenName,sn,suffix,mail,telephoneNumber,mobile,manager,priority,department,category,watchlist,whenCreated,endDate
|rename sAMAccountName as identity, personalTitle as prefix, displayName as nick, givenName as first, sn as last, mail as email, telephoneNumber as phone, mobile as phone2, manager as managedBy, department as bunit, whenCreated as startDate
|outputlookup my_identity_lookup

Note: This search assigns static values for "suffix", "endDate", "category", "watchlist", and "priority". After a working search has been constructed and tested, you can replace static values with information from AD.

Asset collection example:

|ldapsearch domain=<domain name> search="(&(objectClass=computer))"
|eval city=""
|eval country=""
|eval priority="medium"
|eval category="normal"
|eval dns=dNSHostName
|eval owner=managedBy
|rex field=sAMAccountName mode=sed "s/\$//g"
|eval nt_host=sAMAccountName
|makemv delim="," dn
|rex field=dn "(OU|CN)\=(?<org>.+)"
|table ip,mac,nt_host,dns,owner,priority,lat,long,city,country,bunit,category,pci_domain,is_expected,should_timesync,should_update,requires_av 
| outputlookup create_empty=false createinapp=true my_asset_lookup

Note: This search assigns static values for several fields. After a working search has been constructed and tested, you can replace static values with information from AD.

Add asset information from indexed events

Hosts communicating with the Splunk platform can be compared to the existing asset information using search commands. The table of unmatched hosts can be reviewed and exported as an asset list. Example:

| `host_eventcount` 
| search host_is_expected=false NOT host_asset_id=*
| fields - firstTime,recentTime,lastTime,_time, host_owner_*,host_asset_tag,host_asset_id 
| sort -totalCount,dayDiff 
| table host,ip,mac,nt_host,dns,owner,priority,lat,long,city,country,bunit,category,pci_domain,is_expected,should_timesync,should_update,requires_av