Extract asset and identity data in Splunk Enterprise Security

Collect, extract, and add asset and identity data in Splunk Enterprise Security to reduce manual updates and improve data integrity. In a Splunk Cloud Platform deployment, work with Splunk Professional Services to design and implement an asset and identity collection solution.

Perform these procedures to extract asset and identity data in Splunk Enterprise Security:

• Collect asset and identity data
• Format an asset or identity list as a lookup
• Configure a new asset or identity list
• Add the new lookup table file
• Set permissions on the lookup table file
• Add a new lookup definition
• Set permissions on the lookup definition

Collect asset and identity data

Follow these steps to collect asset and identity data:

1. Determine where the asset and identity data in your environment is stored.
2. Collect and update your asset and identity data automatically to reduce the overhead and maintenance that manual updating requires and improve data integrity.

• Use Splunk DB Connect or another Splunk platform add-on to connect to an external database or repository.
• Use scripted inputs to import and format the lists.
• Use events indexed in the Splunk platform with a search to collect, sort, and export the data to a list.

Following are some suggested collection methods for assets and identities:

Technology	Asset or identity data	Collection methods
Active Directory	Both	AD LDAP and a custom search.
	Both	Splunk Supporting Add-on for Active Directory
	Both	SecKit Windows Assets Technology Add-on for Splunk Enterprise Security *
LDAP	Both	AD LDAP and a custom search.
CMDB	Asset	Splunk DB Connect for integrating with 3rd Party structured data sources, and a custom search.
ServiceNow	Both	Splunk Add-on for ServiceNow
Bit9	Asset	Splunk Add-on for Bit9 and a custom search.
Cisco ISE	Both	Splunk Add-on for Cisco ISE and a custom search.
Microsoft SCOM	Asset	Splunk Add-on for Microsoft SCOM and a custom search.
Sophos	Asset	Splunk Add-on for Sophos and a custom search.
Symantec Endpoint Protection	Asset	Splunk Add-on for Symantec Endpoint Protection and a custom search.
Amazon Web Services (AWS)	Both	Create Cloud Asset Lookup and Create Cloud Identity Lookup
Azure	Both	Create Cloud Asset Lookup and Create Cloud Identity Lookup
Google Cloud Platform	Both	Create Cloud Asset Lookup and Create Cloud Identity Lookup
Configuration Management Database (CMDB)	Asset	SecKit SA Common tools for populating assets and identities in Enterprise Security and PCI apps *

For more information on custom search commands, see Create custom search commands for apps in Splunk Cloud Platform or Splunk Enterprise.

Format an asset or identity list as a lookup

Format your collected asset or identity data into a lookup file so that it can be processed by Splunk Enterprise Security.

Prerequisite Collect asset and identity data for Splunk Enterprise Security

Follow these steps to format your asset and identity data as a lookup:

1. Create a plain text, CSV-formatted file with Unix line endings and a .csv file extension.
2. Use the correct headers for the CSV file. See Asset and identity lookup configurations for the headers expected by Splunk Enterprise Security.
3. Populate the rows of the CSV with the asset or identity fields. The maximum number of characters per value in a field is 975. For a multivalue field, each value in the list can be 975 characters. See Asset and identity lookup configurations for reference.

For an example asset list, review the Demonstration Assets lookup. Locate the list in Splunk Web:

• In the Splunk Enterprise Security app, select Security content then Content management.
• Locate the list in the file system. The demo_assets.csv file is located in the SA-IdentityManagement/lookups/ directory.

If you use a custom search to generate a lookup, make sure that the lookup produced by the search results contains fields that match the headers.

Asset and identity lookup configurations

Enterprise Security manages specific props.conf settings as part of the asset and identity framework. In order for these files to be configured properly, all configurations need to be populated in the SPLUNK_HOME/etc/apps/SA-IdentityManagement/local/props.conf file. If there are existing identity correlation lookup definitions in the SPLUNK_HOME/etc/apps/SA-IdentityManagement/default/props.conf file, remove them so they can be managed by the asset and identity framework.

Asset lookup header

ip,mac,nt_host,dns,owner,priority,lat,long,city,country,bunit,category,pci_domain,is_expected,should_timesync,should_update,requires_av,cim_entity_zone

Asset lookup fields Populate the following fields in an asset lookup.

To add multi-homed hosts or devices to the asset list, add each IP address to the ip field for the host, pipe-delimited. Multi-homed support is limited, and having multiple hosts with the same IP address on different network segments can cause conflicts in the merge process.

Field	Data type	Description	Example values
ip	pipe-delimited numbers	A pipe-delimited list of single IP address or IP ranges. An asset is required to have an entry in at least one of the key fields such as: ip, mac, nt_host, or dns fields. All of the key fields are multi-value fields.	2.0.0.0/8|1.2.3.4&#192.168.15.9-192.168.15.27|5.6.7.8|10.11.12.13
mac	pipe-delimited strings	A pipe-delimited list of MAC address. An asset is required to have an entry in at least one of the key fields such as: ip, mac, nt_host, or dns fields. All of the key fields are multi-value fields.	00:25:bc:42:f4:60|00:50:ef:84:f1:21|00:50:ef:84:f1:20
nt_host	pipe-delimited strings	A pipe-delimited list of Windows machine names. An asset is required to have an entry in at least one of the key fields such as: ip, mac, nt_host, or dns fields. All of the key fields are multi-value fields.	ACME-0005|SSPROCKETS-0102|COSWCOGS-013
dns	pipe-delimited strings	A pipe-delimited list of DNS names. An asset is required to have an entry in at least one of the key fields such as: ip, mac, nt_host, or dns fields. All of the key fields are multi-value fields.	acme-0005.corp1.acmetech.org|SSPROCKETS-0102.spsp.com|COSWCOGS-013.cwcogs.com
owner	string	The user or department associated with the device	f.prefect@acmetech.org, DevOps, Bill
priority	string	Recommended. The priority assigned to the device for calculating the Urgency field for findings on the analyst queue. An "unknown" priority reduces the assigned Urgency by default.	unknown, low, medium, high or critical.
lat	string	The latitude of the asset in decimal degrees, using +/- to indicate direction.	37.780080
long	string	The longitude of the asset in decimal degrees, using +/- to indicate direction.	-122.420170
city	string	The city in which the asset is located	Chicago
country	string	The country in which the asset is located	USA
bunit	string	Recommended. The business unit of the asset. Used for filtering by dashboards in .	EMEA, NorCal
category	pipe-delimited strings	Recommended. A pipe-delimited list of logical classifications for assets. Used for asset and identity correlation and categorization. See Asset/Identity Categories.	server|web_farm|cloud
pci_domain	pipe-delimited strings	A pipe-delimited list of PCI domains. See Configure assets in the Splunk App for PCI Compliance Installation and Configuration Manual.	cardholder, trust|dmz, untrust If left blank, defaults to untrust.
is_expected	boolean	Indicates whether events from this asset should always be expected. If set to true, the Expected Host Not Reporting detection performs an adaptive response action when this asset stops reporting events.	"true", or blank to indicate "false"
should_timesync	boolean	Indicates whether this asset must be monitored for time-sync events. It set to true, the Should Timesync Host Not Syncing detection performs an adaptive response action if this asset does not report any time-sync events from the past 24 hours.	"true", or blank to indicate "false"
should_update	boolean	Indicates whether this asset must be monitored for system update events.	"true", or blank to indicate "false"
requires_av	boolean	Indicates whether this asset must have anti-virus software installed.	"true", or blank to indicate "false"
cim_entity_zone	string	Required when entity zones are turned on. Lowercase word to use as a default zone name. For use in situations when you have multiple private IP spaces. This word auto-populates in the cim_entity_zone fields if you do not specify your own values when formatting an asset or identity list as a lookup.	my_zone

Identity lookup header

identity,prefix,nick,first,last,suffix,email,phone,managedBy,priority,bunit,category,watchlist,startDate,endDate,work_city,work_country,work_lat,work_long,cim_entity_zone

Identity lookup fields

Field	Data type	Description	Example
identity	pipe-delimited strings	Required. A pipe-delimited list of username strings representing the identity. After the merge process completes, this field includes generated values based on the identity lookup configuration settings.	a.vanhelsing|abraham.vanhelsing|a.vanhelsing@acmetech.org
prefix	string	Prefix of the identity.	Ms., Mr.
nick	string	Nickname of an identity.	Van Helsing
first	string	First name of an identity.	Abraham
last	string	Last name of an identity.	Van Helsing
suffix	string	Suffix of the identity.	M.D., Ph.D
email	string	Email address of an identity.	a.vanhelsing@acmetech.org
phone	pipe-delimited strings	A pipe delimited field for telephone number of an identity.	123-456-7890
managedBy	string	A username representing the manager of an identity.	phb@acmetech.org
priority	string	Recommended. The priority assigned to the identity for calculating the Urgency field for findings on the analyst queue. An "unknown" priority reduces the assigned Urgency by default.	unknown, low, medium, high or critical.
bunit	string	Recommended. A group or department classification for identities. Used for filtering by dashboards in .	Field Reps, ITS, Products, HR
category	pipe-delimited strings	Recommended. A pipe-delimited list of logical classifications for identities. Used for asset and identity correlation and categorization. See Asset/Identity Categories.	Privileged|Officer|CISO
watchlist	boolean	Marks the identity for activity monitoring.	Accepted values: "true" or empty. See User Activity Monitoring in the Use Splunk Enterprise Security manual.
startDate	string	The start or hire date of an identity.	Formats: %m/%d/%Y %H:%M, %m/%d/%y %H:%M, %s
endDate	string	The end or termination date of an identity.	Formats: %m/%d/%Y %H:%M, %m/%d/%y %H:%M, %s
work_city	string	The primary work site City for an identity.
work_country	string	The primary work site Country for an identity.
work_lat	string	The latitude of primary work site City in decimal degrees, using +/- to indicate direction.	37.780080
work_long	string	The longitude of primary work site City in decimal degrees using +/- to indicate direction.	-122.420170
cim_entity_zone	string	Required when entity zones are turned on. Lowercase word to use as a default zone name. For use in situations when you have multiple private IP spaces. This word auto-populates in the cim_entity_zone fields if you do not specify your own values when formatting an asset or identity list as a lookup.	my_zone

Configure a new asset or identity list

Configure a new asset or identity lookup in Splunk Enterprise Security. This multistep process adds the lookup in Splunk Enterprise Security and defines the lookup for the merge process.

Prerequisites Collect and extract asset and identity data in Splunk Enterprise Security Format the asset or identity list as a lookup in Splunk Enterprise Security. Assets and identities framework supports only exact-matching of IPv6 addresses.

Steps

1. Add the new lookup table file
2. Set permissions on the lookup table file to share it with Splunk Enterprise Security
3. Add a new lookup definition
4. Set permissions on the lookup definition to share it with Splunk Enterprise Security

Add the new lookup table file

These lookup table files are consumed by the asset and identity framework and merged together. The product of the merge is called an "expanded lookup."

1. From the Splunk menu bar, select Settings > Lookups > Lookup table files.
2. Select New.
3. Select a Destination App of SA-IdentityManagement.
4. Select the lookup file to upload.
5. Type the Destination filename that the lookup table file should have on the search head. The name should include the filename extension.
   For example, network_assets_from_CMDB.csv
6. Select Save to save the lookup table file and return to the list of lookup table files.

In a distributed environment, these lookup table files are not replicated from the search heads to the indexers. Only the expanded lookup is replicated to the indexers. However, these lookup files are still replicated between search heads. If an asset or identity lookup table file grows in excess of 1GB+, it should be broken down into smaller files (for example, by location or by type or by easily identifiable category). When making changes to lookup files, only the updated files are replicated across search heads, reducing bundle sizes.

Set permissions on the lookup table file to share it with Splunk Enterprise Security

1. From Lookup table files, locate the new lookup table file and select Permissions.
2. Set Object should appear in to All apps.
3. Set Read access for Everyone.
4. Set Write access for admin or other roles.
5. Select Save.

Add a new lookup definition

1. From the Splunk menu bar, select Settings > Lookups > Lookup definitions.
2. Select New.
3. Select a Destination App of SA-IdentityManagement.
4. Type a name for the lookup source. This name must match the name defined later in the input stanza definition on the Identity Management dashboard.
   For example, network_assets_from_CMDB.
5. Select a Type of File based.
6. Select the lookup table file created.
   For example, select network_assets_from_CMDB.csv.
7. Select Save.

Set permissions on the lookup definition to share it with Splunk Enterprise Security

1. From Lookup definitions, locate the new lookup definition and select Permissions.
2. Set Object should appear in to All apps.
3. Set Read access for Everyone.
4. Set Write access for admin or other roles.
5. Select Save.