Splunk® Enterprise Security

Administer Splunk Enterprise Security

Acrobat logo Download manual as PDF


This documentation does not apply to the most recent version of Splunk® Enterprise Security. For documentation on the most recent version, go to the latest release.
Acrobat logo Download topic as PDF

Manage assets and identities in Splunk Enterprise Security

Use the Asset and Identity Management page to enrich and manage asset and identity data using lookups. The Asset and Identity Management interface replaces the previously separate menus for Identity Management, Identity Correlation, and Identity Lookup Configuration. You need to have the edit_modinput_identity_manager capability to use it. See Configure users and roles in the Installation and Upgrade Manual.

When the identity manager runs, it processes all of the asset and identity input configurations that have changed. If the source has been updated, identity manager dispatches the SPL created by a custom-built search.

The SPL search uses a custom search command that handles the merging and updating of new data to existing data. The custom search command merges data based on key fields and policies that you define here.

Assets and identities that need to be deleted are updated in the KV store with a _delete flag set to True so that the delete operation can persist and be completed at a later time.

The custom search command returns the merged data, which is updated or inserted to the KV store using outputlookup append=T. The identity manager checks and processes rows that are marked for deletion.


If you have customized the menu bar in Splunk Enterprise Security, the Asset and Identity Management navigation and page do not display. See Restore the default navigation to restore.

Prerequisites

Perform the following prerequisite tasks before starting the tasks in this topic:

  1. Collect and extract asset and identity data in Splunk Enterprise Security.
  2. Format the asset or identity list as a lookup in Splunk Enterprise Security.
  3. Configure a new asset or identity list in Splunk Enterprise Security.

Create an asset lookup configuration policy to update and enrich your assets

The asset lookup configuration settings create the policy that updates the inputs.conf file to point to a lookup and update your assets. When you add new items or update current items, the change takes effect in 5 minutes.

Add an asset input stanza for the lookup source

To add a new asset input source, complete the following steps:

  1. From the Splunk Enterprise Security menu bar, select Configure > Data Enrichment > Asset and Identity Management.
  2. Click the Asset Lookup Configuration tab.
  3. Click New.
  4. In the New Asset Manager, do the following:
    1. Select the transforms.conf definition from the Source drop-down list that corresponds to the CSV source file of assets you uploaded in the prerequisite step.
    2. You can provide a name for the asset list stanza, but matching the source file name is a good idea.
    3. Enter a descriptive category for this asset list, such as web_servers or west_coast_servers.
    4. Enter a detailed description of the contents of this asset list.
    5. Check the Blacklist check box to exclude the lookup file from bundle replication.

      The asset and identity source lookup files are excluded from bundle replication in an indexer cluster by default. The merged lookup files are still included in bundle replication to support asset and identity correlation. Changing the default to include asset and identity lookup files in bundle replication might reduce system performance. See Knowledge bundle replication overview in the Splunk Enterprise Distributed Search manual.

    6. In Lookup List Type, asset is selected for you.
    7. In Lookup Field Exclusion List, select fields for the merge process to ignore. This excludes the fields and those values from the KV store collections for that particular lookup. You might use this in the case where you have a field in your source file that you don't want to rely on for information.
    8. Click Save.

Rank the order for merging assets

Any new asset list gets added to the bottom of the list by default. You can rank the order of this list to determine priority for merging assets. If an asset exists in multiple source files as a single value or exists multiple times in the same source file, this ranking is the weighted order for merging them. By default, the single value asset fields are as follows:

  • is_expected
  • priority
  • requires_av
  • should_timesync
  • should_update

These are the fields where the rank takes effect. For example, If you're merging two assets and they both have the is_expected field value, you need to choose one to take precedence. The row at the top of the list takes precedence and the merge process uses that value, as opposed to the row that's ranked second.

To change the rank, do the following from the Asset Lookup Configuration tab:

  1. Drag and drop the rows of the table into a new order.
  2. When finished reordering, click Save Ranking.

Ranking is not considered for a multivalue field field. The merge process combines all the values into the field, and then removes the duplicates.

Key fields are dns, ip, mac, and nt_host. If you store extra information in your key fields, such as the same IP address assigned to multiple systems, these duplicate IP addresses are now merged together as one asset. Make sure that the information in your key fields either belongs to the same asset or does not overlap.

Disable or enable asset lookups

You can disable or enable an asset lookup input. Disabling an input does not delete the data from the associated lookup from Splunk Enterprise Security. Disabling prevents the contents of the corresponding list from being included in the merge process. Enabling a disabled input allows the associated list to be merged at the next scheduled merge of the asset or identity data.

To disable an asset lookup, do the following from the Asset Lookup Configuration tab:

  1. Navigate to the Status column.
  2. Do one of the following options:
    • Click Disable to disable an input.
    • Click Enable to enable a disabled input.

Starting with version 5.0.0, asset and identity lookup inputs are disabled by default after a new installation. However, local settings are respected after an upgrade.

Configure asset settings for lookup matching

You can add a new asset field or enable case sensitive matching.

Add a new asset field

Asset fields are added both by default and by entering custom fields manually. You can add up to 20 custom fields for your lookups. Key fields, such as dns, ip, mac, nt_host are non-editable. However, for custom and default fields you can configure whether the field is a tag field, a multivalue field, or both.

To add a new custom asset field, do the following:

  1. From the Splunk Enterprise Security menu bar, select Configure > Data Enrichment > Asset and Identity Management.
  2. Click the Asset Settings tab.
  3. Click Add New Field.
  4. In the New Asset Field dialog box, do the following:
    1. Enter a field name.
    2. Check the Multivalue check box if the field can output multiple values.
    3. Check the Tag check box if the field can be used as an asset tag. This is a helper field for holding additional values that you want to look up, in addition to the key fields. This is not the same as tagging in Splunk Enterprise.
    4. (Optional) Revise the Limit if you want to change the number of values that display in a multivalue field merge. See Revise field limits for assets.
    5. Click Save.

The Save button is disabled when the limit is reached and is enabled again when any custom field is deleted using the Delete action link.

Enable case-sensitive matching for asset fields

Case sensitive matching is globally available across all fields.

Note that searches using | inputlookup ... where <filter> are case sensitive. Asset and Identity Management pages might use searches that contain where clauses. When case sensitivity is set to false, the merge process stores the values as lowercase so that case insensitive matches can be performed. To avoid this, you can toggle the case sensitive settings to true.

To use case-sensitive matching, do the following:

  1. From the Splunk Enterprise Security menu bar, select Configure > Data Enrichment > Asset and Identity Management.
  2. Click the Asset Settings tab.
  3. Enable the Enable case sensitive asset matching switch.
  4. Click Update to trigger the merge process and rewrite the asset_lookup_by_str and asset_lookup_by_cidr KV store collections.

Revise multivalue field limits for assets

The default number of values in a multivalue asset field that is displayed after merging is 6 for key fields and 25 for non-key fields.

To revise multivalue field limits, perform the following steps:

  1. From the Splunk Enterprise Security menu bar, select Configure > Data Enrichment > Asset and Identity Management.
  2. Click the Asset Settings tab.
  3. Scroll to find the field name that you're looking for and do the following:
    1. Click on the link.
    2. Change the Field Limit value.
  4. Click Save.

The field value range for a non-key multivalue field is 1 - 100. The field value range for a key multivalue field is 1 - 25. The reason that the default multivalue key field limit is 6 for assets is because there are 4 key fields. If each key field contains 6 values, the merge process results in an asset field with 24 key values. Performance issues can occur when a resulting asset field contains 25 key values. You can set a key multivalue field to 25, but performance issues can also occur if multiple key fields have 25 values.

If your source CSV file contains more values in a multivalue field than the limit, these values are truncated during the merge process. This means that in addition to not being displayed in the results, they also are removed from the data altogether. If you search or lookup on the truncated values, you will not find them because they do not exist.

If your data gets truncated, you can revise key multivalue fields to 25, and non-key multivalue fields to 100. Raising the limits has the potential to impact performance.

If your data still gets truncated, but you want to see more than the maximum values, then you need to revise your source CSV files to spread out those values so that they seem to be part of different assets, by making sure that there are no duplicate values in the key fields.

Key fields are dns, ip, mac, and nt_host. If you store extra information in your key fields, such as the same IP address assigned to multiple systems, these duplicate IP addresses are now merged together as one asset. Make sure that the information in your key fields either belongs to the same asset or does not overlap.

Example of revising multivalue field limits

As an example, you have a source CSV file that contains 9 values in the mac key field and 7 values in the bunit field, such as the following:

ip,mac,nt_host,dns,owner,priority,lat,long,city,country,bunit,category,pci_domain,is_expected,should_timesync,should_update,requires_av
192.0.2.2,mac1|mac2|mac3|mac4|mac5|mac6|mac7|mac8|mac9,host1,dns1,owner1,,,,,,bunit1|bunit2|bunit3|bunit4|bunit5|bunit6|bunit7,,,,,,

Using the default limit of 6 for the mac multivalue key field and revising the limit to 5 for the bunit multivalue field, these are merged into an asset where the mac key field values are truncated to 6 and the bunit non-key values are truncated to 5.

bunit pci_domain nt_host ip asset asset_tag mac dns owner

bunit1
bunit2
bunit3
bunit4
bunit5

untrust host1 192.0.2.2

dns1
192.0.2.2
mac1
mac2
mac3
mac4
mac5
mac6
host1

bunit1
bunit2
bunit3
bunit4
bunit5

mac1
mac2
mac3
mac4
mac5
mac6
mac7
mac8
mac9

dns1 owner1

Create an identity lookup configuration policy to update and enrich your identities

Identity lookup settings create the configuration that updates the inputs.conf file to point to a lookup and update your identities. When you add new items, or update current items, the change takes effect in 5 minutes.

Add an identity input stanza for the lookup source

To add a new identity input source, do the following:

  1. From the Splunk ES menu bar, select Configure > Data Enrichment > Asset and Identity Management.
  2. Click the Identity Lookup Configuration tab.
  3. Click New.
  4. In the New Identity Manager, do the following:
    1. Select the transforms.conf definition from the Source drop-down list that corresponds to the CSV source file of assets you uploaded in the prerequisite step.
    2. You can provide a name for the identity list stanza, but matching the source name is a good idea.
    3. Enter a descriptive category for this identity list, such as east_coast_employees or strategic_executives.
    4. Enter a detailed description of the contents of this identity list.
    5. Check the Blacklist check box to exclude the lookup file from bundle replication.

      The asset and identity source lookup files are excluded from bundle replication in an indexer cluster by default. The merged lookup files are still included in bundle replication to support asset and identity correlation. Changing the default to include asset and identity lookup files in bundle replication might reduce system performance. See Knowledge bundle replication overview in the Splunk Enterprise Distributed Search manual.

    6. In Lookup List Type, identity is selected for you.
    7. In Lookup Field Exclusion List, select fields for the merge process to ignore. This excludes the values from the KV store collections. This excludes the fields and those values from the KV store collections for that particular lookup. You might use this in the case where you have a field in your source file that you don't want to rely on for information.
  5. (Optional) Configure the conventions that the identity lookup can use to uniquely identify identities in your data.
    When an email convention check box is checked, the email address is used as an additional primary key for identity. The Email and Email Short conventions are enabled by default.
    1. Click Email to use the full email address.
    2. Click Email Short to use the email username.
    3. Click + Add a new convention to add a custom convention:
      You can identify users by the first few letters of their first name and the first few letters of their last name, based on the columns in the Identities Table. Use the convention of identity_first(n)middle(n)last(n) where identity, first, and last are any columns from the Identities Table, and where n is a number starting with 0. For example:
      • "Claudia Maria Garcia" using the convention first(3)last(3) is "clagar"
      • "Rutherford Michael Sullivan" using the convention first(1)middle(1).last() is "rm.sullivan"
      • "Vanya Patel" using the convention ADMIN_first(1)last() is "ADMIN_vpatel"
      • Multiple matches are resolved automatically by taking the first match in the table or manually by specifying identity values.
  6. Click Save.

Rank the order for merging identities

Any new identity list gets added to the bottom of the page by default. You can rank the order of this list to determine priority for merging identities. If an identity exists in multiple source files as a single value, or exists multiple times in the same source file, this ranking is the weighted order for merging them. By default, the single value identity fields are as follows:

  • endDate
  • priority
  • startDate
  • watchlist

These are the fields where the rank takes effect. For example, if you're merging two identities, that both have the priority field value, you need to choose one to take precedence. The row at the top of the list takes precedence and the merge process uses that value, as opposed to the row that's ranked second.

To change the rank, do the following under the Identity Lookup Configuration tab:

  1. Drag and drop the rows of the table into a new order.
  2. When finished reordering, click Save Ranking.

Ranking is not considered for a multivalue field. The merge process combines all the values into the field, and then removes the duplicates.

Configure identity settings for lookup matching

Identity fields are added both by default and by entering custom fields manually. You can add up to 20 custom fields for your lookups. Key fields, such as identity are non-editable. However, for custom and default fields you are able to configure whether the field is a tag field, a multivalue field, or both.

Add a new identity field

To add a new custom identity field, do the following:

  1. From the Splunk ES menu bar, select Configure > Data Enrichment > Asset and Identity Management.
  2. Click the Identity Settings tab.
  3. Click Add New Field.
  4. In the New Identity Field window, do the following:
    1. Enter a lookup field name.
    2. Check the Multivalue check box if the field can output multiple values.
    3. Check the Tag check box if the field can be used as an identity tag. This is a helper field for holding additional values that you want to look up, in addition to the key fields. This is not the same as tagging in Splunk Enterprise.
    4. Click Save.

The Add New Field' button is disabled when the limit is reached and enabled again when any custom field is deleted using the Delete action link.

Enable case-sensitive matching for identity fields

Case-sensitive matching is globally available across all fields.

Note that searches using | inputlookup ... where <filter> are case sensitive. Asset and Identity Management pages might use searches that contain where clauses. When case sensitivity is set to false, the merge process stores the values as lowercase so the case insensitive matches can be performed. To avoid this, you can toggle the case sensitive settings to true.

To use case-sensitive matching, do the following:

  1. From the Splunk Enterprise Security menu bar, select Configure > Data Enrichment > Asset and Identity Management.
  2. Click the Identity Settings tab.
  3. Enable the Enable case sensitive identity matching switch.
  4. Click Update to trigger the merge process and rewrite the identity_lookup_expanded KV store collection.

Revise multivalue field limits for identities

The default number of multivalue identity fields that display after merging is 25.

To revise multivalue field limits, do the following:

  1. From the Splunk Enterprise Security menu bar, select Configure > Data Enrichment > Asset and Identity Management.
  2. Click the Identity Settings tab.
  3. Scroll to find the field name that you're looking for and do the following:
    1. Click on the link.
    2. Change the Field Limit value.
  4. Click Save.

The field value range for both key and non-key multivalue fields is 1 - 100.

If your source CSV file contains more values in a multivalue field than the limit, these values are truncated during the merge process. This means that in addition to not being displayed in the results, they also are removed from the data altogether. If you search or lookup on the truncated values, you will not find them because they do not exist.

If your data gets truncated, you can revise multivalue fields to 100. Raising the limits has the potential to impact performance.

If your data still gets truncated, but you want to see more than the maximum values, then you need to revise your source CSV files. Spread out the values so that they seem to be part of different assets, by making sure that there are no duplicate values in the key fields.

The key field is identity and the default merge convention is email. If you store extra information in your key fields, such as the same identity or email address assigned to multiple people, these duplicates are now merged together as one identity. Make sure that the information in your key or email fields either belongs to the same person or does not overlap.

Example of revising multivalue field limits

If you have a source CSV file that contains 9 values in the identity key field and 16 values in the phone field, such as the following:

identity prefix first last email phone managedBy priority watchlist startDate
journot Dr. Latoyia Journot ljournot@acmetech.com +1 (800)555-3479 medium americas 3/2/88 2:39 3/8/01 6:21
dr.j Dr. Latoyia Journot ljournot@acmetech.com +1 (800)555-1554 medium americas 3/2/88 2:39 3/8/01 6:21
Dr.L Dr. Latoyia Journot ljournot@acmetech.com +1 (800)555-3480 |
+1 (800)555-1555
medium americas 3/2/88 2:39 3/8/01 6:21
Latoyia.Journot Dr. Latoyia Journot ljournot@acmetech.com +1 (800)555-3481 |
+1 (800)555-1556
medium americas 3/2/88 2:39 3/8/01 6:21
Latoyia.J Dr. Latoyia Journot ljournot@acmetech.com +1 (800)555-3482 |
+1 (800)555-1557
medium americas 3/2/88 2:39 3/8/01 6:21
L.Journot Dr. Latoyia Journot ljournot@acmetech.com +1 (800)555-3483 |
+1 (800)555-1558
medium americas 3/2/88 2:39 3/8/01 6:21
Latoyia Dr. Latoyia Journot ljournot@acmetech.com +1 (800)555-3484 |
+1 (800)555-1559
medium americas 3/2/88 2:39 3/8/01 6:21
toyia Dr. Latoyia Journot ljournot@acmetech.com +1 (800)555-3485 |
+1 (800)555-1560
medium americas 3/2/88 2:39 3/8/01 6:21
dr.toyia Dr. Latoyia Journot ljournot@acmetech.com +1 (800)555-3486 |
+1 (800)555-1561
medium americas 3/2/88 2:39 3/8/01 6:21

Using the default email convention, the default limit of 6 for the identity multivalue key field, and revising the limit to 5 for the phone multivalue field, these are merged into an asset where the identity key field values are truncated to 6 and the phone non-key values are truncated to 5.

email startDate identity_tag last first managedBy prefix identity priority watchlist phone
ljournot@acmetech.com 984050460.000000 3/2/88 2:39 journot latoyia medium dr.

dr.l
ljournot@acmetech.com
ljournot
l.journot
latoyia.journot
latoyia.j

americas 3/2/88 2:39

+1 (800)555-3480
+1 (800)555-1555
+1 (800)555-3483
+1 (800)555-1558
+1 (800)555-3481

Global Settings

Configure the global settings of the identity manager modular input to revise the way the identity manager works by default.

Enable merge for assets or identities

The merge process is enabled for assets and identities by default. However, in situations when you have a source file with duplication in the key fields, and you can't groom the file to make sure that the information belongs to the same asset or identity, then you have the option to disable the merge process.

Use the global settings to enable or disable merge as follows:

  1. From the Splunk Enterprise Security menu bar, select Configure > Data Enrichment > Asset and Identity Management.
  2. Click the Global Settings tab.
  3. Scroll to the Enable Merge for Assets or Identities panel.
  4. Use the toggle to enable or disable for Assets or Identities.

Using assets as an example, consider a source file with duplicates in the key field of nt_hosts, such as the following: ip,mac,nt_hosts,dns,owner,priority,lat,long,city,country,bunit,category,pci_domain,is_expected,should_timesync,should_update,requires_av
192.0.2.2,,host1,,,,,,,,,,,,,,
192.0.2.120,,host1,,,,,,,,,,,,,,
192.0.2.135,,host1,,,,,,,,,,,,,,
192.0.2.242,,host2,,,,,,,,,,,,,,
192.0.2.65,,host2,,,,,,,,,,,,,,

The default is to merge the three rows with nt_hosts of host1 into one asset, and merge the two rows with host2 into another asset.

asset ip nt_hosts pci_domain
192.0.2.2

192.0.2.120
192.0.2.135
host1

192.0.2.2

192.0.2.120
192.0.2.135

host1 untrust
192.0.2.242

192.0.2.65
host2

192.0.2.242

192.0.2.65

host2 untrust

If you disable the merge, then the collection remains the same as the source file, and assets are not merged.

asset ip nt_hosts pci_domain
192.0.2.2

host1

192.0.2.2 host1 untrust
192.0.2.120

host1

192.0.2.120 host1 untrust
192.0.2.135

host1

192.0.2.135 host1 untrust
192.0.2.242

host2

192.0.2.242 host2 untrust
192.0.2.65

host2

192.0.2.65 host2 untrust

When you do a lookup on an non-merged collection, there is no context for how to resolve the overlapping key field values. For example, the asset_lookup_by_str lookup in transforms.conf has max_matches = 1, so the first host it matches in the assets_by_str collection is the only one you'll see in your search results.

Ignored values for Assets or Identities

In situations when you want values to be ignored in your fields, you might want to use special words to represent null values. The default behavior is to merge rows of source data based on a match in any one of the key fields. In many cases your source data might have placeholder values that span multiple rows, which causes them to get merged into one large multivalue row. To avoid this, you can define the placeholder values, and clean them during the merge process, so that independent rows are still maintained in the final lookups.

Set null values

Use the global settings to set your null values as follows:

  1. From the Splunk Enterprise Security menu bar, select Configure > Data Enrichment > Asset and Identity Management.
  2. Click the Global Settings tab.
  3. Scroll to the Asset Ignored Values tab or the Identity Ignored Values tab.
    The default values that are ignored are null, n/a, unknown, and undefined.
    1. For assets, in the Asset Ignored Values section, click Add Row.
    2. Type a lowercase word that you want ignored and not displayed in the merge results. Does not support having a capital letter because all values are squashed first and then checked for ignored values.
    3. For identities, in the Identity Ignored Values section, click Add Row.
    4. Type a lowercase word that you want ignored and not displayed in the merge results. Does not support having a capital letter because all values are squashed first and then checked for ignored values.
  4. Click Save.

The ignored values setting applies to any type of field, such as multivalue field or single value field or key field or non-key field. The strings are saved as ignored_values in SplunkHome/etc/apps/SA-IdentityManagement/local/inputs.conf.

Remove null values

Use the global settings to remove your null values as follows:

  1. From the Splunk Enterprise Security menu bar, select Configure > Data Enrichment > Asset and Identity Management.
  2. Click the Global Settings tab.
    The default values that are ignored are null, n/a, unknown, and undefined.
  3. Scroll to the Asset Ignored Values tab or the Identity Ignored Values tab.
  4. Find the value and click the x to delete it.

Revise the enforcements used by the identity manager framework

Every five minutes when the identity manager runs, it automatically enforces configuration file settings used by the framework, including inputs.conf, props.conf, macros.conf, transforms.conf, and identityLookup.conf (deprecated).

With these enforcements enabled, if there are accidental changes made to your conf files, the settings are reverted back to the way they were. If you're doing manual testing or making changes on purpose to your conf files and you do not want the settings checked or reverted back, you can disable these enforcements.

Use the global settings to enable or disable enforcements as follows. For the majority of users who configure settings through the Splunk Web UI, there is no need to disable these settings:

  1. From the Splunk Enterprise Security menu bar, select Configure > Data Enrichment > Asset and Identity Management.
  2. Click the Global Settings tab.
  3. Scroll to the Enforcements panel.
  4. Use the toggle to enable or disable.

Using the example of Enforce props, you experience the following by default. If you add a custom field in Identity Settings, the field is automatically added to the props.conf file because the settings check occurs to sync and reload props to be consistent with the identity manager.

Using the example of Enforce props, you experience the following by disabling it. If you add a custom field in Identity Settings, then you have to add that custom field to the props.conf file manually because the settings check no longer occurs. With enforce props disabled, any manual identity settings changes made without using the Splunk Web UI are also ignored.

After upgrading to Enterprise Security 6.2.0, you need to enable the Enforce props setting if you want the identity manager to automatically enforce configuration file settings. On a fresh installation, Enterprise Security 6.2.0 has Enforce props set to enabled by default and the setting is enforced continuously. However, prior versions only enforce once and then switch the setting to false right away. If you're already using a previous version of ES with assets and identities, the /local/inputs.conf file already has enforce_props=false and it needs to be set back to true after you upgrade, if you want to ensure that settings are managed for you. The majority of users who configure settings through the Splunk Web UI will benefit from enabling the setting.

Revise the miscellaneous settings used by the identity manager framework

You can revise miscellaneous settings that are specific to the identity manager.

Revise how often the identity manager runs

The identity manager runs every 300 seconds (5 minutes) by default. For performance purposes, you can change this to a larger value so it does not run so frequently.

Use the global settings to change the time:

  1. From the Splunk Enterprise Security menu bar, select Configure > Data Enrichment > Asset and Identity Management.
  2. Click the Global Settings tab.
  3. Scroll to the Miscellaneous Settings panel.
  4. Type a number of seconds in the Time(s) field.

Revise the master host where the identity manager runs

The identity manager runs on the search head captain by default. If you want to separate search head responsibilities, or if the search head is experiencing performance issues due to resource consumption, then you can change the master host.

Use the global settings to change the master host if search head clustering is enabled:

  1. From the Splunk Enterprise Security menu bar, select Configure > Data Enrichment > Asset and Identity Management.
  2. Click the Global Settings tab.
  3. Scroll to the Miscellaneous Settings panel.
  4. Type a name in the Master host field that matches the name of a server in the cluster pool.

See System requirements and other deployment considerations for search head clusters.

Enable correlation setup to compare indexed events with asset and identity data

When asset and identity correlation is enabled, Splunk Enterprise Security compares indexed events with asset and identity data in the asset and identity lists to provide data enrichment and context. The comparison process uses automatic lookups in the props.conf file. You can find information about automatic lookups in the Splunk platform documentation:

Asset and identity correlation enriches events with asset and identity data at search time in the following ways:

  • Asset correlation compares events that contain data in any of the src, dest, or dvc fields against the merged asset lists for matching IP address, MAC address, DNS name, or Windows NT host names. Asset correlation no longer occurs automatically against the host or orig_host fields.
  • Identity correlation compares events that contain data in any of the user or src_user fields against the merged identity lists for a matching identity.
  • Enterprise Security adds the matching output fields to the event. For example, correlation on the asset src field results in additional fields such as src_is_expected and src_should_timesync.

Asset and identity correlation lets you determine whether multiple events can relate to the same asset or identity. You can also perform actions on the identity and asset fields added to events to open additional searches or dashboards scoped to the specific asset or identity. For example, you can open the Asset Investigator dashboard on a src field.

Choose whether to enable asset and identity correlation, disable it, or restrict correlation to occur only for select source types. If in doubt, keep asset and identity correlation enabled.

Disabling asset and identity correlation completely prevents events from being enriched with asset and identity data from the asset and identity lookups. This might prevent correlation searches, dashboards, and other functionality from working as expected. Consult with Splunk Professional Services or Splunk Support before disabling asset and identity correlation.

You can choose from the following options:

  • Enable for all sourcetypes
  • Disable for all sourcetypes
  • Enable selectively by sourcetype

To enable or disable for all sourcetypes, do the following steps:

  1. From the Splunk Enterprise Security menu bar, select Configure > Data Enrichment > Asset and Identity Management.
  2. Click the Correlation Setup tab.
  3. Do one of the following options:
    • Click the Enable for all sourcetypes radio button.
    • Click the Disable for all sourcetypes radio button.
  4. Click Save.

To enable selectively by sourcetype, do the following steps:

  1. From the Splunk Enterprise Security menu bar, select Configure > Data Enrichment > Asset and Identity Management.
  2. Click the Correlation Setup tab.
  3. Click the Enable selectively by sourcetype radio button.
  4. Click + Add a new sourcetype.
  5. Enter the name of the sourcetype.
  6. Toggle Enable asset correlation or Enable identity correlation.
  7. Click Done.
  8. Click Save.

See Modify priority and rank in the Asset and Identity Framework in the Use Splunk Enterprise Security manual for further information about how ranks, correlations, and automatic lookups affect notable event urgency.

Use the search preview to test the merge process

You can test the asset and identity merge process if you want to confirm that the data produced by the merge process is expected and accurate. You can run the search previews to determine what the merge will do with your data without actually performing the merge. These steps aren't required, but can be performed to validate the merge works as expected.

If you used previous versions of ES, note that the search preview shows you the dynamic custom search that replaces the following correlation searches:

  • Identity - Asset CIDR Matches - Lookup Gen
  • Identity - Asset String Matches - Lookup Gen
  • Identity - Identity Matches - Lookup Gen

To preview all your asset and identity searches, do the following:

  1. From the Splunk Enterprise Security menu bar, select Configure > Data Enrichment > Asset and Identity Management.
  2. Click the Search Preview tab.
  3. From each drop-down list, you can run the search preview for each collection, the lookups of which are located in the transforms.conf file:
    • asset_lookup_by_str is the lookup for the assets_by_str collection.
    • asset_lookup_by_cidr is the lookup for the assets_by_cidr collection.
    • identity_lookup_expanded is the lookup for the identities_expanded collection.

The search preview looks into all your lookup tables and creates custom-built searches with what is currently in your inputs.conf file. The search is dynamic and generates the search each time you refresh or load the page. If nothing has changed in the source files since the last merge, you do not see any output.

If you want to see some output regardless if anything has changed, you can remove the inputlookup append=T SPL from the search. For example, in the case of identities, you would remove: | inputlookup append=T "identity_lookup_expanded".

Reset your collections immediately

All the asset and identity source files that are enabled in the Asset and Identity Management page get merged into the following default collections in the collections.conf file: assets_by_str, assets_by_cidr, or identities_expanded.

If your collections get into an undesirable state, you can reset your collections at any time, rather than waiting for the automated process to clear out the KV store collection. It's similar to clearing cache manually.

  1. From the Splunk Enterprise Security menu bar, select Configure > Data Enrichment > Asset and Identity Management.
  2. Click Reset Collections. The button is globally available regardless if you are configuring in a particular tab.

When the identity manager runs again in 5 minutes, it rebuilds the collections based on which source files are enabled in the Asset Lookup Configuration or the Identity Lookup Configuration.

Modify asset and identity lookups

Make changes to the asset and identity lookups in Splunk Enterprise Security to add new assets or identities, or change existing values in the lookup tables. You can also disable or enable existing lookups.

Edit asset and identity lookups

Edit an asset or identity lookup in the Identity Management dashboard.

  1. In Enterprise Security, select Configure > Data Enrichment > Asset and Identity Management.
  2. Find the name of the asset or identity list you want to edit, and select the corresponding lookup from the Source column. The list opens in an interactive editor.
  3. Use the scroll bars to view the columns and rows in the table. Double-click a cell to add, change, or remove content.
  4. Click Save when you are finished.

Manually add static asset or identity data

Manually add new static asset or identity data to Splunk Enterprise Security by editing the Assets or Identities lookups. For example, add internal subnets, IP addresses to allow, and other static asset and identity data.

  1. From the Splunk Enterprise Security menu bar, select Configure > Content > Content Management.
  2. To add asset data, click the Assets lookup to edit it. To add identity data, click the Identities list to edit it.
  3. Use the scroll bars to view the columns and rows in the table. Double-click in a cell to add, change, or remove content.
  4. Save your changes.

Then you can see the lookup registered as static_assets or static_identities or in Configure > Data Enrichment > Asset and Identity Management.

Disable the demo asset and identity lookups

The demo asset and identity lookups are disabled by default. Enable them if needed for testing. Disable the demo asset and identity lookups to prevent the demo data from being added to the primary asset and identity lookups used by Splunk Enterprise Security for asset and identity correlation.

  1. In Enterprise Security, select Configure > Data Enrichment > Asset and Identity Management.
  2. Locate the demo_assets and demo_identities lookups.
  3. Click Disable for each.

Revise asset and identity lookups memory usage behavior

Prior to the release of Splunk Cloud 8.0.2004, KV Store backed lookups do not respect the max_memtable_bytes setting. This means that KV Store backed lookups are always stored in memory on the indexer.

With the release of Splunk Cloud 8.0.2004, KV Store backed lookups do respect the max_memtable_bytes setting. This means that a KV Store backed lookup is stored in memory until it exceeds the definition in the max_memtable_bytes setting.

You might experience the following behavior after upgrading. Using Splunk Enterprise 8.0 as an example, consider a KV Store lookup of 1 GB in size that is used as an automatic lookup, with max_memtable_bytes=25MB. If you upgrade to a Splunk Cloud version of 8.0.2004 or higher, the 1 GB size exceeds the max_memtable_bytes setting, so an index file is created and the lookup occurs on disk, which is slower.

The default setting in Splunk Cloud is max_memtable_bytes=100MB. Splunk Cloud customers need to contact technical support if necessary to revise this behavior.

To revise this behavior in an on-premises environment, increase your max_memtable_bytes in the $SPLUNK_HOME/etc/system/local/limits.conf file. See lookup of limits.conf in the Splunk Enterprise Admin Manual.

Last modified on 26 October, 2020
PREVIOUS
Create a lookup from your current LDAP data in Splunk Enterprise Security
  NEXT
Verify that your asset and identity data was added to Splunk Enterprise Security

This documentation applies to the following versions of Splunk® Enterprise Security: 6.2.0


Was this documentation topic helpful?


You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters