Splunk® Enterprise Security

Administer Splunk Enterprise Security

This documentation does not apply to the most recent version of Splunk® Enterprise Security. For documentation on the most recent version, go to the latest release.

Manage assets and identities in Splunk Enterprise Security

Use the Asset and Identity Management page to enrich and manage asset and identity data using lookups. the Asset and Identity Management interface replaces the previously separate menus for Identity Management, Identity Correlation, and Identity Lookup Configuration. You need to have the edit_modinput_identity_manager capability to use it. See Configure users and roles in the Installation and Upgrade Manual.

When identity manager runs, it processes all of the asset and identity input configurations that have changed. If the source has been updated, identity manager dispatches the SPL created by a custom-built search.

The SPL search uses a custom search command that handles the merging and updating of new data to existing data. The custom search command merges data based on policies that you define here.

Assets and identities that need to be deleted are updated in the KV store with a _delete flag set to True so that the delete operation can persist and be completed at a later time.

The custom search command returns the merged data, which is updated or inserted to the KV store using outputlookup append=T. The identity manager checks and processes rows that are marked for deletion.


If you have customized the menu bar in Splunk Enterprise Security, the Asset and Identity Management navigation and page do not display. See Restore the default navigation to restore.

Prerequisites

Perform the following prerequisite tasks:

  1. Collect and extract asset and identity data in Splunk Enterprise Security.
  2. Format the asset or identity list as a lookup in Splunk Enterprise Security.
  3. Configure a new asset or identity list in Splunk Enterprise Security.

Create an Asset Lookup Configuration

The asset lookup configuration settings create the policy that updates the inputs.conf file to point to a lookup and update your assets. When new items are added or current items are updated, the change takes effect in 5 minutes.

Add an asset input stanza for the lookup source

To add a new asset input source, complete the following steps:

  1. From the Splunk Enterprise Security menu bar, select Configure > Data Enrichment > Asset and Identity Management.
  2. Click the Asset Lookup Configuration tab.
  3. Click New.
  4. In the New Asset Manager, do the following:
    1. Since you uploaded a CSV source file of assets in a prerequisite step, select the corresponding transforms.conf definition from the Source drop-down list.
    2. You can provide a name for the asset list stanza, but matching the source file name is a good idea.
    3. Enter a descriptive category for this asset list, such as web_servers or west_coast_servers.
    4. Enter a detailed description of the contents of this asset list.
    5. Check the Blacklist check box to exclude the lookup file from bundle replication.

      The asset and identity source lookup files are excluded from bundle replication in an indexer cluster by default. The merged lookup files are still included in bundle replication to support asset and identity correlation. Changing the default to include asset and identity lookup files in bundle replication might reduce system performance. See Knowledge bundle replication overview in the Splunk Enterprise Distributed Search manual.

    6. In Lookup List Type, asset is selected for you.
    7. In Lookup Field Exclusion List, select fields for the merge process to ignore. This excludes the fields and those values from the KV store collections for that particular lookup. You might use this in the case where you have a field in your source file that you don't want to rely on for information.
    8. Click Save.

Rank the order for merging assets

Any new asset list gets added to the bottom of the list by default. You can rank the order of this list to determine priority for merging assets. If an asset exists in multiple source files as a single value or exists multiple times in the same source file, this ranking is the weighted order for merging them. By default, the single value asset fields are as follows:

  • is_expected
  • priority
  • requires_av
  • should_timesync
  • should_update

These are the fields where the rank takes effect. For example, If you're merging two assets and they both have the is_expected field value, you need to choose one to take precedence. The row at the top of the list takes precedence and the merge process uses that value, as opposed to the row that's ranked second.

To change the rank, do the following from the Asset Lookup Configuration tab:

  1. Drag and drop the rows of the table into a new order.
  2. When finished reordering, click Save Ranking.

Ranking is not considered for a multivalue field. The merge process combines all the values into the field, and then removes the duplicates.

Key fields are dns, ip, mac, and nt_host. If you store extra information in your key fields, such as the same IP address assigned to multiple systems in pipe-delimited lists, these duplicate IP addresses are now merged together as one asset. Make sure that the information in your key fields either belongs to the same asset or does not overlap.

Disable or enable asset lookups

You can disable or enable an asset lookup input. Disabling an input does not delete the data from the associated lookup from Splunk Enterprise Security. Disabling prevents the contents of the corresponding list from being included in the merge process. Enabling a disabled input allows the associated list to be merged at the next scheduled merge of the asset or identity data.

To disable an asset lookup, do the following from the Asset Lookup Configuration tab:

  1. Navigate to the Status column.
  2. Do one of the following options:
    • Click Disable to disable an input.
    • Click Enable to enable a disabled input.

Starting with version 5.0.0, asset and identity lookup inputs are disabled by default after a new installation. However, local settings are respected after an upgrade.

Asset Settings

You can add a new asset field or enable case sensitive matching.

Add a new asset field

This is the list of asset fields that are added both by default and by entering custom fields manually. You can add up to 20 custom fields for your lookups. Key fields, such as dns, ip, mac, nt_host are non-editable. However, for custom and default fields you can configure whether the field is a tag field, a multivalue field, or both.

To add a new custom asset field, do the following:

  1. From the Splunk Enterprise Security menu bar, select Configure > Data Enrichment > Asset and Identity Management.
  2. Click the Asset Settings tab.
  3. Click Add New Field.
  4. In the New Asset Field dialog box, do the following:
    1. Enter a field name.
    2. Check the Multivalue check box if the field can output multiple values.
    3. Check the Tag check box if the field can be used as an asset tag. This is a helper field for holding additional values that you want to look up, in addition to the key fields. This is not the same as tagging in Splunk Enterprise.
    4. Click Save.

The Save button is disabled when the limit is reached and is enabled again when any custom field is deleted using the Delete action link.

Multivalue field limits for assets

For performance purposes, the default number of multivalue asset fields that display after merging follows:

  • 6 for key fields
  • 25 for non-key fields

The reason that the default multivalue key field limit is 6 for assets is because there are 4 key fields. If each key field contains 6 values, the merge process results in an asset field with 24 key values. Performance issues can occur when a resulting asset field contains more than 25 key values.

If your source csv file contains more values in a multivalue field than the limit, these values are truncated during the merge process. This means that in addition to not being displayed in the results, they also are removed from the data altogether. If you search or lookup on the truncated values, you will not find them because they do not exist.

If your data gets truncated, but you want to see more than the maximum values, then you need to revise your source csv files to spread out those values so that they seem to be part of different assets, by making sure that there are no duplicate values in the key fields.

Key fields are dns, ip, mac, and nt_host. If you store extra information in your key fields, such as the same IP address assigned to multiple systems, these duplicate IP addresses are now merged together as one asset. Make sure that the information in your key fields either belongs to the same asset or does not overlap.

Enable case-sensitive matching for asset fields

Case sensitive matching is now globally available across all fields.

Note that searches using | inputlookup ... where <filter> are case sensitive. Asset and Identity Management pages might use searches that contain where clauses. When case sensitivity is set to false, the merge process stores the values as lowercase so that case insensitive matches can be performed. To avoid this, you can toggle the case sensitive settings to true.

To use case-sensitive matching, do the following:

  1. From the Splunk Enterprise Security menu bar, select Configure > Data Enrichment > Asset and Identity Management.
  2. Click the Asset Settings tab.
  3. Enable the Enable case sensitive asset matching switch.
  4. Click Update to trigger the merge process and rewrite the asset_lookup_by_str and asset_lookup_by_cidr KV store collections.

Create an Identity Lookup Configuration

Identity lookup settings create the configuration that updates the inputs.conf file to point to a lookup and update your identities. When new items are added, or current items are updated, the change takes effect in 5 minutes.

Add an identity input stanza for the lookup source

To add a new identity input source, do the following:

  1. From the Splunk ES menu bar, select Configure > Data Enrichment > Asset and Identity Management.
  2. Click the Identity Lookup Configuration tab.
  3. Click New.
  4. In the New Identity Manager, do the following:
    1. Since you uploaded a CSV lookup file of identities during a prerequisite step, select the corresponding transforms.conf definition from the Source drop-down list.
    2. You can provide a name for the identity list stanza, but matching the source name is a good idea.
    3. Enter a descriptive category for this identity list, such as east_coast_employees or strategic_executives.
    4. Enter a detailed description of the contents of this identity list.
    5. Check the Blacklist check box to exclude the lookup file from bundle replication.

      The asset and identity source lookup files are excluded from bundle replication in an indexer cluster by default. The merged lookup files are still included in bundle replication to support asset and identity correlation. Changing the default to include asset and identity lookup files in bundle replication might reduce system performance. See Knowledge bundle replication overview in the Splunk Enterprise Distributed Search manual.

    6. In Lookup List Type, identity is selected for you.
    7. In Lookup Field Exclusion List, select fields for the merge process to ignore. This excludes the values from the KV store collections. This excludes the fields and those values from the KV store collections for that particular lookup. You might use this in the case where you have a field in your source file that you don't want to rely on for information.
  5. (Optional) Configure the conventions that the identity lookup can use to uniquely identify identities in your data.
    When an email convention check box is checked, the email address is used as an additional primary key for identity. The Email and Email Short conventions are enabled by default.
    1. Click Email to use the full email address.
    2. Click Email Short to use the email username.
    3. Click + Add a new convention to add a custom convention:
      You can identify users by the first few letters of their first name and the first few letters of their last name, based on the columns in the Identities Table. Use the convention of identity_first(n)middle(n)last(n) where identity, first, and last are any columns from the Identities Table, and where n is a number starting with 0. For example:
      • "Jane Marie Johnson" using the convention first(3)last(3) is "janjoh"
      • "John Michael Smith" using the convention first(1)middle(1).last() is "jm.smith"
      • "John Doe" using the convention ADMIN_first(1)last() is "ADMIN_jdoe"
      • Multiple matches are resolved automatically by taking the first match in the table or manually by specifying identity values.
  6. Click Save.

Rank the order for merging identities

Any new identity list gets added to the bottom of the page by default. You can rank the order of this list to determine priority for merging identities. If an identity exists in multiple source files as a single value, or exists multiple times in the same source file, this ranking is the weighted order for merging them. By default, the single value identity fields are as follows:

  • endDate
  • priority
  • startDate
  • watchlist

These are the fields where the rank takes effect. For example, if you're merging two identities, that both have the priority field value, you need to choose one to take precedence. The row at the top of the list takes precedence and the merge process uses that value, as opposed to the row that's ranked second.

To change the rank, do the following under the Identity Lookup Configuration tab:

  1. Drag and drop the rows of the table into a new order.
  2. When finished reordering, click Save Ranking.

Ranking is not considered for a multivalue field. The merge process combines all the values into the field, and then removes the duplicates.

Identity Settings

This is the list of identity fields that are added both by default and by entering custom fields manually. You can add up to 20 custom fields for your lookups. Key fields, such as identity are non-editable. However, for custom and default fields you are able to configure whether the field is a tag field, a multivalue field, or both.

Add a new identity field

To add a new custom identity field, do the following:

  1. From the Splunk ES menu bar, select Configure > Data Enrichment > Asset and Identity Management.
  2. Click the Identity Settings tab.
  3. Click Add New Field.
  4. In the New Identity Field window, do the following:
    1. Enter a lookup field name.
    2. Check the Multivalue check box if the field can output multiple values.
    3. Check the Tag check box if the field can be used as an identity tag. This is a helper field for holding additional values that you want to look up, in addition to the key fields. This is not the same as tagging in Splunk Enterprise.
    4. Click Save.

The button is disabled when the limit is reached and enabled again when any custom field is deleted using the Delete action link.

Multivalue field limits for identities

For performance purposes, the default number of key and non-key multivalue identity fields that display after merging is 25.

If your source csv file contains more values in a multivalue field than the limit, these values are truncated during the merge process. This means that in addition to not being displayed in the results, they also are removed from the data altogether. If you search or lookup on the truncated values, you will not find them because they do not exist.

If your data gets truncated, but you want to see more than the maximum values, then you need to revise your source csv files to spread out those values so that they seem to be part of different assets, by making sure that there are no duplicate values in the key fields.

The key field is identity and the default merge convention is email. If you store extra information in your key fields, such as the same identity or email address assigned to multiple people, these duplicates are now merged together as one identity. Make sure that the information in your key or email fields either belongs to the same person or does not overlap.

Enable case-sensitive matching for identity fields

Case-sensitive matching is now globally available across all fields.

Note that searches using | inputlookup ... where <filter> are case sensitive. Asset and Identity Management pages might use searches that contain where clauses. When case sensitivity is set to false, the merge process stores the values as lowercase so the case insensitive matches can be performed. To avoid this, you can toggle the case sensitive settings to true.

To use case-sensitive matching, do the following:

  1. From the Splunk Enterprise Security menu bar, select Configure > Data Enrichment > Asset and Identity Management.
  2. Click the Identity Settings tab.
  3. Enable the Enable case sensitive identity matching switch.
  4. Click Update to trigger the merge process and rewrite the identity_lookup_expanded KV store collection.

Enable Correlation Setup

When asset and identity correlation is enabled, Splunk Enterprise Security compares indexed events with asset and identity data in the asset and identity lists to provide data enrichment and context. The comparison process uses automatic lookups in the props.conf file. You can find information about automatic lookups in the Splunk platform documentation:

Asset and identity correlation enriches events with asset and identity data at search time in the following ways:

  • Asset correlation compares events that contain data in any of the src, dest, or dvc fields against the merged asset lists for matching IP address, MAC address, DNS name, or Windows NT host names. Asset correlation no longer occurs automatically against the host or orig_host fields.
  • Identity correlation compares events that contain data in any of the user or src_user fields against the merged identity lists for a matching identity.
  • Enterprise Security adds the matching output fields to the event. For example, correlation on the asset src field results in additional fields such as src_is_expected and src_should_timesync.

Asset and identity correlation lets you determine whether multiple events can relate to the same asset or identity. You can also perform actions on the identity and asset fields added to events to open additional searches or dashboards scoped to the specific asset or identity. For example, you can open the Asset Investigator dashboard on a src field.

Choose whether to enable asset and identity correlation, disable it, or restrict correlation to occur only for select source types. If in doubt, keep asset and identity correlation enabled.

Disabling asset and identity correlation completely prevents events from being enriched with asset and identity data from the asset and identity lookups. This might prevent correlation searches, dashboards, and other functionality from working as expected. Consult with Splunk Professional Services or Splunk Support before disabling asset and identity correlation.

You can choose from the following options:

  • Enable for all sourcetypes
  • Disable for all sourcetypes
  • Enable selectively by sourcetype

To enable or disable for all sourcetypes, do the following steps:

  1. From the Splunk Enterprise Security menu bar, select Configure > Data Enrichment > Asset and Identity Management.
  2. Click the Correlation Setup tab.
  3. Do one of the following options:
    • Click the Enable for all sourcetypes radio button.
    • Click the Disable for all sourcetypes radio button.
  4. Click Save.

To enable selectively by sourcetype, do the following steps:

  1. From the Splunk Enterprise Security menu bar, select Configure > Data Enrichment > Asset and Identity Management.
  2. Click the Correlation Setup tab.
  3. Click the Enable selectively by sourcetype radio button.
  4. Click + Add a new sourcetype.
  5. Enter the name of the sourcetype.
  6. Toggle Enable asset correlation or Enable identity correlation.
  7. Click Done.
  8. Click Save.

See Modify priority and rank in the Asset and Identity Framework in the Use Splunk Enterprise Security manual for further information about how ranks, correlations, and automatic lookups affect notable event urgency.

Use the search preview to test the merge process

You can test the asset and identity merge process if you want to confirm that the data produced by the merge process is expected and accurate. You can run the search previews to determine what the merge will do with your data without actually performing the merge. These steps aren't required, but can be performed to validate the merge works as expected.

If you used previous versions of ES, note that the search preview shows you the dynamic custom search that replaces the following correlation searches:

  • Identity - Asset CIDR Matches - Lookup Gen
  • Identity - Asset String Matches - Lookup Gen
  • Identity - Identity Matches - Lookup Gen

To preview all your asset and identity searches, do the following:

  1. From the ES menu bar, select Configure > Data Enrichment > Asset and Identity Management.
  2. Click the Search Preview tab.
  3. From each drop-down list, you can run the search preview for each collection, the lookups of which are located in the transforms.conf file:
    • asset_lookup_by_str is the lookup for the assets_by_str collection.
    • asset_lookup_by_cidr is the lookup for the assets_by_cidr collection.
    • identity_lookup_expanded is the lookup for the identities_expanded collection.

The search preview looks into all your lookup tables and creates custom-built searches with what is currently in your inputs.conf file. The search is dynamic and generates the search each time you refresh or load the page. If nothing has changed in the source files since the last merge, you do not see any output.

If you want to see some output regardless if anything has changed, you can remove the inputlookup append=T SPL from the search. For example, in the case of identities, you would remove: | inputlookup append=T "identity_lookup_expanded".

Reset your collections

All the asset and identity source files that are enabled in the Asset and Identity Management page get merged into the following default collections in the collections.conf file: assets_by_str, assets_by_cidr, or identities_expanded.

If your collections get into an undesirable state, you can reset your collections at any time, rather than waiting for the automated process to clear out the KV store collection. It's similar to clearing cache manually.

  1. From the Splunk Enterprise Security menu bar, select Configure > Data Enrichment > Asset and Identity Management.
  2. Click Reset Collections. The button is globally available regardless if you are configuring in a particular tab.

When the identity manager runs again in 5 minutes, it rebuilds the collections based on which source files are enabled in the Asset Lookup Configuration or the Identity Lookup Configuration.

Modify asset and identity lookups

Make changes to the asset and identity lookups in Splunk Enterprise Security to add new assets or identities, or change existing values in the lookup tables. You can also disable or enable existing lookups.

Edit asset and identity lookups

Edit an asset or identity lookup in the Identity Management dashboard.

  1. In Enterprise Security, select Configure > Data Enrichment > Asset and Identity Management.
  2. Find the name of the asset or identity list you want to edit, and select the corresponding lookup from the Source column. The list opens in an interactive editor.
  3. Use the scroll bars to view the columns and rows in the table. Double click a cell to add, change, or remove content.
  4. Click Save when you are finished.

Manually add static asset or identity data

Manually add new static asset or identity data to Splunk Enterprise Security by editing the Assets or Identities lookups. For example, add internal subnets, IP addresses to be whitelisted, and other static asset and identity data.

  1. From the Splunk ES menu bar, select Configure > Content > Content Management.
  2. To add asset data, click the Assets lookup to edit it. To add identity data, click the Identities list to edit it.
  3. Use the scroll bars to view the columns and rows in the table. Double click in a cell to add, change, or remove content.
  4. Save your changes.

Then you can see the lookup registered as static_assets or static_identities or in Configure > Data Enrichment > Asset and Identity Management.

Disable the demo asset and identity lookups

The demo asset and identity lookups are disabled by default. Enable them if needed for testing. Disable the demo asset and identity lookups to prevent the demo data from being added to the primary asset and identity lookups used by Splunk Enterprise Security for asset and identity correlation.

  1. In Enterprise Security, select Configure > Data Enrichment > Asset and Identity Management.
  2. Locate the demo_assets and demo_identities lookups.
  3. Click Disable for each.
Last modified on 07 April, 2020
Create a lookup from your current LDAP data in Splunk Enterprise Security   Verify that your asset and identity data was added to Splunk Enterprise Security

This documentation applies to the following versions of Splunk® Enterprise Security: 6.0.2

Acrobat logo Download manual
Acrobat logo Download this page

Was this topic useful?







You must be logged into splunk.com in order to post comments. Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic. If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk, consider posting a question to Splunkbase Answers.

0 out of 1000 Characters