Personalization in Splunk AI Assistant for SPL

Personalization is a new feature of Splunk AI Assistant for SPL that provides more accurate and contextual SPL results based on your unique data and environment. When you opt-in for Personalization the Write SPL feature considers index, source type, field names, and past search queries when generating SPL results.

Personalization only collects non-personal data from your environment when generating results.

Personalization supports role based access controls (RBAC) and users will not see indexes to which they don't have access.

Personalization can help you build better SPL searches that are based on your real needs and not theoretical needs. Results from Personalization can help you better understand your own data and environment by offering SPL suggestions that are contextual and specific. This is a key feature of Splunk AI Assistant for SPL versus a third-party, public AI tool.

Personalization is optional. Splunk administrators can opt-out or opt-in for Personalization from the Settings tab within Splunk AI Assistant for SPL.

Configure Personalization settings

Only users with administrator privileges can opt-in or opt-out of this feature. Splunk administrators see the following modal window when first using Splunk AI Assistant for SPL version 1.1.0 or higher:

The check box to turn Personalization on is pre-selected by default in the modal window seen by administrators when first accessing version 1.1.0.

Splunk administrators can opt-in or out of data personalization at any time. This setting applies at the app level, across all users, and not at the individual user level.

If you want to opt-in or out of this feature, navigate to the Settings tab of the assistant. Select or de-select the Personalize results option, as shown in the following image:

If you opt-out of Personalization, Splunk AI Assistant for SPL will not be able to use the context of your data and environment in generating the response, leading to less relevant responses.

Users without administrator privileges see the Personalization information on the Settings tab, and the setting chosen, but cannot change this setting.

How Personalization works

Personalization works by taking the following actions:

• Collection of metadata: Scheduled jobs run daily that collect metadata from the stack. Metadata includes names of indexes, source types, fields, and search query logs.
• Add metadata to knowledge base: The collected metadata and AI-generated descriptions of the metadata are added into a Splunk AI Assistant for SPL knowledge base. Each stack has its own knowledge base, and the knowledge of one stack can not be used by another.
• Retrieval Augmented Generation (RAG): The user prompt is augmented with the most relevant metadata which helps the large language model (LLM) generate a more specific response with tailored index, source type, field information, and past search queries.

Data collected by Personalization

Activating Personalization allows for the sharing of aggregated usage data and indexed metadata with Splunk. When you use the Write SPL feature, the generated SPL considers index, source type, field information, and past search queries when generating results.

The data does not leave the region in which the stack belongs.

For details on what data is collected see, Share data in Splunk AI Assistant for SPL.

If you opt-in for Personalization, collected data is stored in a Splunk database. If you opt-out of Personalization at a later date, a cleanup job runs weekly which deletes any collected data.

Personalization search macros

The Personalization feature runs the following search to gather the sourcetype metadata used for tailored SPL results:

| tstats count where `saias_field_summary_indexes` by sourcetype index 
| dedup sourcetype, index 
| rename index as indexname, sourcetype as sourcetypename 
| map maxsearches=1000 search="| search index=\"$indexname$\" sourcetype=\"$sourcetypename$\" | `saias_field_summary_limit` | fieldsummary | eval index=\"$indexname$\", sourcetype=\"$sourcetypename$\"" 
| submitfielddata

The search consists of 2 parts:

• A tstats command to determine all of the unique index and sourcetype combinations present.
• A map subsearch which runs a fieldsummary command over each unique index and sourcetype combination. This determines what fields exist within that index and source type combination.

The following 2 macros within the search are configurable:

These macros can only be configured by your stack administrator.

Configurable macro	Details
saias_field_summary_indexes	Defaults to (index=* OR index=_*) . You can choose to select specific indexes be searched by the Personalization saved search used for gathering source type metadata. This change can reduce the total surface area over which the search runs, and reduce computational costs of the saved search.
saias_field_summary_limit	Limits the total number of events scanned over for each unique index and source type combination found by the fieldsummary subsearch. The macro is set to head 50000 to limit the performance impact of the map subsearch on large indexes.

Changing these values can lead to app performance problems. For example, if you find searches are taking too long, your adjusted macro values might need review from your stack administrator.

Monitor scheduled searches for metadata collected by Personalization

You can monitor the scheduled, saved searches for metadata collected by Personalization. Complete the following steps:

1. Open Splunk AI Assistant for SPL. Select Settings from the top navigation bar.
2. Select Searches, reports, and alerts.
3. Set the Owner filter to All.
4. To check the status of the user search logs saved searches select View Recent for the saved search or "Splunk AI Assistant for SPL - Search Logs".
5. To check the status of the metadata modular input (modinput) go to the Search tab in your Splunk instance and run the following SPL with a 24 hour lookback:

   index =_internal source=/opt/splunk/var/log/splunk/splunk_ai_assistant.log "Index metadata submitted successfully"

   The log events that appear indicate a successful modinput execution.

   These scheduled searches must not be interrupted or modified for Personalization to work properly.

Personalization known issues

The following are issues you might experience if you opt-in for Personalization:

• The generation speed for personalized results takes marginally longer than non-personalized results. This slight increase in generation speed allows for the search results to be specific to your environment and data.
• Saved searches that include Personalization, especially those collecting source type metadata, can be expensive. You can fine-tune using the 2 provided search macros to help this.
• Saved searches that include Personalization can run up against workload management rules and return partial results. Admins can double-check results of the Personalization saved searches and make sure that no errors occurred while running the saved search.
• The saias_field_summary_indexes macro has a default value of "(index=" OR index=_"). This can be redefined to only select indexes that admins want to be searched by the Personalization saved search for gathering source type metadata. Doing so can reduce the total surface area over which the search runs, and reduce computational costs of the saved search.