Use Standalone LLM

You can use Standalone LLM through a set of dashboards. The following processes are covered:

• Download LLM models
• Inference with LLM

All the dashboards are powered by the fit command. The dashboards showcase Standalone LLM functionalities. You are not limited to the options provided on the dashboards. You can tune the parameters on each dashboard, or embed a scheduled search that runs automatically.

Download LLM models

Complete the following steps:

1. In Splunk App for Data Science and Deep Learning (DSDL), navigate to Assistants, then LLM-RAG, then Querying LLM with Vector Data , and select Manage your LLMs.
2. On the settings panel, select PULL from the Task drop-down menu and LLM from Model Type, as shown in the following image:
3. In the Model Name field, enter the name of the LLM you want to download. For model name references, see the Ollama library page at: https://ollama.com/library
4. Select Submit to start downloading. You see an on-screen confirmation after the download completes.
5. Confirm your download by selecting Refresh Page. Make sure that the LLM model name shows on the Existing LLM models panel.

Inference with LLM

Complete the following steps:

1. In DSDL, navigate to Assistants, then LLM-RAG, then Querying LLM with Vector Data, and select Standalone LLM.
2. A search bar is provided in the dashboard to conduct inference and natural language processing (NLP) tasks on the text data stored in Splunk. Use the rename command in combination with as text to search the text data stored in the Splunk platform.
   

   If you do not want to use text data, leave the default search command empty or tune the prompt with the search command.

3. After inferencing completes, a panel for inference settings is available. On this panel you can choose LLM models and input your search.

Examples

The following example uses text data and a Standalone LLM search:

The following example uses a one-shot prompt: