Predictive Analytics performance considerations in ITSI

While training and testing models in ITSI, there are several performance considerations to take into account. This topic provides guidance for choosing services to model, selecting training inputs, and retraining models.

While it's impossible to provide prescriptive advice for maximizing performance in every situation, the following observations and tips can help you tune and improve performance in your unique environment:

1. Avoid creating models for services that contain more than approximately 20 KPIs and 50 entities. The training time for a model depends on three factors: the number of KPIs in the service, the number of entities in the service, and the frequency of KPI searches. For a service with 20 total KPIs, 50 total entities, and 1-minute KPI searches, the average training time is 5 minutes. Scaling any of these three factors increases the training time.
2. Do not create models for more than 75 total services. Management of models is manual and can be difficult if you create too many.
3. Configure the MLTK to handle more memory and events. For more information, see Set up Predictive Analytics in ITSI.
4. Use at least 14 days worth of data to train your model. The training period is determined by the time period you specify before training a model. 30 days or more is recommended. In general, it is best to use as much data as you have available to train a model.
5. Base your training/test split ratio on the size of your dataset. Use a large split when dealing with large datasets (for example, 1 million data points). Use a smaller split when dealing with smaller datasets (for example, 10,000 data points). For more information, see Split your data into training and test sets.
6. Do not change the Test Period until you select a final model for the service. Leaving the data in separate train and test partitions, as configured in the training/test split, provides an honest assessment of the model's performance.
7. Retrain a service's model if KPIs or entities are added, removed, or changed. Changes in service architecture can cause changes in KPI behavior and service health score trends. For more information, see Retrain a predictive model in ITSI.
8. Before retraining a model, test it on at least 90 minutes of data. If you test on fewer than 90 minutes, the results could be inaccurate or incomplete.
9. Do not use the training search as a scheduled search. Predictive Analytics models do not require repetitive retraining, such as retraining every night. Retraining them too often can cause scale implications. Training is specifically designed to be an intermittent expense.

More performance help

If you experience performance issues, or want to receive feedback tailored to your setup, you have the following options:

• Post a request to the community on Splunk Answers.
• Contact Splunk Support.