The SRU formula in Splunk App for Chargeback

The app uses a formula to calculate an internal unit of measurement called Search Resource Usage (SRU). It's a universal hourly percentage of search consumption at the highest level of the business, i.e., the business units with a departmental/user breakdown.

• The percentage is stored in the chargeback_summary index, which is later used to estimate SVC/vCPU consumption by B-Unit/Department/User. At the user level, we can continue the breakdown.
• The formula works the same for Splunk Cloud on workload or ingest pricing and for Splunk Enterprise customers. Splunk Enterprise customers require an additional job to calculate hourly vCPU usage, which is being calculated by the chargeback_introspection_vcpu_usage_tracker job.
• The app summary job summarizes hourly the percentage of SRUs consumed by a given Job. A job is all the executions together of any type by [shcluster_label, App, User, Search_Type, Provenances, Job]. For example: All runs of a scheduled job or dashboard on the same search head or search head cluster by the same app, the same search type and provenance, the same user and job within the hour the measurement took place.
• During the summarization, the app uses the 8 Enrichment Principles to store information about the B-Units, departments and users within these departments' combination that handled the usage measured using the logic described.
• SRUs do not replace or contradict Splunk Cloud SVCs or Splunk Enterprise vCPUs. SRU is the estimated overhead search exhorts on the Splunk Platform to help organizations estimate usage, so it can be associated with the various business units using the platform. In Splunk Cloud, the App fetches the SVCs used for the day/hour and uses already defined B-Unit/Department information. The app applies the SRU percentage calculated to estimate the amount of SVCs the B-Units and their departments consumed that day/hour.
• The main objective of SRUs is to estimate hourly how much each B-Unit in the business consumed. The following chart illustrates an example of one hour of usage:

• Customers should use SRUs for Chargeback and Showback use cases only, and not a replacement of SVCs or vCPUs. Customers should use the CMC dashboards to review their SVC usage.
• The splunk-system-user runs all data models and reports accelerations, regardless of the user that created these knowledge objects. It also runs all scheduled jobs that were created by an App downloaded and installed from Splunkbase or Apps installed by Splunk, including premium apps. The App uses the 8 Enrichment Principles to estimate the amount of usage originating from this special user and allocates it to the correct B-Unit(s) and departments responsible for this usage.
• Customers should consider using an app naming convention that is used as a container to store reports and alerts belonging to a specific B-Unit. For example: create an app and call it devops_prod_app and store all DevOps prod reports and alerts in it. You can then configure the app to classify any usage originating from the App, no matter how these searches originated. The searches can be ad hoc users running searches and dashboards from within this app or scheduled jobs or accelerated data models, etc.
• The formula that drives SRUs uses these 6 metrics with a configurable weighted average at calculation time:

1. CPU usage from _introspection multiplied by configurable weighted average (9)
2. Memory usage from _introspection multiplied by configurable weighted average (7)
3. Runtime from _audit multiplied by configurable weighted average (3.5)
4. Buckets searched from _audit multiplied by configurable weighted average (1.5)
5. Raw events scanned from _audit multiplied by configurable weighted average (0.25)
6. Sparseness from _audit multiplied by configurable weighted average (2.5)

The metrics have a configurable weighted average using real numbers (0.1-9.9) that can be applied by simply placing more emphasis on one metric over the other.

Customers can alter the weight if they think one metric should have a higher or lower weight by simply increasing or decreasing the emphasis using this logic:

• A value of 1 means that there is no reduction or increase, aka neutral.
• A value less than 1 is a reduction from what Splunk produces.
• A value greater than 1 means more emphasis or increase.

Here is an example of when we need to adjust the weights: Splunk runs in either Splunk Cloud or Splunk Enterprise. Splunk Enterprise customers can run Splunk in their data center or in AWS/GCP/Azure. Because CPU and CPU usage may differ depending on where you run Splunk, it might be beneficial to increase or decrease the emphasis on CPU if you find the default 9 exaggerating the usage. For example, the overall vCPU usage is not making sense to you, either too high or too low. In that case, adjust the weight accordingly and review the data after allowing the app to run for an entire day.

Below is a diagram of the SRU formula, illustrated in a pie chart, with the emphasis of each metric, and its default weight: