Splunk Inc. collects critical data so that we can enhance the value of your investment in Splunk software.
We use this data to optimize your deployment, prioritize our features, improve your experience, notify you of patches, and develop high quality product functionality.
Changes in version 8.0.0
Splunk has changed its data collection practices and default settings in version 8.0.0. Even if you opted out of data collection in a previous release, version 8.0.0 resets aggregated usage data, support usage data, and license usage data collection to new default settings, which enable sharing this data with Splunk. When you upgrade to version 8.0.0 or install Splunk Enterprise 8.0.0 for the first time, the first user who logs in and is a member of the Splunk Admin role sees a pop-up notification summarizing the new data collection practices. This pop-up appears once in each deployment, new or upgraded, regardless of what previous opt-in or opt-out settings previously applied to the deployment.
Splunk also collects software version data. If you opted out of sharing software version data using configuration settings in previous releases, those settings are unchanged by the upgrade.
You can opt out of data sharing at any time. See How to opt out.
Benefits of sharing data with Splunk
When you share data with Splunk Inc., you receive the following benefits:
- Improved product quality. By collecting accurate information about the topology decisions and deployment scale used by our customers, we can replicate those topology configurations and scale in our internal testing, helping us improve your product experience.
- Timely notification of known bugs, version incompatibilities, and configuration issues. When you share data about the product versions you have deployed, we can provide accurate messages and support to help you with bugs, upgrade tasks, version compatibility problems, and other configuration issues you might experience.
- Relevant feature enhancements. We prioritize what features to develop and enhance first based on the features customers use the most. By sharing your data, you influence these data-driven decisions in favor of the features you use at your organization.
For more information, see How Splunk uses the data it collects.
What data Splunk collects
The table below summarizes the data that your Splunk platform deployment sends to Splunk when data collection is enabled. Follow the links to see examples of this data.
Type of data | Description | Examples |
---|---|---|
Aggregated usage data | Includes features used, deployment topology, and performance metrics in both the platform and apps. This data is not associated with your license ID. | Aggregated usage data examples App usage data examples |
Support usage data | Support usage data is the same as the aggregated usage data, but the license ID remains associated with your data when it reaches Splunk Inc. | Aggregated usage data examples App usage data examples |
License usage data | Includes your license ID, active license group and subgroup, total license stack quota, total license pool consumption, license stack type, license pool quota, license pool consumption. | License usage data examples |
Software version data | Includes the version of Splunk Enterprise and of each installed app, along with relevant metadata about deployment architecture. | Software version data examples |
Splunk does not collect the contents of your indexed data.
Some cloud and hybrid products modify the kinds of data that Splunk collects. When that happens, a separate agreement or notification states how the data collection differs for that product.
For instructions on how to view the data that your deployment collects and sends to Splunk, see View what data is sent from your deployment.
Examples of data sent to Splunk
Aggregated usage, support usage, and license usage data is sent to Splunk as a JSON packet that includes information like the component name and deployment ID, in addition to the data for the specific data collection component. The deploymentID is unique to a deployment and does not change on upgrade or even after uninstall and reinstall of Splunk Enterprise on the same machine.
Here is an example of a complete JSON packet:
{ component: deployment.app data: { [-] enabled: true host: 878e7b21bf98580dbdb4ed3baf6c35d78aa5bc3d3c824eb8714a313c name: search version: 8.0.0 } date: 2019-09-23 deploymentID: d6d8e776-a8d3-5467-a03b-375577646cbb executionID: 2FC293C59049AC0D44B677D3A9D786 timestamp: 1569294102 transactionID: 4E1CFC7E-BE9F-355D-7DDE-D4F8D5E4852D version: 3 visibility: anonymous,support }
The following tables list the component names, descriptions, and an example of what data is collected for that component. For ease of use, the examples for aggregated usage and license data show examples of only the data
field from the JSON object.
Aggregated usage data examples
The following example demonstrates the data sent to Splunk when sharing of aggregated usage data is enabled.
Component | Description | Example |
---|---|---|
app.session.dashboard.load
|
Dashboard characteristics, generated as session data when a dashboard loads. | { [-] app: search dashboard: { [-] autoRun: false hideAppBar: false hideChrome: false hideEdit: false hideExport: false hideFilters: false hideSplunkBar: false hideTitle: false isScheduled: false isVisible: true numCustomCss: 0 numCustomJs: 0 refresh: 0 submitButton: false theme: light } elementTypeCounts: { [-] statistics: 1 } formInputTypeCounts: { [-] } layoutType: row-column-layout numElements: 1 numFormInputs: 0 numPanels: 1 numPrebuiltPanels: 0 numSearches: 1 page: splunker searchTypeCounts: { [-] saved: 1 } } |
app.session.page.load
|
Tracks loads and whether web services are supported, generated as session data when a page loads. | { [-] allowWebService: true app: $SPLUNK_PLATFORM page: manager/search/adddata } |
app.session.pageview
|
Page view session data, generated whenever a user visits a new page. | { [-] app: launcher page: home } |
app.session.pivot.interact
|
Changes to pivots, generated as session data when a user makes a change to a pivot. | { [-] app: search context: pivot eventAction: change eventCategory: PivotEditorReportContent eventLabel: Pivot - Report Content eventValue: { [-] transient: true } numAggregations: 1 numColumnSplits: 0 numCustomFilters: 0 numRowSplits: 1 page: pivot reportProps: { [-] display.general.type: visualizations display.statistics.show: 1 display.visualizations.charting.chart: area display.visualizations.charting.chart.rangeValues: [0,30,70,100] display.visualizations.charting.gaugeColors: ["0x53a051","0xf8be34","0xdc4e41"] display.visualizations.charting.legend.placement: none display.visualizations.show: 1 display.visualizations.singlevalue.rangeColors: ["0x53a051","0x0877a6","0xf8be34","0xf1813f","0xdc4e41"] display.visualizations.singlevalue.trendInterval: auto display.visualizations.type: charting earliest: -24h@h latest: now windowedEarliest: 2019-09-23T03:00:00.000+00:00 windowedLatest: 2019-09-24T03:58:52.000+00:00 } } |
app.session.pivot.load
|
Pivot characteristics, generated as session data when a pivot loads. | { [-] app: search context: pivot eventAction: load eventCategory: PivotEditor eventLabel: Pivot - Page numAggregations: 1 numColumnSplits: 0 numCustomFilters: 0 numRowSplits: 1 page: pivot reportProps: { [-] display.general.type: visualizations display.statistics.show: 1 display.visualizations.charting.chart: area display.visualizations.charting.chart.rangeValues: [0,30,70,100] display.visualizations.charting.gaugeColors: ["0x53a051","0xf8be34","0xdc4e41"] display.visualizations.charting.legend.placement: none display.visualizations.show: 1 display.visualizations.singlevalue.rangeColors: ["0x53a051","0x0877a6","0xf8be34","0xf1813f","0xdc4e41"] display.visualizations.singlevalue.trendInterval: auto display.visualizations.type: charting earliest: -24h@h latest: now windowedEarliest: 2019-09-23T03:00:00.000+00:00 windowedLatest: 2019-09-24T03:58:52.000+00:00 } } |
app.session.roles.srchFilter
|
Event actions on the authoritzation/roles page of Splunk Web | { [-] app: $SPLUNK_PLATFORM context: authorization/roles eventAction: CreateEditRole eventCategory: SrchFilterInRoles eventLabel: Search Filter in role - admin eventValue: * page: manager/launcher/authorization/roles } |
app.session.search.interact
|
Search page interactions, session data generated by each user interaction with the search page. | { [-] app: search context: search eventAction: submit eventCategory: CreateReportDialog eventLabel: Search App - Actions eventValue: success page: search reportProps: { [-] dispatch.sample_ratio: 1 display.events.table.sortDirection: asc display.general.type: statistics display.page.search.mode: smart display.prefs.events.offset: 0 display.prefs.statistics.offset: 0 display.statistics.format.0: display.statistics.format.0.colorPalette: display.statistics.format.0.colorPalette.colors: display.statistics.format.0.field: display.statistics.format.0.scale: display.statistics.format.0.scale.thresholds: display.statistics.sortColumn: Number of Users display.statistics.sortDirection: asc display.visualizations.charting.chart: bar earliest: -24h@h latest: now workload_pool: } } |
app.session.session_start
|
Session data generated when a user is first authenticated. Contains the deploymentID (identifier for deployment), eventID (identifier for this specific event), experienceID (identifier for this session), userID (hashed username), data.guid (GUID for instance serving the page). | { [-] app: launcher browser: Chrome browserVersion: 68.0.3440.106 device: Linux x86_64 guid: 0C4C7528-375A-4DA5-ABF8-09189051BB51 locale: en-US os: Linux osVersion: not available page: home splunkVersion: 8.0.0 } |
authentication.jwt
|
Metrics tracking usage of the authentication token (JsonWebToken) feature: Whether or not the feature is enabled, the number of times tokens were created, whether or not the scripted extensions feature is being used, and the number of failures associated with token usage. | { [-] disabled: 0, created: 8, scriptedExtensionsEnabled: 1, failures: 0 } |
deployment.app
|
Apps installed on search head and peers. | { [-] enabled: true host: 878e7b21bf98580dbdb4ed3baf6c35d78aa5bc3d3c824eb8714a313c name: search version: 8.0.0 } |
deployment.clustering.indexer
|
Host name of an indexer, replication factor, and search factor for indexer cluster. | { [-] enabled: false host: 06d3392e0644587c3c3131833c81bfa6a7be78361e35e2ba8edf9c92 timezone: -0700 } |
deployment.clustering.member
|
Indexer cluster member status. | { [-] master: 1b83dc9e131f02b53329dfc1d3700aea92dd8223a22325d274e5aa3a member: { [-] guid: 14B1E1C3-ABD1-4D02-88D5-3A6964EF8376 host: 942796f349f59b3ae64b47e507299b64b9a638fc9fc7a2580863f951 status: Up } site: default } |
deployment.clustering.searchhead
|
Indexer cluster and search head connection status. | { [-] master: 1b83dc9e131f02b53329dfc1d3700aea92dd8223a22325d274e5aa3a searchhead: { [-] guid: 141D5E4A-3C5C-4051-B2DB-E679027A0D57 host: f7724a2690f17f0fe3ea97418c92fffde62a890b517261377b1060f4 status: Connected } site: default } |
deployment.distsearch.peer
|
Distributed search peer status. | { [-] host: 33b1957bfe1d0f7d3aac34e8655cf49f74375fb5043cb756f9a48405 peer: { [-] guid: 676F6738-BA57-44EC-94F0-A6821739DF8C host: 76e4ed3636a6f4dc9737d119fde51e0007713c7f87af7acf0dc057a7 status: Up } } |
deployment.forwarders
|
Forwarder architecture: Number of hosts, number of forwarder instances, OS/version, CPU architecture, Splunk Enterprise version, distribution of forwarding volume | { [-] architecture: x86_64 bytes: { [-] avg: 632367800 max: 689339847 min: 602231091 p10: 602891365 p20: 603551640 p30: 604211914 p40: 604872189 p50: 605532463 p60: 622293940 p70: 639055417 p80: 655816893 p90: 672578370 } hosts: 3 instances: 3 os: Linux splunkVersion: 8.0.0 type: full } |
deployment.index
|
Index type and configuration. | { [-] app: search buckets: { [-] cold: { [-] count: 0 events: 0 sizeGB: 0 } coldCapacityGB: unlimited homeCapacityGB: unlimited homeEventCount: 871 hot: { [-] count: 0 max: 3 sizeGB: 0 } thawed: { [-] count: 0 events: 0 sizeGB: 0 } warm: { [-] count: 6 sizeGB: 0 } } host: 6aac2d36b0f11492299b161a6c5a4f79451708e195b98a5dbaa47b9b name: uba_alarms total: { [-] buckets: 6 currentDBSizeGB: 0 events: 871 maxDataSizeGB: 500 maxTime: 1568987048 minTime: 1567603567 rawSizeGB: 0 } type: event } |
deployment.licensing.slave
|
License slaves. | { [-] master: 33b1957bfe1d0f7d3aac34e8655cf49f74375fb5043cb756f9a48405 slave: { [-] guid: 1E7D1EA4-9E76-410B-825F-36CDA037F377 host: 33b1957bfe1d0f7d3aac34e8655cf49f74375fb5043cb756f9a48405 pool: auto_generated_pool_enterprise } } |
deployment.node
|
GUID, host, number of virtual and physical cores, CPU architecture, memory size, storage (partition) capacity, OS/version, Splunk Enterprise version | data: { [-] cpu: { [-] architecture: x86_64 coreCount: 8 utilization: { [-] avg: 0.01 max: 0.15 min: 0.01 p10: 0.01 p20: 0.01 p30: 0.01 p40: 0.01 p50: 0.01 p60: 0.01 p70: 0.01 p80: 0.01 p90: 0.02 } virtualCoreCount: 8 } guid: XXXXXXXXXX host: YYYYYYYYY memory: { [-] capacity: 32655630402 utilization: { [-] avg: 0.67 max: 0.74 min: 0.5 p10: 0.6 p20: 0.62 p30: 0.64 p40: 0.66 p50: 0.67 p60: 0.69 p70: 0.7 p80: 0.71 p90: 0.72 } } os: Linux osExt: Linux osVersion: 4.15.0-1031-aws partitions: [ [-] { [-] capacity: 208111882207 fileSystem: ext4 utilization: 0.91 } ] splunkVersion: 8.0.0 } |
deployment.shclustering.member
|
Search cluster member status. | { [-] captain: 208999515adad3c46696443afe61049c8f8bfe56b6330feadbc64b48 member: { [-] guid: 45B3EA5E-4868-4243-9BEA-109C2F76F02A host: 258a814c13167915bedd945acd0f5e16c058a8b1bab8972206f82120 status: Up } site: default } |
instrumentation.performance
|
Performance of instrumentation queries. | { [-] instance_type: Single queries: [ [-] { [-] component: deployment.app isFailed: 0 resultCount: 145 runDuration: 0.843 scanCount: 0 searchProviders: 3 sid: 1569294993.84 } { [-] component: deployment.app isFailed: 0 resultCount: 145 runDuration: 1.079 scanCount: 0 searchProviders: 3 sid: 1569294995.85 } { [-] component: deployment.distsearch.peer isFailed: 0 resultCount: 2 runDuration: 0.211 scanCount: 0 searchProviders: 3 sid: 1569294996.86 } { [-] component: deployment.licensing.slave isFailed: 0 resultCount: 1 runDuration: 0.781 scanCount: 0 searchProviders: 3 sid: 1569294997.87 } { [-] component: usage.search.report_acceleration isFailed: 0 resultCount: 1 runDuration: 0.387 scanCount: 0 searchProviders: 3 sid: 1569294998.88 } { [-] component: usage.search.report_acceleration isFailed: 0 resultCount: 1 runDuration: 0.36 scanCount: 0 searchProviders: 3 sid: 1569294998.89 } { [-] component: usage.search.searchTelemetry isFailed: 0 resultCount: 1 runDuration: 1.2650000000000001 scanCount: 14 searchProviders: 3 sid: 1569294999.90 } { [-] component: usage.lookups.lookupDefinitions isFailed: 0 resultCount: 1 runDuration: 0.28700000000000003 scanCount: 0 searchProviders: 1 sid: 1569295000.91 } { [-] component: performance.bundleReplication isFailed: 0 resultCount: 3 runDuration: 1.238 scanCount: 2784 searchProviders: 3 sid: 1569295001.92 } { [-] component: performance.indexing isFailed: 0 resultCount: 8 runDuration: 6.098 scanCount: 35273 searchProviders: 3 sid: 1569295010.93 } { [-] component: performance.search isFailed: 0 resultCount: 3 runDuration: 21.253 scanCount: 213234 searchProviders: 3 sid: 1569295016.94 } { [-] component: usage.search.concurrent isFailed: 0 resultCount: 8 runDuration: 8.671 scanCount: 167724 searchProviders: 3 sid: 1569295038.96 } { [-] component: usage.users.active isFailed: 0 resultCount: 3 runDuration: 9.34 scanCount: 56960 searchProviders: 3 sid: 1569295047.97 } { [-] component: deployment.node isFailed: 0 resultCount: 15 runDuration: 9.965 scanCount: 1166 searchProviders: 3 sid: 1569295056.98 } { [-] component: deployment.index isFailed: 0 resultCount: 113 runDuration: 14.809000000000001 scanCount: 0 searchProviders: 3 sid: 1569295067.99 } { [-] component: usage.search.type isFailed: 0 resultCount: 3 runDuration: 17.365000000000002 scanCount: 167724 searchProviders: 3 sid: 1569295082.100 } { [-] component: licensing.stack isFailed: 0 resultCount: 5 runDuration: 1.772 scanCount: 10 searchProviders: 3 sid: 1569295100.101 } { [-] component: deployment.forwarders isFailed: 0 resultCount: 28 runDuration: 8.309000000000001 scanCount: 268106 searchProviders: 3 sid: 1569295102.102 } { [-] component: usage.indexing.sourcetype isFailed: 0 resultCount: 1373 runDuration: 45.673 scanCount: 735929 searchProviders: 3 sid: 1569295111.103 } { [-] component: deployment.clustering.indexer isFailed: 0 resultCount: 1 runDuration: 3.157 scanCount: 0 searchProviders: 1 sid: 1569295160.104 } { [-] component: usage.app.page isFailed: 0 resultCount: 9 runDuration: 0.795 scanCount: 65 searchProviders: 3 sid: 1569295163.105 } ] roles: { [-] cluster_master: false in_cluster: false indexer: true kv_store: true lead_node: true license_master: true search_head: true } timezone: +0000 } |
licensing.stack
|
Licensing quota and consumption. | { consumption: 127025471 guid: C131C257-98FE-4E8B-9595-CB4D93246F98 host: Splunk name: enterprise pools: [ { consumption: 127025471 quota: 6442450944 } ] product: enterprise quota: 6442450944 subgroup: Production type: enterprise } |
performance.bundleReplicationCycle
|
Metrics for the bundle replication cycle. | { [-] avgBundleBytes: 0 avgPeerCount: 1 avgPeerSuccessCount: 1 avgReplicationTimeMsec: 1 cycleCount: 144 replicationPolicy: classic } |
performance.indexing
|
Indexing performance: Core utilization, storage utilization, memory usage, indexing throughput, search latency. | { [-] host: 3c4681a5be1881de8554c8bab7be78e8d151557ef571e6a72bdad589 thruput: { [-] avg: 1903 max: 7854 min: 4 p10: 1419 p20: 1433 p30: 1452 p40: 1806 p50: 1860 p60: 1865 p70: 1878 p80: 2046 p90: 2326 total: 7138077 } } |
performance.search
|
Search performance: Core utilization, storage utilization, memory usage, indexing throughput, search latency. | { [-] buckets: { [-] avg: 1.9 max: 27 min: 0 p10: 0 p20: 0 p30: 0 p40: 0 p50: 0 p60: 0.88 p70: 2 p80: 6 p90: 6 } dayRange: { [-] avg: 876.81 max: 18162.29 min: 0 p10: 0 p20: 0 p30: 0 p40: 0 p50: 0 p60: 0.01 p70: 0.01 p80: 0.01 p90: 0.03 } latency: { [-] avg: 2.31 max: 19744.69 min: 0.01 p10: 0.02 p20: 0.02 p30: 0.09 p40: 0.47 p50: 1.6 p60: 1.85 p70: 2.05 p80: 2.23 p90: 2.64 } scanCount: { [-] avg: 344030.32 max: 38060408 min: 0 p10: 0 p20: 0 p30: 0 p40: 0 p50: 1.59 p60: 90.32 p70: 1156.18 p80: 25454.25 p90: 308440.56 } searches: 30576 slices: { [-] avg: 5034.33 max: 219740 min: 0 p10: 0 p20: 0 p30: 0 p40: 0 p50: 0 p60: 0 p70: 2246.06 p80: 11491.43 p90: 14170.42 } } |
usage.app.page
|
App name, page name, locale, number of users, number of page loads, generated as session data. | { [-] app: search locale: en-US occurrences: 1 page: users: 1 } |
usage.authMethod.config
|
Authentication method: Hashed host and GUID, authentication method (Splunk, LDAP, or SAML), MFA type (none, Duo, or RSA). | { [-] authentication method: Splunk guid: C099BFA3-E5B5-4AB1-AB64-471703C54388 host: 8cd44b23a1bd3ae283f21a7d9c5434163181efc8 mfa type: none } |
usage.healthMonitor.report
|
Health report manager: Alert actions and enabled status, feature thresholds and enabled status. | { [-] alert: { [-] alert_action:email: { [-] action/ action.to/ action.url/ action.integration_url_override: empty disabled: 0 } alert_action:webhook: { [-] action/ action.to/ action.url/ action.integration_url_override: empty disabled: 0 } health_reporter: { [-] action/ action.to/ action.url/ action.integration_url_override: email disabled: 0 } } feature:batchreader: { [-] enabled: 1 threshold: { [-] indicator:data_out_rate:red: 2 indicator:data_out_rate:yellow: 1 } } feature:buckets: { [-] enabled: 1 threshold: { [-] indicator:buckets_created_last_60m:red: 60 indicator:buckets_created_last_60m:yellow: 40 indicator:percent_small_buckets_created_last_24h:red: 50 indicator:percent_small_buckets_created_last_24h:yellow: 30 } } feature:cluster_bundles: { [-] enabled: 1 threshold: { [-] indicator:cluster_bundles:yellow: 1 } } feature:data_durability: { [-] enabled: 1 threshold: { [-] indicator:cluster_replication_factor:red: 1 indicator:cluster_search_factor:red: 1 } } feature:data_searchable: { [-] enabled: 1 threshold: { [-] indicator:data_searchable:red: 1 } } feature:ddaa_archived_buckets: { [-] enabled: 1 threshold: { [-] indicator:archived_buckets_failed_last_24h:red: 80 indicator:archived_buckets_failed_last_24h:yellow: 40 } } feature:disk_space: { [-] enabled: 1 threshold: { [-] indicator:disk_space_remaining_multiple_minfreespace:red: 1 indicator:disk_space_remaining_multiple_minfreespace:yellow: 2 } } feature:indexers: { [-] enabled: 1 threshold: { [-] indicator:detention:red: 1 indicator:detention:yellow: 1 indicator:missing_peers:red: 1 indicator:missing_peers:yellow: 1 } } feature:indexing_ready: { [-] enabled: 1 threshold: { [-] indicator:indexing_ready:red: 1 } } feature:master_connectivity: { [-] enabled: 1 threshold: { [-] indicator:master_connectivity:red: 1 } } feature:replication_failures: { [-] enabled: 1 threshold: { [-] indicator:replication_failures:red: 10 indicator:replication_failures:yellow: 5 } } feature:s2s_autolb: { [-] enabled: 1 threshold: { [-] indicator:s2s_connections:red: 70 indicator:s2s_connections:yellow: 20 } } feature:search_lag: { [-] enabled: 1 threshold: { [-] indicator:count_extremely_lagged_searches_last_hour:red: 1 indicator:count_extremely_lagged_searches_last_hour:yellow: 0 indicator:percent_searches_lagged_high_priority_last_24h:yellow: 10 indicator:percent_searches_lagged_non_high_priority_last_24h:yellow: 40 } } feature:searches_delayed: { [-] enabled: 1 threshold: { [-] indicator:percent_searches_delayed_high_priority_last_24h:red: 10 indicator:percent_searches_delayed_high_priority_last_24h:yellow: 5 indicator:percent_searches_delayed_non_high_priority_last_24h:red: 20 indicator:percent_searches_delayed_non_high_priority_last_24h:yellow: 10 } } feature:searches_skipped: { [-] enabled: 1 threshold: { [-] indicator:percent_searches_skipped_high_priority_last_24h:red: 10 indicator:percent_searches_skipped_high_priority_last_24h:yellow: 5 indicator:percent_searches_skipped_non_high_priority_last_24h:red: 20 indicator:percent_searches_skipped_non_high_priority_last_24h:yellow: 10 } } feature:searchheadconnectivity: { [-] enabled: 1 threshold: { [-] indicator:master_connectivity:red: 1 indicator:master_version_compatibility:yellow: 1 } } feature:shc_captain_common_baseline: { [-] enabled: 1 threshold: { [-] indicator:common_baseline:red: 1 } } feature:shc_captain_connection: { [-] enabled: 1 threshold: { [-] indicator:captain_connection:red: 1 indicator:captain_existence:red: 1 } } feature:shc_captain_election_overview: { [-] enabled: 1 threshold: { [-] indicator:dynamic_captain_quorum:yellow: 1 } } feature:shc_members_overview: { [-] enabled: 1 threshold: { [-] indicator:detention:red: 1 indicator:detention:yellow: 1 indicator:replication_factor:yellow: 1 indicator:status:red: 1 indicator:status:yellow: 1 } } feature:shc_snapshot_creation: { [-] enabled: 1 threshold: { [-] indicator:snapshot_creation:red: 20 indicator:snapshot_creation:yellow: 10 } } feature:slave_state: { [-] enabled: 1 threshold: { [-] indicator:slave_state:red: 1 indicator:slave_state:yellow: 1 } } feature:slave_version: { [-] enabled: 1 threshold: { [-] indicator:slave_version:red: 1 } } feature:splunkoptimize_processes: { [-] enabled: 1 threshold: { [-] indicator:concurrent_optimize_processes_percent:yellow: 100 } } feature:tailreader: { [-] enabled: 1 threshold: { [-] indicator:data_out_rate:red: 2 indicator:data_out_rate:yellow: 1 } } feature:wlm_configuration_check: { [-] enabled: 1 threshold: { [-] indicator:configuration_check:red: 0 } } feature:wlm_system_check: { [-] enabled: 1 threshold: { [-] indicator:system_check:red: 0 } } } |
usage.indexing.sourcetype
|
Indexing volume, number of events, number of hosts, source type name. | { [-] bytes: 90962 events: 354 hosts: 1 name: splunk_telemetry } |
usage.kvstore
|
Metrics and performance data about KV store. | { [-] usage.flushAverageMs: 5.3538461538461535 usage.instanceType: primary usage.memRamMb: 0 usage.memVirtualMb: 0 usage.oplogEndTime: 1569301264 usage.oplogStartTime: 1569222045 usage.oplogTimeRange: 79219 usage.readLatencyToUpTime: 0.000153653421585191 usage.readLatencyUsPerOp: 0.02158053280617528 usage.storageEngine: mmapv1 usage.upTime: 3956 usage.version: 3.6.12-splunk usage.writeLatencyToUpTime: 0.000153653421585191 usage.writeLatencyUsPerOp: 0.00048009036995199094 } |
usage.lookups.lookupDefinitions
|
Lookup definition metadata with hashed lookup names. | { [-] lookups: [ [-] { [-] _timediff: is_temporal: 0 name: 96117ed21e74f16d452027ed8e16c5d32fddd229 sharing: system size: type: external } { [-] _timediff: is_temporal: 0 name: 256d0fae9448acc55cd2e5cbabe7dbec576158c2 sharing: global size: 18053 type: file } { [-] _timediff: is_temporal: 0 name: 88767984d9dc6308309ffde5dc3591fa3865e7f2 sharing: global size: 832 type: file } { [-] _timediff: is_temporal: 0 name: 1b0131dbc851786586e269a2ba8b2f08bbd6834f sharing: global size: type: geo } { [-] _timediff: is_temporal: 0 name: 6d47b91d0c0753e9332ec2c0f8c956151c9b1e16 sharing: global size: type: geo } ] } |
usage.passwordPolicy.config
|
Password policy management: hashed host and GUID, attribute configurations. | { [-] constant login time: 0.000 days until password expires: 90 enable lockout users: false enable password expiration: false enable password history: false enable verbose login fail message: true expiration alert in days: 15 failed login attempts: 5 force existing users to change weak passwords: false guid: 32BEE8DE-E64D-4B02-B2FE-4F13F18A0CAE host: b8758da2f94fd58e648bce573fa3d9dc5797566d lockout duration in minutes: 30 lockout threshold in minutes: 5 minimum number of characters: 1 minimum number of digits: 0 minimum number of lowercase letters: 0 minimum number of special characters: 0 minimum number of uppercase letters: 0 password history count: 24 } |
usage.python
|
Default setting for Python version in the app, path of the script with its name hashed, version of Python used in the script. | { [-] pythonDefault: python2 scriptPath: /usr/local/bamboo/splunk-install/current/etc/apps/SplunkEnterpriseSecuritySuite/bin/D7A80DE23601F645B8A06995DF910A3D08AB9EAA scriptPythonVersion: python2 } |
usage.search.concurrent
|
Distribution of concurrent searches. | { [-] host: 3c4681a5be1881de8554c8bab7be78e8d151557ef571e6a72bdad589 searches: { [-] avg: 2 max: 2 min: 1 p10: 1 p20: 1 p30: 1 p40: 1 p50: 2 p60: 2 p70: 2 p80: 2 p90: 2 } } |
usage.search.report_acceleration
|
Report acceleration metrics. | { [-] existing_report_accelerations: 0 } |
usage.search.searchTelemetry
|
List of commands and corresponding counts for all searches run on the system in the span of one day. | { [-] commands: [ [-] { [-] count: 1 name: addinfo } { [-] count: 5 name: eval } { [-] count: 6 name: external_command } { [-] count: 9 name: fields } { [-] count: 1 name: inputlookup } { [-] count: 1 name: join } { [-] count: 1 name: litsearch } { [-] count: 2 name: makemv } { [-] count: 1 name: mvcombine } { [-] count: 2 name: mvexpand } { [-] count: 2 name: noop } { [-] count: 4 name: prerest } { [-] count: 1 name: prestats } { [-] count: 4 name: presummarize } { [-] count: 2 name: rename } { [-] count: 4 name: rest } { [-] count: 1 name: search } { [-] count: 3 name: stats } { [-] count: 4 name: summarize } { [-] count: 6 name: timeliner } { [-] count: 1 name: where } ] } |
usage.search.searchtelemetry.type
|
Search type, count, average bytes read, max bytes read, duration. | { [-] searchTypeInformation: [ [-] { [-] avg(bytes_read): 90531.02683363149 count: 559 duration: 1488.45949719 max(bytes_read): 46382154 type: adhoc } { [-] avg(bytes_read): 0 count: 3224 duration: 199.042348043 max(bytes_read): 0 type: scheduled } ] } |
usage.search.searchtelemetry.sourcetypeUsage
|
Sourcetype usage. | { [-] sourcetypeUsage: [ [-] { [-] http_event_collector_metrics: 1 kvstore: 1 mongod: 3 search_telemetry: 1 splunk_disk_objects: 1 splunk_resource_usage: 1 splunk_web_service: 3 splunkd: 11 splunkd_remote_searches: 3 splunkd_ui_access: 2 } ] } |
usage.search.type
|
Number of searches of each type. | { [-] ad-hoc: 3619 datamodel acceleration: 1 other: 2 report acceleration: 1 scheduled: 34412 summary index: 506 } |
usage.smartStore.Config
|
SmartStore global configuration, per index configuration, hashed internal and external index names. | { [-] global config: { [-] cachemanager: { [-] eviction_padding: 5120 hotlist_bloom_filter_recency_hours: 360 hotlist_recency_secs: 86400 max_cache_size: 0 } clustering: { [-] mode: disabled } diskUsage: { [-] minFreeSpace: 5000 } } list of indexes: { [-] non-SmartStore enabled: ea9f4255e269599dd961c3efd8775ab5ac1d3948,f1b1f1f40216ee2e2b5a526eec43c8f71cccef5d,302a11446cd560395417c9e2d2177a7a0fa8d74d,568b2f85dcc1c8608d713a66a0eabd5b88956547,d140ef99de26b2f8b6f54081084d0b8b2f59f36f,5a74588fcf73bdd06619007f6659c41827885700,66f79d8a6327c82c9033e6d65ff03322a3766c87,b28b7af69320201d1cf206ebf28373980add1451,f4f77578164d1b03fb4c931f727a3e2966e541d4,0d176ba3aa7be325bcaeaf13ea2da4d155f04e33,87da723b9f33eb0f1bcad8ea3405d8c2d248f862,05535ecff78ef61038725b6ed3016b8c9a037496,f397214775e4f8191c17e838b4d518cb90051672 } per index config: { [-] external_05535ecff78ef61038725b6ed3016b8c9a037496: { [-] frozenTimePeriodInSecs: 188697600 hotlist_bloom_filter_recency_hours: none hotlist_recency_secs: none maxGlobalDataSizeMB: 0 maxHotSpanSecs: 7776000 } external_0d176ba3aa7be325bcaeaf13ea2da4d155f04e33: { [-] frozenTimePeriodInSecs: 188697600 hotlist_bloom_filter_recency_hours: none hotlist_recency_secs: none maxGlobalDataSizeMB: 0 maxHotSpanSecs: 7776000 } external_66f79d8a6327c82c9033e6d65ff03322a3766c87: { [-] frozenTimePeriodInSecs: 604800 hotlist_bloom_filter_recency_hours: none hotlist_recency_secs: none maxGlobalDataSizeMB: 0 maxHotSpanSecs: 7776000 } external_87da723b9f33eb0f1bcad8ea3405d8c2d248f862: { [-] frozenTimePeriodInSecs: 188697600 hotlist_bloom_filter_recency_hours: none hotlist_recency_secs: none maxGlobalDataSizeMB: 0 maxHotSpanSecs: 7776000 } external_b28b7af69320201d1cf206ebf28373980add1451: { [-] frozenTimePeriodInSecs: 188697600 hotlist_bloom_filter_recency_hours: none hotlist_recency_secs: none maxGlobalDataSizeMB: 0 maxHotSpanSecs: 7776000 } external_f397214775e4f8191c17e838b4d518cb90051672: { [-] frozenTimePeriodInSecs: 188697600 hotlist_bloom_filter_recency_hours: none hotlist_recency_secs: none maxGlobalDataSizeMB: 0 maxHotSpanSecs: 7776000 } external_f4f77578164d1b03fb4c931f727a3e2966e541d4: { [-] frozenTimePeriodInSecs: 188697600 hotlist_bloom_filter_recency_hours: none hotlist_recency_secs: none maxGlobalDataSizeMB: 0 maxHotSpanSecs: 7776000 } internal_302a11446cd560395417c9e2d2177a7a0fa8d74d: { [-] frozenTimePeriodInSecs: 1209600 hotlist_bloom_filter_recency_hours: none hotlist_recency_secs: none maxGlobalDataSizeMB: 0 maxHotSpanSecs: 7776000 } internal_568b2f85dcc1c8608d713a66a0eabd5b88956547: { [-] frozenTimePeriodInSecs: 1209600 hotlist_bloom_filter_recency_hours: none hotlist_recency_secs: none maxGlobalDataSizeMB: 0 maxHotSpanSecs: 7776000 } internal_5a74588fcf73bdd06619007f6659c41827885700: { [-] frozenTimePeriodInSecs: 2419200 hotlist_bloom_filter_recency_hours: none hotlist_recency_secs: none maxGlobalDataSizeMB: 0 maxHotSpanSecs: 7776000 } internal_d140ef99de26b2f8b6f54081084d0b8b2f59f36f: { [-] frozenTimePeriodInSecs: 63072000 hotlist_bloom_filter_recency_hours: none hotlist_recency_secs: none maxGlobalDataSizeMB: 0 maxHotSpanSecs: 7776000 } internal_ea9f4255e269599dd961c3efd8775ab5ac1d3948: { [-] frozenTimePeriodInSecs: 188697600 hotlist_bloom_filter_recency_hours: none hotlist_recency_secs: none maxGlobalDataSizeMB: 0 maxHotSpanSecs: 7776000 } internal_f1b1f1f40216ee2e2b5a526eec43c8f71cccef5d: { [-] frozenTimePeriodInSecs: 2592000 hotlist_bloom_filter_recency_hours: none hotlist_recency_secs: none maxGlobalDataSizeMB: 0 maxHotSpanSecs: 432000 } } total storage capacity: { [-] 0: { [-] available: 130459.672 capacity: 476802.039 free: 142405.105 fs_type: apfs } } } |
usage.users.active
|
The number of active users per day. | { [-] active: 1 } |
usage.workloadManagement.report
|
Workload management: Hashed host and GUID, OS/version, server roles, WLM support and enable status, pool configurations, rule configurations. | { [-] categories: { [-] ingest: { [-] allocated cpu percent: 20.00 allocated mem limit: 100.00 } misc: { [-] allocated cpu percent: 10.00 allocated mem limit: 10.00 } search: { [-] allocated cpu percent: 70.00 allocated mem limit: 70.00 } } guid: F3DC7C6B-DF89-4585-A7A6-B4A3510D957D host: eadc124359ea492c6b04c079dcf3bec3be2fb32c os: Linux osVersion: 4.9.184-linuxkit pools: { [-] total count: 0 } rules: { [-] total count: 0 } server roles: indexer, license_master, kv_store wlm enabled: 0 wlm supported: 1 } |
Support usage data examples
Support usage data is the same data as the aggregated usage data, but if you opt to send support usage data, Splunk can use the license GUID to identify usage data from a specific customer account to help troubleshoot support cases.
See Aggregated usage data examples.
Support usage data is distinct from diagnostic file data. Diagnostic files are never automatically generated and can only be sent to Splunk Support manually by a user with the appropriate permissions. For more about diagnostic files, see Generate a diag in the Troubleshooting Manual.
License usage data examples
The following example demonstrates the type of data sent to Splunk when sharing of license usage data is enabled.
Component | Description | Example |
---|---|---|
licensing.stack
|
Licensing quota and consumption | { [-] consumption: 14462827 guid: 12345678-90AB-CDE host: abcdea12b3456789012c345dea name: enterprise pools: [ [-] { [-] consumption: 14462827 quota: 53687091200 } ] product: enterprise quota: 53687091200 subgroup: Production type: enterprise } |
deployment.node
|
GUID, host, number of virtual and physical cores, CPU architecture, memory size, storage (partition) capacity, OS/version, Splunk Enterprise version | { [-] data: { [-] cpu: { [-] architecture: x86_64 coreCount: 4 utilization: { [-] avg: 0.34 max: 1 min: 0.01 p10: 0.11 p20: 0.15 p30: 0.2 p40: 0.24 p50: 0.29 p60: 0.34 p70: 0.41 p80: 0.49 p90: 0.63 } virtualCoreCount: 8 } guid: 12345678-90AB-CDE host: abcdea12b3456789012c345dea memory: { [-] capacity: 31618072838 utilization: { [-] avg: 0.17 max: 0.29 min: 0.07 p10: 0.1 p20: 0.11 p30: 0.12 p40: 0.12 p50: 0.14 p60: 0.22 p70: 0.22 p80: 0.23 p90: 0.23 } } os: Linux osExt: Linux osVersion: 4.14.121-85.96.amzn1.x86_64 partitions: [ [-] { [-] capacity: 6341226332684 fileSystem: ext4 utilization: 0.85 } ] splunkVersion: 8.0.3 } } |
Software version data examples
The following example demonstrates the software version data sent to Splunk for Splunk Enterprise when sharing of software version data is enabled.
Description | Example |
---|---|
CPU architecture | x86_64 |
Operating system | Linux |
Product | enterprise |
Splunk roles | admin |
License group, subgroup, and hashed GUID | Enterprise, Production, <GUID> |
Splunk software version | 7.0.0 |
The following example demonstrates the software version data sent to Splunk for each app when sharing of software version data is enabled for that app.
Description | Example |
---|---|
App ID, name, and version | gettingstarted, Getting Started, 1.0 |
Splunk version | 7.0 |
Platform, architecture | Darwin, x86_64 |
App usage data examples
In addition to the data enumerated in this topic, certain apps collect usage data. See the documentation for each app for details and examples.
- Splunk Add-on Builder: Share data in Splunk Add-on Builder
- Splunk Analytics Workspace: Share data in the Splunk Analytics Workspace
- Splunk App for AWS: Share data in the Splunk App for AWS
- Splunk Business Flow: Share data in Splunk Business Flow
- Splunk DB Connect: Share data in Splunk DB Connect
- Splunk Enterprise Security: Share data in Splunk Enterprise Security
- Splunk Industrial Asset Intelligence: Share data in Splunk Industrial Asset Intelligence
- Splunk IT Service Intelligence: Share data in Splunk IT Service Intelligence
- Splunk Machine Learning Toolkit: Share data in the Splunk Machine Learning Toolkit
- Splunk Security Essentials: Splunk Security Essentials Telemetry
How Splunk collects the data
If aggregated, support, or license usage data collection is enabled, a few instances in your Splunk Enterprise deployment collect data through scheduled searches. Most of the searches run in sequence, starting at 3:05 AM on the node that runs the searches, unless you change the schedule. All searches are triggered with a scripted input.
In addition, when aggregated or support data collection is enabled, session data about user activity transmits from the browser directly to the Splunk telemetry API.
Which instance runs the searches and sends data to Splunk
One primary instance in your deployment runs distributed searches that collect most of the usage data. This primary instance is also responsible for sending the data to Splunk. The instance that acts as the primary instance depends on the details of your deployment:
- If indexer clustering is enabled, the cluster master is the primary instance. If you have more than one indexer cluster, each cluster master is a primary instance.
- If search head clustering is enabled but not indexer clustering, each search head captain is a primary instance.
- If your deployment does not use clustering, the searches run on a search head.
If you opt out of instrumentation, the searches from the primary instance do not run.
Additional instances in your deployment run a smaller number of searches, depending on colocation details. If data collection is enabled, the data from these searches is collected by the primary node and sent to Splunk. If you opt out, these searches still run, but no data is sent.
For your deployment to send data to Splunk, the primary instance responsible for the searches must be connected to the internet with no firewall rules or proxy server configurations that prevent outbound traffic to https://quickdraw.splunk.com/telemetry/destination
or https://*.api.splkmobile.com
. If necessary, whitelist these URLs for outbound traffic.
Instrumentation in the Splunk Enterprise file system
After the searches run, the searched data is packaged and sent to Splunk, as well as indexed to the _telemetry
index. Session data is transmitted directly to the telemetry API from the browser and is not persisted in the _telemetry
index. The _telemetry
index is retained for two years by default and is limited in size to 256 MB.
The instrumentation app resides in the file system at $SPLUNK_HOME/etc/apps/splunk_instrumentation
.
How Splunk uses the data it collects
If you share aggregated usage data, Splunk collects data about your Splunk software usage and aggregates it together with similar data from other deployments so Splunk can understand what features and workflows are most important to users and improve its products and services over time. Collected license IDs are used only to verify that data is received from a valid Splunk product and persisted only for deployments opting into license or support usage reporting. These license IDs help Splunk analyze how different Splunk products are being deployed across the population of customers and are not attached to any aggregated usage data.
If you share support usage data, Splunk links the data about your software usage to your installed license ID so that Splunk can provide improved support and services for your deployment. Support usage data is used by Support and Customer Success teams to troubleshoot support issues that you file and improve your Splunk software implementation.
If you share license usage data, Splunk uses the data to ensure compliance with your purchased offering.
If you share Splunk product version data, Splunk uses the data to track how many deployments use particular versions of Splunk software offerings and to provide in-product notifications when updates are available. For apps, version data is correlated with information about app downloads to populate app analytics views on Splunkbase provided to the app's developer, and to compute the number of installs on the app details page.
How Splunk transmits and stores the data it collects
When you enable aggregated, support, and license usage data sharing, Splunk Enterprise runs searches to collect this data and sends the search summaries to a collection endpoint. Session data and Splunk software version data is not included in the searches. Session data is sent from your browser as the events are generated. Version data about Splunk Enterprise is sent to Splunk by your browser after you log into Splunk Web. Version data about your Splunk apps is sent to Splunk daily through a REST call from splunkd to splunkbase.splunk.com. Data is transmitted to Splunk from a single primary instance in your deployment. See Which instance runs the searches and sends data to Splunk.
Telemetry data is SSL encrypted before it leaves your deployment, and the certificates are verified before the data is securely stored in a Splunk Cloud instance. The Splunk Cloud instance used for customer telemetry has strict access controls that are subject to regular audit. For more information about how Splunk collects, uses, and discloses information about the data collected, see the Splunk Privacy Policy. For more information about Splunk's data privacy, security, and compliance practices, see Splunk Protects.
View what data is sent from your deployment
You can view aggregated usage, support usage, and license usage data that your deployment has recently sent in Splunk Web.
- Navigate to Settings > Instrumentation.
- Click the category of data you wish to view in Search.
This log is available only after the first run of the collection. To inspect the type of data that gets sent before you opt in on your production environment, you can opt in on your sandbox environment.
To view the browser session data, use JavaScript logging in your browser. Look for network events sent to a URL containing splkmobile
. Events are triggered by user actions such as navigating to a new page in Splunk Web.
To view version data that is sent for Splunk Enterprise, watch JavaScript network traffic as you log into Splunk Web. The data is sent inside a call to quickdraw.splunk.com.
How to opt out
Splunk collects support usage, aggregated usage, license data, and software version data by default. You can opt in or out at any time.
Prerequisite
To enable or disable collection of usage data, your user role must include the edit_telemetry_settings
capability.
Opt out of sharing aggregated or support usage data
To change your aggregated or support usage data sharing settings, follow these steps:
- Click Settings > Instrumentation in Splunk Web.
- Click the gear icon next to Usage Data.
- Adjust the sliders to enable or disable sharing aggregated or support usage data.
Opt out of sharing license data automatically
By default, Splunk collects license usage data based on your installed license to ensure compliance with your purchased offering. To disable sharing license data automatically, edit your local copy of telemetry.conf
file and set sendLicenseUsage = false
in the [general]
stanza, then restart Splunk Enterprise.
Certain license programs require that you report your license usage. The easiest way to do this is to automatically send this information to Splunk. If you disable automatic license data sharing, you can send license data manually. Follow these steps each time you want to send data manually:
- On a search head, log into Splunk Web.
- Select Settings > Instrumentation.
- Click Export.
- Select a date range and data type.
- Click Send to send data to Splunk directly or click Export to export the data to your local machine and send the data to Splunk using another mechanism.
Opt out of sharing software version data
To stop sending Splunk data about the version of Splunk Enterprise you have installed, set the value of the updateCheckerBaseURL
setting to 0
in your local copy of web.conf.
In addition, you can turn off version data sharing for each Splunk app. To disable notifications of new versions and stop sending Splunk data about the app version, set check_for_updates
to false
in the local copy of the app.conf
file for each app.
Opt out of sharing data and prevent future admins from opting in
To opt out from all collection of aggregated usage, support, and license data and prevent other admins from enabling it in the future, do the following on one search head in each cluster and on each nonclustered search head:
- Click Settings > Instrumentation in Splunk Web.
- Click the gear icon next to Usage Data.
- Disable all options.
- Click Settings > Access controls > Roles.
- Remove the
edit_telemetry_settings
capability from theadmin
role. Users with this role no longer receive notifications about data collection, nor can they access Settings > Instrumentation in Splunk Web.
If you want to disable collection of usage information across multiple deployments of the Splunk platform that are not centrally managed, block DNS resolution of e1345286.api.splkmobile.com
.
How to adjust your data collection schedule
If you share data, the collection process begins daily at 3:00 AM by default. You can change the frequency and timing of this collection.
If all instances in your deployment are running Splunk Enterprise version 7.1.0 or later, you can schedule instrumentation to run starting at any hour of the day on a daily or a weekly schedule. The collection process runs a few searches in sequence on several instances in your deployment. Depending on the size of your deployment and whether you run instrumentation daily or weekly, it can take a few minutes before the final searches run on the primary instance to package and send the data to Splunk. See Which instance runs the searches.
Changing the instrumentation collection schedule has trade-offs. Scheduling the collection to run weekly instead of daily might decrease the total search load for the week. A weekly collection takes longer than a daily collection, because it gathers data from all seven days. If you choose weekly collection, set it for a day and time when you expect the search load to be low.
Change the collection schedule using Splunk Web
- On a search head, in Splunk Web, navigate to Settings > Instrumentation.
- Next to Usage Data, click the gear icon.
- Click Edit usage data schedule.
- Select a frequency, day, and time.
- Click Save.
You do not need to restart the search head.
Change the collection schedule using configuration files
You can change the collection schedule by editing the telemetry.conf
file. For guidelines on editing this file, see telemetry.conf.
- At the command line on any search head, navigate to
$SPLUNK_HOME/etc/apps/splunk_instrumentation/local/
. - Create or edit
telemetry.conf
. - Edit the values for any of
scheduledHour
,scheduledDay
, andreportStartDate
according to the guidelines intelemetry.conf.spec
.
Performance impact
Aggregated usage, support usage, and license usage data is summarized and sent once per day at 3 AM by default. Splunk tested the performance impact on a deployment of one search head and three indexers and found the following performance impacts during the time that the searches were running:
- 4.5% increase in CPU overhead
- Negligible effects on memory, disk, and network overhead
- Up to 5% increase on the search time of regular search workloads
Session data and update checker data is sent from your browser as the events are generated. The performance implications are negligible.
Secure your configuration | How Splunk Enterprise licensing works |
This documentation applies to the following versions of Splunk® Enterprise: 8.0.8, 8.0.9, 8.0.10
Feedback submitted, thanks!