Docs » System limits for Splunk Infrastructure Monitoring

System limits for Splunk Infrastructure Monitoring ๐Ÿ”—

Splunk Infrastructure Monitoring has system limits that help ensure good performance, stability, and reliability. These limits also protect the Infrastructure Monitoring multi-tenant environment. Exceeding these limits might degrade your Infrastructure Monitoring experience.

To help you optimize your product experience, this topic describes the following:

  • The name and value of each system limit

  • If available, the organization metrics associated with the limit

  • The impact you observe when you exceed the limit.

The first section in this topic contains tables of limit names and values, organized by product area. Each entry in the table shows the name of the limit and its default value. The limit name is also a link to more information about the limit.

The first table in the section lists the most important limits to pay attention to as you scale your deployment.

Abbreviations ๐Ÿ”—

This documentation uses the following abbreviations:

  • MTS: Metric time series

  • AMTS: Active MTS

  • IMTS: Inactive MTS

  • ETS: Event time series

Limit summaries ๐Ÿ”—

The following tables summarize limits for these product areas:

Important limits ๐Ÿ”—

Limit name

Default limit value

AMTS limit

Determined by your subscription

Maximum number of detectors per organization

1,000

Maximum number of MTS allowed per chart data() function

  • 5,000 for standard subscriptions

  • 10,000 for enterprise subscriptions

Maximum number of MTS per detector data() function

  • 5,000 for standard subscriptions

  • 10,000 for enterprise subscriptions

MTS creations per minute limit

6,000 or determined by your subscription

Number of input MTS per job

250,000

Charts, detectors, and SignalFlow limits ๐Ÿ”—

Limit name

Default limit value

Maximum max delay setting for SignalFlow programs

15 min

Maximum min delay setting for SignalFlow programs

15 min

Maximum number of allocated datapoints per SignalFlow program

60,000,000

Maximum number of data(), graphite() or newrelic() functions per SignalFlow program

200

Maximum number of derived MTS per SignalFlow program

500,000

Maximum number of detectors per organization

1,000

Maximum number of functions and methods per SignalFlow program

1,000

Maximum number of MTS allowed per chart data() function

  • 5,000 for standard subscriptions

  • 10,000 for enterprise subscriptions

Maximum number of MTS analyzed across all SignalFlow programs

The larger of 10,000,000 AMTS or 20% of your total AMTS.

Maximum number of MTS per detector data() function

  • 5,000 for standard subscriptions

  • 10,000 for enterprise subscriptions

Maximum number of prefix wildcards per filter() function

150

Maximum number of query arguments in a filter() function

256

Maximum number of wildcards per filter() function

35

Maximum SignalFlow program stack size

64

Maximum SignalFlow program text size

50,000

Maximum SignalFlow programs per minute

1,000 SignalFlow programs per minute

Number of input MTS per job

250,000

Data ingestion limits ๐Ÿ”—

Limit name

Default limit value

New dimension or property key name limit

40 per week

Dimension/Metric value length

256

Events per minute

Determined by your subscription

Maximum dimension name length

128

MTS creations per hour limit

500,000 MTS per hour or 50 times your MTS per minute limit.

MTS creations per minute limit

6,000 or determined by your subscription

Number of dimensions per MTS

36

MTS metadata limits ๐Ÿ”—

Limit name

Default limit value

Maximum number of API calls per minute

100,000

Number of properties per dimension

75

Number of tags per dimension

50

Subscription limits ๐Ÿ”—

Limit name

Default limit value

AMTS limit

Determined by your subscription

Burst DPM limit

Multiples of entitlement

Bundled MTS limit

500,000

Container burst/overage limit

Multiples of entitlement

Container entitlement

Set by your contract entitlement

DPM limit

Set by your contract entitlement

Custom MTS burst/overage limit

Multiples of entitlement

Custom MTS entitlement

Set by your contract entitlement

High resolution custom metrics burst/overage limit

Multiples of entitlement

High resolution custom metrics entitlement

Set by your contract entitlement

Host burst/overage limit

Multiples of entitlement

Host entitlement

Contract entitlement

IMTS Limit

Determined by your subscription

Web app limits ๐Ÿ”—

Limit name

Default limit value

Dashboard group links per team

50

Detector links per team

700

Maximum number of dashboards you can retrieve

20,000

Maximum rendered MTS for area or stacked column visualizations

500

Maximum rendered MTS for column chart visualizations

20

Maximum rendered MTS for line, histogram, or heatmap visualizations

1,000

Other limits ๐Ÿ”—

Limit name

Default limit value

timeserieswindow API datapoint limit

1,000,000

Limit details ๐Ÿ”—

This section provides details about each Infrastructure Monitoring system limit.

Maximum number of dashboards you can retrieve ๐Ÿ”—

  • Default limit value: 20,000

  • Notes: Maximum number of dashboards you can retrieve per query using either the UI or the API. If you reach this limit, you receive an error.

  • Customer impact: When you exceed this limit, the user interface displays the error message โ€œUnexpected error has occurredโ€. After you exceed the limit, the dashboards page stops displaying dashboards.

Number of input MTS per job ๐Ÿ”—

  • Default limit value: 250,000

  • Notes: Maximum number of input MTS per job. When you use the same MTS multiple times in a job, each use counts towards the maximum.

  • Customer impact: If the job is for a chart, the chart doesnโ€™t load and you receive an error message. If the job is for a detector, the job is aborted. You can monitor aborted detector SignalFlow programs using a built-in metric. Your organization also receives an event with information about the detector that aborted. Your job might reach this limit after it starts. A chart might initially load, but fail when the limit is reached.

Maximum number of derived MTS per SignalFlow program ๐Ÿ”—

  • Default limit value: 500,000

  • Notes: Maximum number of derived MTS per SignalFlow program, where derived MTS are temporary MTS that a SignalFlow SignalFlow function or method has to maintain in memory. For example, if there are 20,000 MTS for the metric jvm.load, and each MTS comes from a unique host , then "data('jvm.load').sum(by=['host']).publish()" tracks 40,000 derived MTS. The data() SignalFlow function or method uses 20,000, and the sum() uses another 20,000. The number of input MTS is still 20,000.

  • Customer impact: If the SignalFlow program is for a chart, the chart doesnโ€™t load and you receive an error message. If the SignalFlow program is for a detector, the SignalFlow program is aborted. You can monitor aborted detector SignalFlow programs using a built-in metric. Your organization also receives an event with information about the detector that aborted. Your SignalFlow program might reach this limit after it starts. A chart might initially load, but fail when the limit is reached.

Maximum number of MTS allowed per chart data() function ๐Ÿ”—

  • Default limit value:

    • 5,000 for standard subscriptions

    • 10,000 for enterprise subscriptions

  • Notes: If youโ€™re using Enterprise Edition, this limit is 10,000. If you have auto-sharding enabled, you can have the limit set higher depending on your subscription. Please contact sales or customer support.

  • Customer impact: If you exceed the limit, only the most recently created MTS are kept, based on the creation timestamp maintained for each MTS. This might result in inaccurate computations.

Maximum number of MTS per detector data() function ๐Ÿ”—

  • Default limit value:

    • 5,000 for standard subscriptions

    • 10,000 for enterprise subscriptions

  • Notes: If youโ€™re using Enterprise Edition, this limit is 10,000. If you have auto-sharding enabled, you can have the limit set higher depending on your subscription. Please contact sales or customer support.

  • Customer impact: If you exceed the limit, only the most recently created MTS are kept, based on the creation timestamp maintained for each MTS. Detectors might not trigger, or they might trigger incorrectly.

Maximum number of allocated datapoints per SignalFlow program ๐Ÿ”—

  • Default limit value: 60,000,000

  • Notes: Total number of datapoints a SignalFlow program must buffer to satisfy time window transformations. This is at least the number of input MTS, but if the SignalFlow has a time window calculation, the actual value might be much more. For example, a sum over 1m at 1s resolution requires 60 datapoints per MTS. If the SignalFlow has 10,000 MTS and only one window transform, the SignalFlow needs 10,000*60=600,000 datapoints.

  • Customer impact: If the SignalFlow program is for a chart, the chart doesnโ€™t load and you receive an error message. If the SignalFlow program is for a detector, the SignalFlow program is aborted. You can monitor aborted detector SignalFlow programs using a built-in metric. Your organization also receives an event with information about the detector that aborted. Your SignalFlow program might reach this limit after it starts. A chart might initially load, but fail when the limit is reached.

Maximum number of functions and methods per SignalFlow program ๐Ÿ”—

  • Default limit value: 1,000

  • Notes: The SignalFlow program "A = data().sum(by="az").sum().publish()" has 4 functions and methods (data, sum, sum, publish).

  • Customer impact: SignalFlow programs which are violating the limit canโ€™t start. You immediately get an error message.

Maximum number of data(), graphite() or newrelic() functions per SignalFlow program ๐Ÿ”—

  • Default limit value: 200

  • Notes: Maximum number of data(), graphite() or newrelic() functions per SignalFlow program.

  • Customer impact: SignalFlow programs which are violating the limit canโ€™t start. You immediately get an error message. Note that this puts a limit on how many detect() you can use if you use different data(), graphite() or newrelic() methods in the detect().

Maximum SignalFlow program stack size ๐Ÿ”—

  • Default limit value: 64

  • Notes: A SignalFlow function canโ€™t recursively call itself more than this limit.

Maximum number of MTS analyzed across all SignalFlow programs ๐Ÿ”—

  • Default limit value: The larger of 10,000,000 AMTS or 20% of your total AMTS.

  • Notes: Maximum number of MTS that can concurrently use SignalFlow programs in your organization, including detector chart SignalFlow. For example, suppose you open 10 charts and keep them open. If each chart uses on average 5,000 MTS, youโ€™re using 50,000 MTS, even if each chart looks at the same 5,000 MTS. If you close the charts, your usage goes to zero. Detector SignalFlow programs are always running, so they always use a portion of your MTS usage limit. This limit only applies to streaming SignalFlow programs, not ones that look at historical data.

  • Customer impact: If the SignalFlow program is for a chart, the chart doesnโ€™t load and you receive an error message. If the SignalFlow program is for a detector, the SignalFlow program is aborted. You can monitor aborted detector SignalFlow programs using a built-in metric. Your organization also receives an event with information about the detector that aborted. Your SignalFlow program might reach this limit after it starts. A chart might initially load, but fail when the limit is reached.

Maximum max delay setting for SignalFlow programs ๐Ÿ”—

  • Default limit value: 15 min

  • Notes: The maximum allowed max delay value that you can set for a SignalFlow program. Higher values arenโ€™t allowed, because they cause SignalFlow programs to use too much memory when data is slow to arrive.

  • Customer impact: SignalFlow programs which are violating the limit canโ€™t start. You immediately get an error message.

Maximum min delay setting for SignalFlow programs ๐Ÿ”—

  • Default limit value: 15 min

  • Notes: The maximum allowed min delay value that you can set for a SignalFlow program. Higher values arenโ€™t allowed, because they cause SignalFlow programs to use too much memory when data is slow to arrive.

  • Customer impact: SignalFlow programs which are violating the limit canโ€™t start. You immediately get an error message.

Maximum number of wildcards per filter() function ๐Ÿ”—

  • Default limit value: 35

  • Notes: "data('jvm.load', filter=filter('host', 'kafka*east'))" counts as 1 wildcard filter

  • Customer impact: SignalFlow programs which are violating the limit canโ€™t start. You immediately get an error message.

Maximum number of prefix wildcards per filter() function ๐Ÿ”—

  • Default limit value: 150

  • Notes: "data('jvm.load', filter=filter('host', 'kafka*'))" counts as 1 prefix filter

  • Customer impact: SignalFlow programs which are violating the limit canโ€™t start. You immediately get an error message.

Maximum SignalFlow program text size ๐Ÿ”—

  • Default limit value: 50,000

  • Notes: Maximum character length of a SignalFlow program allowed in charts and detectors.

  • Customer impact: SignalFlow programs which are violating the limit canโ€™t be saved. You immediately get an error message.

Maximum SignalFlow programs per minute ๐Ÿ”—

  • Default limit value: 1,000 SignalFlow programs per minute

  • Notes: Maximum number of SignalFlow programs started per minute

  • Related metrics:

    • sf.org.computations.started

    • sf.org.computations.throttled

  • Customer impact: SignalFlow programs which are violating the limit canโ€™t start. You immediately get an error message.

Maximum number of query arguments in a filter() function ๐Ÿ”—

  • Default limit value: 256

  • Notes: Limit to the number of query arguments in a SignalFlow filter

  • Customer impact: Maximum number of derived MTS per SignalFlow program, where derived MTS are temporary MTS that a SignalFlow function or method has to maintain in memory. For example, if there are 20,000 MTS for the metric jvm.load, and each MTS comes from a unique host , then "data('jvm.load').sum(by=['host']).publish()" tracks 40,000 derived MTS. The data() function uses 20,000, and the sum() uses another 20,000. The number of input MTS is still 20,000.

Maximum number of detectors per organization ๐Ÿ”—

  • Default limit value: 1,000

  • Notes: The maximum number of detectors that you can create in a single organization.

  • Related metrics:

    • sf.org.limit.detector

    • sf.org.num.detector

  • Customer impact: The user interface displays an error reporting that the limit has been exceeded.

New dimension or property key name limit ๐Ÿ”—

  • Default limit value: 40 per week

  • Notes: The maximum number of new custom fields (property or dimension keys) you can create, per organization per week. This limit applies to MTS and ETS. For example, host:1 and host:2 are considered to have 1 key which is host. foo:1 and bar:1 are considered to have two keys: foo and bar.

  • Related metrics:

    • sf.org.numPropertyLimitedMetricTimeSeriesCreateCalls

    • sf.org.numPropertyLimitedMetricTimeSeriesCreateCallsByToken

  • Customer impact: MTS creations above the limit are rejected, and no error message appears.

Events per minute ๐Ÿ”—

  • Default limit value: Determined by your subscription

  • Notes: Maximum number of custom events youโ€™re allowed ingest per minute

  • Customer impact: If you have this limit set for an org token, you will receive a HTTP 429 error from Data Ingestion APIs when you exceed the limit.

MTS creations per minute limit ๐Ÿ”—

  • Default limit value: 6,000 or determined by your subscription

  • Notes: Maximum number of MTS you can create per minute

  • Related metrics:

    • sf.org.numMetricTimeSeriesCreated

    • sf.org.limit.metricTimeSeriesCreatedPerMinute

  • Customer impact: Infrastructure Monitoring drops new MTS above the limit without returning an error.

MTS creations per hour limit ๐Ÿ”—

  • Default limit value: 500,000 MTS per hour or 50 times your MTS per minute limit.

  • Notes: If you exceed your MTS limit per minute for more than 30 minutes of any hour, you canโ€™t create any more MTS for the remainder of the hour.

  • Customer impact: Infrastructure Monitoring drops new MTS above the limit without returning an error. Data points for existing MTS are still accepted.

Number of dimensions per MTS ๐Ÿ”—

  • Default limit value: 36

  • Notes: Maximum number of dimensions per MTS. Infrastructure Monitoring silently drops invalid data points, but valid data points in the same request are kept.

  • Customer impact: Infrastructure Monitoring accepts valid data points but drops invalid data points. For invalid data points, Infrastructure Monitoring doesnโ€™t send an error message.

Dimension/Metric value length ๐Ÿ”—

  • Default limit value: 256

  • Notes: No error is returned. Invalid data points are silently dropped. Valid data points in the same request are let in.

  • Customer impact: No error is returned. Invalid data points are silently dropped. Valid data points in the same request are let in.

Maximum dimension name length ๐Ÿ”—

  • Default limit value: 128

  • Notes: Maximum number of dimensions per MTS. Infrastructure Monitoring silently drops invalid data points, but valid data points in the same request are kept.

  • Customer impact: Infrastructure Monitoring accepts valid data points but drops invalid data points. For invalid data points, Infrastructure Monitoring doesnโ€™t send an error message.

Maximum number of API calls per minute ๐Ÿ”—

  • Default limit value: 100,000

  • Notes: Maximum number of REST API calls you can make per endpoint per minute. The limit for GET calls is 10 times the rate for other calls. The limit protects the system from gross misuse or attacks. This applies to metadata API to api.signalfx.com

  • Related metrics: sf.org.numRestCalls

  • Customer impact: The API returns an HTTP error code 429 that indicates that youโ€™ve reached your API call limit.

Number of tags per dimension ๐Ÿ”—

  • Default limit value: 50

  • Notes: Maximum number of tags per dimension. Infrastructure Monitoring silently drops excess tags.

  • Customer impact: Infrastructure Monitoring drops tags that exceed the limit but doesnโ€™t issue an error message.

Number of properties per dimension ๐Ÿ”—

  • Default limit value: 75

  • Notes: Maximum number of custom properties per dimension. Infrastructure Monitoring silently drops excess properties.

  • Customer impact: Infrastructure Monitoring drops properties that exceed the limit but doesnโ€™t issue an error message.

timeserieswindow API datapoint limit ๐Ÿ”—

  • Default limit value: 1,000,000

  • Notes: The maximum number of datapoints you can retrieve in a single call to GET /v2/timeserieswindow.

  • Customer impact: The request fails and returns an HTTP error code 400

Custom MTS entitlement ๐Ÿ”—

  • Default limit value: Set by your contract entitlement

  • Notes: Number of custom MTS entitled, as determined by your contract.

  • Related metrics: sf.org.numCustomMetrics

  • Customer impact: Splunk charges an overage of 1.5 times the normal price for usage above contractual entitlement.

Custom MTS burst/overage limit ๐Ÿ”—

  • Default limit value: Multiples of entitlement

  • Notes: Maximum number of active custom MTS, within a moving window of the previous 60 minutes, that youโ€™re allowed to have in your organization.

  • Related metrics:

    • sf.org.numCustomMetrics

    • sf.org.limit.customMetricTimeSeries

  • Customer impact: If you exceed this limit, Infrastructure Monitoring stops accepting data points for new custom MTS, but it continues to accept data points for custom MTS that already existed.

Host entitlement ๐Ÿ”—

  • Default limit value: Contract entitlement

  • Notes: Number of hosts in your contract, if applicable.

  • Related metrics: sf.org.numResourcesMonitored

  • Customer impact: Splunk charges an overage of 1.5 times the normal price for usage above contractual entitlement.

Host burst/overage limit ๐Ÿ”—

  • Default limit value: Multiples of entitlement

  • Notes: For host-based pricing contracts, the maximum number of hosts that can send data to your organization. This limit is higher than your contractual limit to allow for burst and overage usage.

  • Related metrics:

    • sf.org.numResourcesMonitored (filter for the dimension resourceId:hosts)

    • sf.org.limit.hosts

  • Customer impact: If you exceed this limit, Infrastructure Monitoring drops data points from new hosts but keeps accepting data points for existing hosts.

Container entitlement ๐Ÿ”—

  • Default limit value: Set by your contract entitlement

  • Notes: Number of containers in your contract, if applicable

  • Related metrics: sf.org.numResourcesMonitored

  • Customer impact: Splunk charges an overage of 1.5 times the normal price for usage above contractual entitlement.

Container burst/overage limit ๐Ÿ”—

  • Default limit value: Multiples of entitlement

  • Notes: For host-based pricing contracts, maximum number of containers that can send data to your organization. This limit is higher than your contractual limit to allow for burst and overage usage.

  • Related metrics:

    • sf.org.numResourcesMonitored (filter for the dimension resourceId:containers)

    • sf.org.limit.containers

  • Customer impact: If you exceed this limit, Infrastructure Monitoring drops data points from new containers but keeps accepting data points for existing containers.

High resolution custom metrics entitlement ๐Ÿ”—

  • Default limit value: Set by your contract entitlement

  • Notes: Number of high resolution metrics allowed in your contract

  • Customer impact: Splunk charges an overage of 1.5 times the normal price for usage above contractual entitlement.

High resolution custom metrics burst/overage limit ๐Ÿ”—

  • Default limit value: Multiples of entitlement

  • Notes: This limit is to protect the SaaS platform. Itโ€™s typically a multiple of your contractual limit. For example, if you purchase 500 hosts, Infrastructure Monitoring might set limit to 800. The multiple decreases as your contractual limit increases.

  • Customer impact: MTS creations for high resolution metrics above the limit are rejected.

Bundled MTS limit ๐Ÿ”—

  • Default limit value: 500,000

  • Notes: This limit applies to host based contracts only. It is the total number of bundled MTS a customer could has, beyond the standard host based or container based MTS. MTS for SQS queues is an example.

  • Customer impact: MTS creations above the limit are rejected, and no error message appears.

IMTS Limit ๐Ÿ”—

  • Default limit value: Determined by your subscription

  • Notes: Maximum number of inactive MTS, as allowed by your contract.

  • Related metrics: sf.org.numInactiveTimeSeries

  • Customer impact: When you reach this limit, the system deletes the inactive MTS that have been inactive the longest.

AMTS limit ๐Ÿ”—

  • Default limit value: Determined by your subscription

  • Notes: Maximum number of active MTS in a 25 hour period. If youโ€™re using Kubernetes, the period is 1 hour.

  • Related metrics:

    • sf.org.numActiveTimeSeries

    • sf.org.limit.activeTimeSeries

  • Customer impact: When you exceed this limit, Infrastructure Monitoring refuses new MTS without issuing an error message. Infrastructure Monitoring continues to ingest data points for existing MTS.

DPM limit ๐Ÿ”—

  • Default limit value: Set by your contract entitlement

  • Notes: Limit on the number of datapoints you can send to Infrastructure Monitoring per minute. If you exceed the limit, Infrastructure Monitoring stops creating new MTS and rejects the data points.

  • Customer impact: Infrastructure Monitoring drops new data points and MTS above the limit without returning an error.

Burst DPM limit ๐Ÿ”—

  • Default limit value: Multiples of entitlement

  • Notes: Limit on the number of datapoints you can send to Infrastructure Monitoring each minute. If you have this limit set on the organization token, the data ingest API returns HTTP response code 429 when you exceed the limit.

  • Customer impact: If you have this limit set for an org token, you will receive a HTTP 429 error from Data Ingestion APIs when you exceed the limit.

Maximum rendered MTS for line, histogram, or heatmap visualizations ๐Ÿ”—

  • Default limit value: 1,000

  • Notes: When a visualization exceeds the limit, the UI arbitrarily selects the MTS it renders.

Maximum rendered MTS for area or stacked column visualizations ๐Ÿ”—

  • Default limit value: 500

  • Notes: When a visualization exceeds the limit, the UI arbitrarily selects the MTS it renders.

Maximum rendered MTS for column chart visualizations ๐Ÿ”—

  • Default limit value: 20

  • Notes: When a visualization exceeds the limit, the UI arbitrarily selects the MTS it renders.